Basically, Pepper's application development can be done with a style interface that connects "boxes" on a visual programming environment called Choregraphe, and you can develop applications relatively easily. Developers can select the required boxes from the "box library" and connect them to define the behavior of the application ("behavior").

Some behaviors can be achieved with the standard box libraries, but these box libraries do not necessarily provide boxes comprehensively, and you have to customize the box implementation (Python script, etc.). The reality is that there are quite a few cases where you have to. For these parts, I wonder if it is realistic for box libraries to be developed on a community basis.

Then I think I'll make a box library, but of course I don't want to have a bug in the box and have trouble in the later phase. However, I can't officially find anything that seems to be a box test framework ... With that kind of feeling, I'm trying to make a prototype while thinking about how to test Pepper's box library and improve the quality, so that memo.

Take a quick look

Box test behavior?

Looking at tests in other programming languages, such as Python, it seems that the input, output, and parameter formats of boxes can be replaced with Python "classes" as they are.

For example, in Python, a test case describes whether (an instance of) the class under test behaves as intended. In this test case, write the code that actually inputs the value to the instance to be tested and checks whether the output value is the same as the expected value. In Choregraphe, it seems natural to prepare a test behavior in the same way and connect the box to be tested and the box for checking the result. For example, when testing the Wait box, make sure that the Wait box outputs n seconds after receiving input, and that it does not output before n seconds.

Relationship between box library and box?

Obviously, the test must be done on the exact same box in the box library. Testing against a different version of the box from the box library does not mean that the box in the box library is valid.

As I wrote a little in Pepper's file exploration (project file), the box dragged and dropped from the box library to the behavior is copied during the behavior. It seems that it is arranged in the form of being. In this case, even if you create a test behavior, the box to be tested there may be different from the one in the box library, and there is a risk that it will not be able to fully fulfill its role as a test. I will come.

In order to maintain the identity of the box implementation like this, it seems that it is necessary to be able to somehow identify the relationship between the box in the behavior and the box library of the drag and drop source. Looking at the format of the behavior definition file, I can't find any information that can uniquely identify the box ... For example, in the latest document, Improved boxes of updates in Choregraphe 2.0.5,

Here is the list of boxes you should cut/replace:

Or (a nuance that can be replaced by hand?),

Now these boxes call an ALMotion method, so they will be updated automatically by new releases.

There is also a description like (It will be updated automatically (without replacing the box) because it is a change in the API implementation), so if there is a change in the box in the box library, the behavior side will be automatically changed. I feel that there is no official mechanism to follow it.

Challenges to think about

That's why I'll consider the following two things.

How to replace (commonize) a box in a behavior with a box in a box library
How to make a test behavior

So, first of all, let's think about how to maintain the identity of (a copy of) the box.

Trial: Replace the box in the behavior with the one in the box library

I made a prototype of an automatic replacement tool. The code is below.

https://github.com/yacchin1205/choregraphe-box-util

I have only tested it on Linux, but it should be installed by cloning it properly and doing pip install -r requirements.txt; python setup.py install. maybe. The code is written in Python. It's my first time to write an independent tool in Python, so I learned a lot ...

In addition, lxml is used for XML processing. This is because the XML file of the box uses the CDATA section, and the Python standard ElementTree did not output the CDATA section well ...

As for how to use

$ replace-boxes -l (Box library path(Only directories are supported)) (Of the project.Path to pml file)

For example, regarding the behavior in the specified project

Search the specified box library for a box that matches the box in the behavior.
Replace the box in the behavior with the one in the box library (if any)

In this way, the idea is to maintain the identity of the boxes in the behavior and the boxes in the box library. It's a tool I'm still using as a test, so be sure to back up your project before trying it out. If you just want to see how the conversion looks, you can add the --dry-run -v option.

Below are the details of what we are doing ...

Box matching

First, it is necessary to match the correspondence between the box library and the behavior side of each box. This time we are looking at whether the following elements match.

Box name
Number of inputs / outputs, each name, ID, type
Number of parameters, each name, ID, type

The box name is cumbersome to process. Box names do not have namespaces such as "Say" and "Basic Awareness", so multiple box libraries with the same name may be duplicated. Also, if multiple boxes are placed in the same flow, numbers indicating copies such as "Say (1)" and "Say (2)" will be assigned, which is even more complicated.

Box identification tag

For box name matching, matching is performed assuming a format such as "box name (n)", but this alone does not resolve the conflict of box names provided by different box libraries.

Therefore, by adding the following annotations to the explanation of the box, I try to see if this also matches.

@source https://github.com/yacchin1205/choregraphe-box-util

Looking at the value of this @ source, I am trying to determine if it comes from the same box library. If you want to remove this judgment, you can add the --ignore-tags option.

Replacement process

When replacing, it basically copies the boxes in the matched box library into the behavior. This will replace the box script, description, etc. The exception is the value of the Parameter, which is supposed to inherit the one in the source box.

In addition, plugin type boxes such as Text Edit that display a text box on Choregraphe are not targeted for conversion (this plugin seems to rewrite the script itself in the box, so it is a good replacement method. I can't think of ...) Also, replacement is not supported when a resource (audio file, etc.) is attached to the box.

That's why, although it's limited, it seems that we can maintain the identity of the box between the behavior and the box library, so let's create a test behavior.

Trial: Test behavior

It's hard to make it comprehensively suddenly, so of the box libraries I started making with I tried HTML5 TEST with Pepper, the theme is Show URL box. I decided to make a simple test behavior.

https://github.com/yacchin1205/pepper-web-boxes

Testing behaviors can be found at tests / test-show-url.

Test of success

The test for the success of the Show URL looks like this.

Roughly, it consists of the following two flows.

Confirm that the tablet is in the CONNECTED state in the Enable WiFi box and Get WiFi status box (both are unique boxes).
Run the Show URL box to see if the onSuccess output fires within a certain amount of time.

The point is 2. The Wait box under the Show URL tells the log that if the onSuccess output is not fired within 10 seconds from the start (if nothing is input to the onInput input of Assert Bang), it is an error. It is designed to be output. In addition, when the onFailure output is input to the Failure box, it is also output to the log as an immediate error treatment.

Simply run this behavior to see if the Show URL box is behaving as intended, with an error message.

Test of failure

Conversely, the test also states that running Show URL after Disable WiFi will result in an error.

Here, after executing Disable WiFi, execute Show URL and confirm that the onFailure output is executed by the Assert Bang box. If onSuccess is output unintentionally, the Failure box clearly indicates that it is an error. This makes it possible to check whether the processing in the situation where the box does not work is as intended.

The Show URL boxes in these tests are described as separate boxes during the behavior, but the replace-boxes command above can be used to maintain identity with the box library. If you make a mistake in the Show URL box, you can rewrite the box in the test behavior and test again.

Summary

With that in mind, I wrote a tool to maintain the identity of the box and a test behavior to see if the box behaves as expected.

Personally, I like Choregraphe's choreography very much, so I think I can develop such auxiliary tools from time to time.

I'd like to see a document generation tool for the box library, too ... I wonder if the box library attached to Choregraphe doesn't have any documents.

[PYTHON] Pepper box library test consideration