[Back to Implement a Genie Skill Backend]
Use:
make lint
To check all devices statically for errors.
Add unit tests in a file called test/unit/$YOUR_DEVICE_NAME.js
, which should export a list of test cases. Each test case
should be a list containing:
query
or action
)Some examples of test files can be found here.
If your device needs authentication, add a file $YOUR_DEVICE_NAME.cred.json
containing
the information needed in the directory test/
. All the information can be accessed from the state
parameter of your device's constructor
in JS.
An example can be found here
Use:
node ./test/unit $YOUR_DEVICE_NAME
to run the unit tests.
Scenario tests will load your device in a complete assistant, and test end-to-end that the assistant responds correctly to the user's commands. It is a way to catch regressions and unexpected failures.
At first, you'll write scenario tests using \t
commands, which emulates the user typing ThingTalk code in their chat window directly. You can use these tests to check that your skill is returning data compatible with the function signatures declared in the manifest and that the agent replies correctly. Later, once a model has been trained for the skill, the user commands can be replaced with natural language commands, to act as an end-to-end regression test.
To add a scenario test, add a new dialogue in the eval/scenarios.txt
file in the device folder. Dialogues are separated by ====
(4 equal signs). The format of a dialogue alternates user turns, prefixed with U:
, and agent turns, prefixed with A:
. The user speaks first, and the agent speaks last. The first line in a dialogue that starts with #
contains the ID of the test, and the rest of #
lines are comments.
At every turn, the system emulates inputs with the given user utterance, then checks
that the reply from the agent matches the regular expression indicated in the agent's turn.
See thingpedia-common-devices for examples for each of the devices.
Use:
node ./test/scenarios $YOUR_DEVICE_NAME
to run scenario tests.
[Back to Implement a Genie Skill Backend]