The following requirements and criteria are used to assess the minimum expected level of quality in an Almond release. A violation of these criteria is a blocker issue that might delay the release. Whether an issue fits one of these criteria or not is evaluated by the project maintainers.
There are three levels of blocking:
At all time, the tests in the main branch must be green in Travis.
Blocks: merging
At release time, applicable builds must succeed for the supported modules. At the time of writing, the blocking builds are:
Blocks: all releases (Alpha, Beta, RC, and Final)
It must be possible to configure any of the supported Almond apps and complete the basic configuration steps, following the documented setup step. For almond-cloud, this involves creating a new account on the website. For other platforms, this includes downloading and installing Almond on the local device. At the time of writing, the supported Almond apps are:
This criterion explicitly excludes installing a custom almond-cloud, or installing genie-toolkit. Those are part of the advanced setup criterion.
Blocks: all releases (Alpha, Beta, RC, and Final)
For each of the skills in the main release set, it is expected that basic functionality is available.
Basic functionality includes simple commands with the minimum numbers of parameters necessary for the skill to function, using common, expected wording. The command must be understood from natural language. The skill must not crash or return an unexpected error. The agent must slot fill any required parameters, and then return a readable and factually correct answer.
Basic functionality includes only correct operation of the API, and excludes any platform-dependent behavior. An example of basic functionality is "get weather" and "get weather for San Francisco". An example of platform-dependent behavior is whether the Spotify skill plays music on the Web Player or requires a local running Spotify app.
This criteria is only applicable for correctly configured skills.
Blocks: all releases (Alpha, Beta, RC, and Final)
For each of the skills in the main release set, it is expected that the skill can be configured from the applicable configuration page in Almond. Configuration must succeed across all supported Almond apps, with no additional user intervention, and without modifying the default Almond configuration.
Blocks: all releases (Alpha, Beta, RC, and Final)
It must be possible to install a customized Almond, following the instructions in the documentation. This includes a customized almond-cloud, both for development and for production setup, as well as a customized genie-toolkit. It must be possible to point a local version of Almond to customized libraries for development.
This criterion is satisfied even if tweaking of local configuration files or a specific development environment is necessary, as long as those requirements are documented in the installation guides.
Blocks: Final release
For each of the skills in the main release set, it is expected that full functionality is available, as long as the commands are interpreted correctly from natural language. Full functionality includes all features that can be expressed with the current version of ThingTalk, for the declared APIs in the manifest. The skill must not crash or return an unexpected error. It must return a result consistent with the semantics of the ThingTalk command. The dialogue agent must issue any relevant search refinement or slot filling question, and then describe the result in a readable and factually correct manner.
Blocks: Final release
We expect our main model (covering main release set and the universe release set) will have the following accuracy:
Blocks: deployment of the new model
Blocks: Final release
A reported failure to install or operate in a supported configuration blocks a Final release. Not all configuration are tested at all releases.
OS:
HW requirements:
OS:
HW requirements:
OS:
HW requirements: