Thingpedia is the open repository of virtual assistant skills used by Almond. Just like how Wikipedia stores knowledge about the world, Thingpedia stores knowledge about devices and services in the world. Wikipedia is organized around articles; Thingpedia is organized around devices, such as Twitter, a light bulb, or a thermostat. (We use the term device to refer to both IoT devices and Internet services.). Specifically, Thingpedia is organized in device classes, such as Spotify, Twitter, Lights, or Yelp, and device instances, corresponding to a specific account on Spotify/Twitter/etc. The Thingpedia repository host the information of each device class, while each Almond contains the credentials to access a device instance.
A device in Thingpedia can have two types of functions: query and action. Queries are free of side-effects and can return results, and actions have side effects, and cannot return results. Users can connect different queries and actions to accomplish complicated tasks with a single command, such as "Get my latest Instagram picture and post it on Twitter".
Any Almond virtual assistant has two major components: the Almond agent communicates with the users in natural language, and the Almond engine runs the commands issued by the users. Both the Almond agent and the Almond engine are part of the Genie Toolkit.
The following diagram illustrates how Almond works internally:
A user issues a command, say “get my recent tweets”, in English.
Almond agent translates the command to the corresponding ThingTalk program and send it to Almond engine.
If the user has Twitter set up in his Almond, go to step 6. Otherwise, Almond engine requests the Twitter interface from Thingpedia.
Thingpedia returns the Twitter interface, and Almond engine run the ThingTalk program with the interface returned.
If needed, Almond prompts to the user to do authentication. For Twitter, user will be redirected to OAuth page to link his account to Almond.
Then Almond engine will call the Twitter API and get the result from Twitter.
The result will be reported to Almond agent.
Almond agent will send the result to the user.
Thingpedia is an open repository, and anyone can contribute their favorite skill. We have prepared some resources so you can learn how to write Thingpedia devices.
We recommend newcomers start with the first tutorial, which will give a fully hands-on experience. You can then proceed through the Guide, or complete more tutorials to see more and more complex Thingpedia devices.
You can also find the skills we developed ourselves in our thingpedia-common-devices Git repository. See the documentation there for how to set up the developer environment and build a custom agent.
To use existing Thingpedia devices in a custom voice assistants, you should use the Genie Toolkit. Genie is all-in-one solution that takes care of training the natural language understanding model associated with the Thingpedia devices you are interested in. Genie also provides an implementation of the dialogue agent and the execution engine. Both are necessary to execute commands provided by Thingpedia devices. Check out the Genie Guide To Training Natural Language Models and the Genie Guide To Custom Assistants.
If you do not need a full-fledged voice assistant, and are only interested in the uniform APIs provided by Thingpedia devices, a lower-level interface is provided by the Thingpedia SDK. The Thingpedia SDK allows to instantiate and control Thingpedia devices directly. The SDK interfaces with the Thingpedia REST API. The API can be queried directly to search for devices in Thingpedia and download them.