This page contains the list of all the annotations that are used by Genie to construct training data and build the dialogue agents.
Annotations that are used by the Thingpedia SDK to provide the implementation of Thingpedia devices are described in the Thingpedia Annotation Reference.
Anywhere an annotation is described as taking an array of strings, a single string is also accepted.
Canonical forms are the minimal natural language annotations used to build dialogues.
A canonical description of the Thingpedia device, used to augment commands with the device name.
Syntax: #_[canonical=$form]
Scope: class
Type: string
Grammar Category: proper noun
Restrictions: must be lowercase, no punctuation
Example:
class @com.spotify
#_[canonical="spotify"] {
...
One or more phrases used to refer to a Thingpedia function.
Syntax: #_[canonical=[$form1, $form2, ...]]
Scope: function
Type: array of strings
Grammar Category: singular noun phrase for query, imperative verb phrase for action
Restrictions: must be lowercase, no punctuation
Example:
query song(...)
#_[canonical=["song", "music", "track"]];
action play_song(...)
#_[canonical=["play a song"]];
...
Phrases used to refer to a single parameter of a function.
Values are strings where we use #
to indicate where the actual value should be placed. The grammar types indicate how the values will be used in a sentence.
Syntax:
#_[canonical = {
$type_1: [$value_1_1, $value_1_2, ... ],
$type_2: [$value_2_1, $value_2_2, ... ],
...
}]
Scope: parameter
Type: object
Restrictions: must be lowercase, no punctuation, can use #
to refer to a parameter value
Base form (base
)
The base form of a parameter is a noun phrase used for describing the parameter without context. In particular, it is supposed to be used without a value round it. This includes projection questions. A projection question asks about one parameter of the subject, such as "what is the address of a restaurant?". The base form is also used for questions related to the count of a parameter. For example, we can ask "how many college degrees does Bob have?", "show me a person with 3 awards."
out alumniOf: Entity(org.schema:Organization)
#_[canonical = {
base = ["college degree"]
}],
// projection
"what college degree does Bob have?"
// count
"how many college degrees does Bob have?"
"show me a person with 3 or more college degrees."
Property (Has-a Noun Phrase) (property
)
This describes what the subject has, i.e., a property that the subject owns. If property
annotation is missing, we will use base form
instead.
out alumniOf: Entity(org.schema:Organization)
#_[canonical = {
property = ["college degree from #", "# degree"]
}],
"Show me persons with college degree from Stanford."
"Show me persons with a Stanford degree"
Reverse property (Is-a Noun Phrase) (reverse_property
)
This describes what the subject is, i.e., the identity of the subject. It can also be considered as a reversed property where the subject is a property of the value. For example, worksFor
has reverse property annotation "employee" - Bob works for Stanford, and Stanford has "employee" Bob.
out alumniOf: Entity(org.schema:Organization)
#_[canonical = {
reverse_property = ["# grads", "# alumni"]
}]
"show me Stanford grads."
"who are Stanford Alumni?"
Active (verb
)
This describes the property in the form of an active verb phrase
out alumniOf: Entity(org.schema:Organization)
#_[canonical = {
verb = ["went to #", "graduated from #"]
}]
"Who went to Stanford University?"
"Show me a person that graduated from Stanford."
Passive (passive_verb
)
This describes the property in the form of a passive verb phrase.
out worksFor: Entity(org.schema:Organization)
#_[canonical = {
passive_verb = ["employed by #"]
}]
"Who is employed by Stanford University?"
"Show me a person employed from Stanford."
Adjective (adjective
)
The adjective form of a parameter is used in front of the subject as an adjective. In the paper, this is only a boolean flag. Now we extend it to also take values.
out rating: Number
#_[canonical = {
adjective = ["#-star"]
}]
out servesCuisien: String
#_[canonical = {
adjective = ["#"]
}]
"Show me a 5-star restaurant"
"Show me a Chinese restaurant"
Preposition (preposition
)
out location: Location
#_[canonical = {
preposition = ["at", "in"]
"Show me a restaurant in Palo Alto"
Base form (base_projection
)
The projection base annotation is a noun phrase used to compose the questions together with other projection annotations.
It could be the name of the property or the possible entity types of the property.
For example, for sister_city
property in the city domain, we would have "city" as a base_projection
, so that we can form a sentence like
"which city is Palo Alto's sister city?".
Note that this is different from the base
form, as the base
form should be informative enough to identify the property by itself.
However, this is unnecessarily true for base_projection
, since it is a helper noun phrase that is supposed to be used together with other projection annotations.
When absent, we will use base
annotation as a backup.
Projection annotations in different POS (verb_projection
, passive_verb_projection
, preposition_projection
, reverse_verb_projection
, property_projection
, reverse_property_projection
)
These annotations describes how a property can be used in projections. They are often the same as their corresponding non-projection version, with value place holder removed.
reverse_verb_projection
is used when the subject of the verb is the property value.
For example, inAlbum
property in a music domain, might have reverse_verb_projection
canonical "contains",
with which we can construct question "what album contains the song?"; and we can also have verb_projection
canonical "appear in",
with which we can construct question "what album does the song appear in?"
You can add a divider |
if the canonical ends with a prepositional phrase.
For example the verb phrase "appear in" can be annotated as "appear | in". In which case
we will generate sentences like "In what album does the song appear?".
out inAlbum: Entity(org.schema.Music:MusicAlbum)
#_[canonical = {
verb_projection = ['appear | in', 'appear | on'],
reverse_verb_projection = ['have', 'has', 'contain', 'contains', 'includes'],
passive_verb_projection = ['included | in', 'included | on'],
}]
"what album does the song appear in?"
"what album contains the song? "
"what album is the song included?"
Similarly, reverse_property_projection
is used when the subject is a "property" of the parameter value, and property_projection
is used when the parameter value is a property of the subject. Check below for an example for each.
out capital: Entity(org.wikidata:p_capital)
#_[canonical = {
base_projection = ["city"]
property_projection = ["capital"]
}]
"which city is the capital of Japan?"
out capital_of: Entity(org.wikidata:p_capital_of)
#_[canonical = {
base_projection = ["country", "state"]
reverse_property_projection = ["a captial of"]
}]
"which country/state is Tokyo a captial of?"
adjective_argmin
, adjective_argmax
This describe how a single adjective phrase can be used to get the argmin/argmax.
out rating: Number
#_[canonical = {
adjective_argmin = ["worst"],
adjective_argmax = ["best", "best-rated"]
}]
"Show me the best restaurants nearby"
These are the annotations used to construct multi-turn dialogues
One or more questions from the agent used to refine a search or fill a missing parameter.
Syntax: #_[prompt=[$question1, $question2, ...]]
Scope: parameter
Type: array of strings
Grammar Category: question
Restrictions: must be lowercase, no punctuation, must not include "?"
Example:
class @org.thingpedia.weather {
query current(in opt location : Location
#_[prompt=["what location would you like the weather for"]],
...);
}
A question the user can ask about the current result returned by the user.
Syntax: #_[question=[$question1, $question2, ...]]
Scope: parameter
Type: array of strings
Grammar Category: question
Restrictions: must be lowercase, no punctuation, must not include "?"
Example:
class @com.spotify {
query song(...,
out popularity : Number
#_[question=["is that a popular song"]);
}
A phrase that the agent uses to describe the result of a query of an action.
Syntax: #_[result=[$question1, $question2, ...]]
Scope: function
Type: array of strings
Grammar Category: declarative sentence
Restrictions: must be lowercase, can use punctuation and placeholders for both input and output parameters
Example:
class @org.thingpedia.weather {
query current(in opt location : Location,
out status : Enum(raining,cloudy,sunny,snowy,sleety,drizzling,windy))
#_[result=["the current weather in ${location} is ${status}";
}
A phrase that the agent uses to describe an error condition that arose when invoking a Thingpedia function.
Syntax:
#_[on_error={
$code1: [$phrase1, $phrase2, ...],
$code2: [$phrase3, $phrase4, ...],
}]
Scope: function
Type: object
Grammar Category: declarative sentence
Restrictions: must be lowercase, can use punctuation and placeholders for input parameters only
Example:
query make_reservation(in req restaurant : Entity(...),
...)
#_[on_error={
no_slot_available=["there are no tables available at ${restaurant}",
"${restaurant} has no available tables"]
}];
The following annotations provide Genie with refined knowledge of the types used by Thingpedia functions.
A reference to a dataset of string values to provide representative samples of values for the parameter.
Syntax: #[string_values=$ref]
Scope: parameters of type String, Entity, or Array of those types
Type: reference to a string dataset
Example:
class @com.spotify {
query song(out id : Entity(com.spotify:song)
#[string_values="tt:song_names"],
...);
The minimum acceptable value for a numeric parameter. This is only a hint to the agent and it is not enforced at runtime.
Syntax: #[min_number=$value]
Scope: parameters of type Number, Measure or Currency
Type: a constant value of the same type as the parameter
Example:
class @com.yelp {
query restaurant(out rating : Number
#[min_number=1],
...);
The maximum acceptable value for a numeric parameter. This is only a hint to the agent and it is not enforced at runtime.
Syntax: #[max_number=$value]
Scope: parameters of type Number, Measure or Currency
Type: a constant value of the same type as the parameter
Example:
class @com.yelp {
query restaurant(out rating : Number
#[max_number=5],
...);
The implied value for an omitted optional input parameter. Any sentence that explicitly refers to the default value will have the value removed from the program. The value will be added to the program explicitly before execution.
Syntax: #[default=$value]
Scope: optional input parameters
Type: a value of the same type as the parameter (need not be a constant)
Example:
class @org.thingpedia.weather {
query current(in opt location : Location
#[default=$context.location.current_location]);
The following annotations specify non-type-based constraints on how parameters should be used.
If this annotation is specified and true
, the query must return a unique value of the parameter for each row.
This annotation is equivalent to the
UNIQUE
clause in SQL.
Most likely,
id
parameters should be annotated with#[unique=true]
.
Syntax: #[unique=true]
Scope: output parameters
Type: boolean
If unspecified: false
Example:
class @com.yelp {
query restaurant(out id : Entity(com.yelp:restaurant)
#[unique=true],
...);
If this annotation and specified and false
, no filters will be applied to this parameter during training set generation. This annotation affects only the training set, and does not restrict ThingTalk execution.
Syntax: #[filterable=false]
Scope: output parameters
Type: boolean
If unspecified: true
Example:
class @com.yelp {
query restaurant(out phone : Entity(tt:phone_number)
#[filterable=false],
...);
This annotation indicates that a filter on the annotated parameter should not be added if there is already another filter on the given parameters. This annotation affects only the training set, and does not restrict ThingTalk execution.
Syntax: #[conflict_filter=[$p1, $p2]]
Scope: output parameters
Type: list of parameter names
Example:
class @com.yelp {
query restaurant(out id : Entity(com.yelp:restaurant)
#[conflict_filter=['price_range', 'cuisines'],
...);
(In the example, at training time a phrase that searches for a restaurant by price or cuisine will not be extended with a filter on name, because the name implies the price and the cuisine so the search would be overconstrained.)
This annotation requires that either one optional parameter in the specified lists has a defined value.
Syntax:
#[require_either=[
[$p1, $p2, ...], // constraint 1
[$p3, $p4, ...], // constraint 2
]
Scope: function
Type: list of list of input parameter names; each list is an OR requirement over all parameters; the whole annotation is an AND requiremenet
Example:
query book_taxi(in opt departure_time : Time,
in opt arrival_time : Time)
#[require_either=[['departure_time', 'arrival_time']]];
The default list of parameters that will always be returned by the function. Those parameters are removed from explicit projections in training, and are always included at runtime.
Syntax: #[minimal_projection=[$p1, $p2, ...]]
Scope: function
Type: array of output parameter name
If unspecified: ["id"]
if a parameter called id
is present, []
otherwise
Example:
class @org.thingpedia.weather {
query current(out status: Enum(raining,cloudy,sunny,snowy,sleety,drizzling,windy))
#[minimal_projection=["status"]];