Topics Extraction API version 2.0

Entity/Concept object

Both entities and concepts have the same basic structure even if some of the specific values found in each field are different. In the following explanation element will refer to both entity and concept objects.

Each element found will be a node in our ontology. There are two types of information associated to each element:

• Basic element information, that is, the information specific to the element found. It tells which element it is (form), how many times and in which form it appears in the text (variant_list), its global relevance, if it belongs to a specific dictionary, and in the cases where it is a known element, its unique identifier, id in the ontology and known standards (standard_list).
• Semantic information, or the different nodes to which the element node is related to. There are different aspects of semantic information: type of entity (sementity), geographical and thematic information (semgeo_list and semtheme_list) and other, more generic types (semrefer_list).

sementity will be the only semantic aspect of the element that will be mandatory, as it will be associated to the sense of the element found and each sense translates into an entity/concept object in the output. In terms of the ontology, sementity contains information from the node in the ODENTITY_TOP branch to which the element node found is related to. For example, London has two senses, last name and city, so in a scenario with no disambiguation, this will mean two entities will be found, each one with a different sementity object, one with the id ODENTITY_LAST_NAME and the other with the id ODENTITY_CITY.

The sementity element contains a field called type with the expanded hierarchy of the entity type that provides a much more intuitive grasp of the sense associated to the element. Each level of the hierarchy will follow a notation a bit more user-friendly than the node names seen until now: the entity type id will lose the prefix ODENTITY_, the underscores will be deleted and the capitalization will follow the upper CamelCase style. Using the previous examples:

• ODENTITY_CITY -- City
• ODENTITY_LAST_NAME -- LastName

sementity will also include an attribute called class, which will indicate if the element in question is an instance of the entity type, or if it is a class. In the case of an entity, this value will always be an instance, as a named entity is always an example of the class the node sementity represents. Elements with class=class will appear as concept objects.

semtheme_list is conformed by semtheme objects. semtheme is quite similar to sementity, instead of refering to the entity type (a node in the ODENTITY_TOP branch of the ontology), it points to the theme or themes the node belongs to (a node in the ODTHEME_TOP branch of the ontology). semtheme also contains a type field with the expanded version of the hierarchy; it follows the same pattern mentioned in sementity before but with ODTHEME_ as prefix:

• ODTHEME_BASIC_SCIENCES -- BasicSciences
• ODTHEME_MYTHOLOGY -- Mythology

There will be as many semtheme elements as themes the node relates to.

Both sementity and semtheme are characterized by always refering to class nodes. The rest of the semantic information associated to the entity will refer to instance nodes. The main difference this will show in the output is that classes will be identified by their name (e.g. ODENTITY_CITY) while instances will be referred to by a unique alphanumeric string that univocally identifies the node in the ontology (id).

Similarly to sementity and semtheme, semgeo (each element contained in semgeo_list) provides information on the node's hierarchy, although in this case the hierarchy corresponds to a geopolitical criteria. Instead of including the values in a single field and taking into account that some cases may be multiple inheritance (for instance, a mountain chain that belongs to two different countries), there will be specific object for each level which will be identified by its form and its node id.

semrefer_list will contain other references between the entity/concept node and other instances in the ontology. There are currently two types: organization, which links an instance of the ODENTITY_ORGANIZATION type (or its descendants), and affinity which shows an affinity relationship between the entity node and another instance in the ontology. Each object in semrefer will be represented but its form and its node id.

The last field included in an entity/concept, semld, is a mix of the two types of information described: it contains information specific to the node but said information are links to external ontologies such as SUMO, Wikipedia or YAGO.

The following table contains the fields that will appear in an entity and concept objects.

Semgeo object

The geographical information extracted from the text is represented assemgeoobjects with the following structure:

Semld gateways

The following table includes the gateways associated to an identifier, and how to use it:

Semrefer object

Semtheme object

Standard object

For example, the ISO3166-1 standard for countries will be identified as ISO3166-1-a2 when it refers to the two letters that identify each country and as ISO3166-1-a3 for the three-letter id. NYSE will be the value used to identify the ticker of a company that trades in the NY stock exchange.

These are all the values that may appear in id:

Variant object

Time expression object

For time expressions that refer to a specific date, the following format will be used to represent its associated value:

century|era|season|weekday|year|month|day|hour|minutes|seconds|timezone

These are the values each field may have:

• century, year, month, day, hour, minutes, seconds: numeric values
• era: after Christ (aC), before Christ (dC)
• season: spring (s), summer (v), autumn (a), winter (w)
• weekday: Monday (m), Tuesday (t), Wednesday (w), Thursday (j), Friday (f), Saturday (s), Sunday (d)
• timezone: must be specified either by using the standard timezones designations (CET, EST, etc.) or with the offset with respect to GMT, e.g.: GMT+02:00
• +/- indicate references after/before the returned value (e.g. +2 days)
• ~ indicates approximate values

If an expression has no value for one of the positions, it will be empty.

These would be some examples of how this would look:

• It's 7:30 in the evening -- |||||||19|30||
• 27th February at 3pm -- |||||2|27|15|||
• 5th june 2008 -- 21||||2008|6|5||||

This representation of the time will be used to calculate the value in actual_time, which will use as reference timeref and will return a date value in one of the following three formats: YYYY-MM-DD hh:mm:ss GMT±HH:MM, YYYY-MM-DD and hh:mm:ss GMT±HH:MM. For the examples seen and using as reference 2013-01-01 12:12:12 GMT+01:00, the result would be:

• It's 7:30 in the evening -- 19:30:00 GMT+01:00
• 27th February at 3pm -- 2013-01-27 15:00:00 GMT+01:00
• 5th june 2008 -- 2008-06-05

In some cases, actual_time returns values that are not certain (for example, minutes and seconds in the second example), so a precision value is added to filter these out. The values for precision are the positions of the normalized_form field plus hourAMPM, minutesAMPM and secondsAMPM. This will result in obtaining different objects for it's 7:30 and it's 7:30 in the evening.

Money expression object

Lists of money expressions found in the text and represented as money_exppression objects.

It will be considered that there is a money expression when there is both a currency and an amount in a valid structure. The currency will be expressed using the ISO4217 and in the cases where more than one currency may apply, all the possible values will be returned separated by | and ordered alphabetically.

Other expression object

Some specific patterns will be considered known ones, and identified as such through the field type. The patterns detected are the following:

• Spanish:
• bank account number: 20 digits with the format xxxx xxxx xx xxxxxxxxxx
• license plate: Spanish license plate with two formats: dddd-LLL and ddddLLL (where d are digits and L are capital letters)
• id: national id document: ddddddddL or dddddddd-L (with d digits, L a capital letter)
• All languages:
• flight number: detects flight numbers with the format LLdddd (where d are digits and L are capital letters)

Quotation object

Quotations in direct speech will not always include information regarding who they are attributed to; in those cases the fields who and verb will not appear.

Relation object

The syntactic triples will be defined by subject-verb pairs, and all the complements associated to that verb. There are two possible exceptions to this:

• Cases where the existing relation has an omitted verb (for example, appositions). In this case, the verb is assumed to be "to be" (or its equivalent, depending on the language), and its form will appear between parentheses.
• The subject is omitted (very common in some languages such as Spanish), in which the subject will not appear.

If a subject-verb pair appears several times in the same text, they will only appear once associated to the sentence they first appear in; the complement_list of the following appearances will be included in that relation.