Skip to content

Sending values to the platform

Prerequisites

Any registered client can become a values producer regardless of the authentication method (client on behalf of the user or client as itself, see Security). However, there are two prerequisites before values can be sent:

  1. A Scope must be defined for the project or experiment that "owns" the values (see also Scopes explained)
  2. The client used for posting the values must be assigned the appropriate access rights for this Scope. Contact a system administrator or project manager to acquire these rights (see Request access).

Data point specification

The platform should be able to handle large volumes of incoming sensor values, which are then processed and stored so useful values can be retrieved later on. The term sensor values can be interpreted in a broad sense, allowing the platform to be used for a wide range of applications. Nevertheless, each values point must conform to having the following attributes.

Index Concept Type Description
0 timestamp long Each event must have a time dimension expressed as a long integer. By default the timestamp value is interpreted as milliseconds UTC.
1 metric id string This value specifies the series to which the event values belongs to. (a.k.a. what is being measured, for example temperature.celsius or airquality.no2)
2 source id string This value identifies the event source. (e.g. a sensor device, an anonymized user id)
3 value number
boolean
string
array
object
Measured by the source. Currently the framework supports the following datatypes:
• numbers (integer or floating point)
• booleans
• strings (max 32k characters)
• number arrays (limited to max size of 128)
• JSON Objects (max 32k characters as text)
4 location [number, number] A longitude-latitude coordinate pair encoded as a JSON array is used to optionally assign a location to the event. The array is structured as [longitude, latitude].
5 elevation number This value can optionally be used to assign a relative altitude to the event (in meters).

Warning

With geolocations, the order of longitude and latitude is a common source of bugs. The format Obelisk uses is [longitude, latitude]!

The values for these attributes are represented as a JSON array and can be posted in batch to the framework (see next section: Using the API).

Hint

Make sure the limitations imposed by this format are clear before integrating the framework in your own application or service.

Using the API

Request body

The following endpoint is used to post new values to the framework:

POST /api/v2/scopes/{scopeId}/ingest

The endpoint expects a JSON array request body representing a batch of events corresponding to the format explained above.

1
2
3
4
5
[
  [1526569675478, "button.pressed", "demo.things.button1", true],
  [1526569675478, "humidity.rh", "demo.things.multisensor1", 435, [4.41818582, 51.22154617]],
  [1526569675478, "temperature.celsius", "demo.things.multisensor1", 12.0, [4.41818582,51.22154617], 4.3]
]
Both the geolocation and elevation attributes are optional and can be omitted. Default values are [0.0, 0.0] (for geolocation) and 0.0 (for elevation).

Response codes

If the post was successful, a 204 No content HTTP response is returned. If the batch was malformed or one of the input validation steps failed, a 400 Bad Request HTTP response is returned. The body of the response will include more information on what went wrong, including the batch index of the event that caused the issue.

Metric type suffix

A new feature of Obelisk is type validation. To facilitate this, all metric names have been suffixed with their type.

The platform automatically detects the metric type based on the value of the event, if the value is either a number, a string or a boolean. For number arrays and JSON Objects, the metric type must explicitly be specified in the metric id.

You can explicitly specify the type for number, string or boolean as well, but this is not required.

The benefit of doing so, is that the platform can validate if the type of the value matches the expected type, which can be useful for preventing invalid values being added to the metric series. E.g. posting the following event will return a 400 Bad Request because the metric is defined to be of type string.

1
2
3
[
  [1526569675478, "humidity.rh::string", "demo.things.multisensor1", 435.5, [4.41818582, 51.22154617]]
]

Valid suffixes

Valid type suffixes are:

suffix descriptions
::number To express that the metric contains numbers (integer or doubles / floating point).
::number[size] To express that the metric contains a fixed number array (max size 128).
E.g. ::number[3] for representing 3D points.
::string To express that the metric contains string valules (max 16k characters).
::bool To express that the metric contains boolean values (true or false).
::json To express that the metric contains JSON Object values (since Obelisk v2.0).

The fixed size number array type is supported to facilitate use cases where heavily correlated values are being measured at the same time, e.g. in the case of 2D or 3D coordinates, sound frequency levels, etc...
By modeling the values in one metric as an array instead of multiple metrics, querying efficiency is greatly improved. Example of an event with an array type of size 2:

1
2
3
[
  [1543234226431, "xy::number[2]", "drone1", [40.5,20.7], [4.41818582, 51.22154617], 15.0]
]

The JSON Object type was introduced later to support more advanced use cases and to facilitate system integration (e.g. bridge to NGSI v2 ecosystem). Ingesting these type of metrics works in exactly the same way as the Obelisk primitive types:

1
2
3
[
  [1543234226431, "color::json", "lightsensor1", {"r": 145, "g": 166, "b": 18}, [4.41818582, 51.22154617], 15.0]
]

Consider your options before ingesting data for a Metric

JSON Objects are the easiest way to get started in a lot of use cases, but will perform slightly worse compared to the primitive types when querying. In addition, not all querying options will be available (e.g. stats operations will only work for top-level attributes of the JSON structure).

Timestamp precision

We allow modifying the precision of the timestamp value of the events when posting a batch.

!!! Warning: dd All timestamp are UTC!

By default a precision of milliseconds is assumed. This can be modified by adding the query parameter precision to the request with one of the following values:

  • microseconds
  • milliseconds
  • seconds

Warning

The precision applies to the whole batch, so make sure to use different batches when needing to post a set of events with multiple precisions!

JSON Schema validation

A new feature of Obelisk (since v2.0) is JSON Schema based validation for JSON Object based Metrics (::json). To enable this for a specific Metric, first upload a schema by updating the metadata for the Metric:

E.g. POST /api/v2/scopes/office-lab/metrics/color::json

HTTP Request Body:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
{
  "id" : "color::json",
  "schema" : {
    "$schema": "http://json-schema.org/draft-07/schema",
    "type": "object",
    "required": [
      "r",
      "g",
      "b"
    ],
    "additionalProperties": false,
    "properties": {
      "r": {
        "type": "number",
        "minimum": 0,
        "exclusiveMaximum": 255
      },
      "g": {
        "type": "number",
        "minimum": 0,
        "exclusiveMaximum": 255
      },
      "b": {
        "type": "number",
        "minimum": 0,
        "exclusiveMaximum": 255
      }
    }
  }
}

You can also use the JSON Schema $ref directive to link to externally hosted schema specifications (e.g. FIWARE Data Models JSON schemas).

Once the schema has been registered, you can use the alternative ingest endpoint. For example:

POST /api/v2/scopes/office-lab/ingest/color::json

Request body:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
[[
  1554115364486,
  "lightsensor1",
  {
    "r" : 148,
    "g" : 47,
    "b" : 210
  },
  [
    3.6996241439859525,
    51.03256244518894
  ]
]]

The schema is resolved once for each posted batch and is then used to validate the value payload.