Write API

The HTTP POST method is used to transfer data to the server. The request is composed of two parts: the Request URL and the Request Body. Last segments of the Request URL are used to specify the Storage Bucket as well as the Key within that bucket. The Request Body is written in JSON format and contains the actual data (e.g. sensor values) as well as the authorization key for the Storage Bucket.

The Write API endpoint is: https://megasense-server.cs.helsinki.fi/write/v3/.

To use the API, you will need to have the following credentials:

• Storage bucket name
• Storage bucket access key

You can apply for these on the landing page.

Basic usage

The following simplified example demonstrates writing values using the POST method:

POST https://megasense-server.cs.helsinki.fi/write/v3/bucket/<bucket>/<key>
{
    "accessKey": "fbebeb8ae7f94930fd07dd8249287973e786d047e9820a254c4a3aecb36fc238",
    "deviceId": "outdoors_u871a",
    "data": {
        "timestamp": 1584156800,
        "temperature": 4.7
    }
}

URL Parameters:

• bucket: is the storage space of your project
• key is the target object in your bucket

Request Body (JSON):

• deviceId is an optional identifier for the device generating the data
• accessKey is the authorization key for your bucket
• data is whatever data you want to write

By default, the maximum size of a request is 10 megabytes. The server expects all requests to include the Content-Type (application/json) and Content-Length headers but those are usually handled by most HTTP clients and thus omitted here.

See Glossary for more in-depth explanation of the concepts.

Examples

Writing a Single Item

Here is an example of writing a single item with the command line utility curl:

curl -X POST "https://megasense-server.cs.helsinki.fi/write/v3/bucket/homelab/outdoors_temp1" \
--header "Content-Type: application/json" \
--data @- << 'EOF'
{
    "accessKey": "fbebeb8ae7f94930fd07dd8249287973e786d047e9820a254c4a3aecb36fc238",
    "data": {
        "timestamp": 1584156800,
        "temperature": 4.7
    }
}
EOF

In this request, the destination bucket is homelab and the target object is outdoors_temp1. This request would create an object at homelab/outdoors_sensor_1.jsonl with the following contents:

{"data":{"timestamp":1584156800,"temperature":4.7},"receivedTimestamp":1574180504054,"key":"outdoors_temp1"}

All writes use the JSON Lines format, where each written object ends up as a separate line. If you keep writing to the same object, each entry is appended as a new line like so:

{"data":{"timestamp":1584156800,"temperature":4.7},"receivedTimestamp":1574180504054,"key":"outdoors_temp1"} {"data":{"timestamp":1584176800,"temperature":5.2},"receivedTimestamp":1574181626641,"key":"outdoors_temp1"} {"data":{"timestamp":1584196800,"temperature":3.0},"receivedTimestamp":1574181635895,"key":"outdoors_temp1"}

By reusing the same file, you can maintain a one-to-one mapping between your devices and keys. This allows you to easily collect e.g. a time series of readings from a sensor. Because all lines in JSONL are valid JSON records, reading and processing the file should be easy in any language.

Writing Multiple Items

Multiple items can be written in one HTTP call by wrapping them in a JSON array. Items will be processed as if they were received in separate requests.

{
    ...
    "data": [
        <item 1>,
        <item 2>,
        <item 3>,
    ]
}

This format is recommended if your application generates a lot of requests.

Writing Images

Images and other binary blobs less than the maximum request size of 10 megabytes can be included in a request by encoding the data in Base64 format. All encoding and decoding must be handled by the application. Please note that the API is not suitable for processing video streams and alike.

Glossary

Storage Bucket

Storage Bucket is the storage space for the data in your project. When you create a new project, you should be assigned a bucket name as well as an Access Key. Buckets can accept and contain data from multiple sources. To differentiate between objects in your project, you use different Keys to refer to them.

Keys

Keys (not to be confused with an Access Keys) are unique identifiers for the objects in a bucket. You can think of keys being like file names pointing to files within the root directory (bucket). The combination of a bucket name and key uniquely identify an object.

Access Key

Access Key is a random alphanumeric string that authorizes you to perform actions on a bucket. Any person with the correct key can access your bucket, so keep it in a safe place and never share it publicly.

Device ID

Device ID is an optional identifier that can be used to differentiate between devices. It is recommended that you specify a device ID whenever possible to allow per-device configuration in the future. If you do not specify a device ID, a single malfunctioning device could lead to the temporary suspension of an entire project.

Data

Data is any object or an array of objects you want to write. The data can be a simple numeric, boolean or string value, or an object containing multiple values. Other formats like small images or binary blobs are also supported to a limited extent, see above for more details.