REST source connector

Features

Use the REST Connector to send data to Decodable over HTTPS. After you’ve created a REST connection, you can send data to Decodable using the /v1alpha2/connections/<connection-id>/events endpoint.

Connector name

rest

Delivery guarantee

At least once

Supported task sizes

The standard task sizes are not applicable to this connector.

Supported stream types

This connector will write an Append stream.

Configuration properties

Property Description Required Default

Property	Description	Required	Default
`api-key`	The ID of the secret whose value will be used to authenticate writes to the REST connection via the Decodable API. To write events to this connection, encode the plaintext value of this secret in Base64 and use it as a bearer token on POST requests to this REST endpoint’s URL.	Yes
`parse-error-policy`	Error handling policy. Must be one of the following: `FAIL`: When set to FAIL, Decodable stops the connection if any validation errors are encountered in the incoming data stream. `IGNORE`: When set to IGNORE, Decodable ignores all invalid records. All validated records are sent. With this policy, the connection isn’t stopped nor will any errors or warnings be shown if there is an invalid record. A total successful `count` will be returned in the response body.	—	`FAIL`
`format`	Must be set to JSON.	—	`JSON`

api-key

The ID of the secret whose value will be used to authenticate writes to the REST connection via the Decodable API.

To write events to this connection, encode the plaintext value of this secret in Base64 and use it as a bearer token on POST requests to this REST endpoint’s URL.

Yes

parse-error-policy

Error handling policy. Must be one of the following:

FAIL: When set to FAIL, Decodable stops the connection if any validation errors are encountered in the incoming data stream.
IGNORE: When set to IGNORE, Decodable ignores all invalid records. All validated records are sent. With this policy, the connection isn’t stopped nor will any errors or warnings be shown if there is an invalid record. A total successful count will be returned in the response body.

—

FAIL

format

Must be set to JSON.

—

JSON

Example

Base64-encode your API key and store it in an environment variable in your shell. This is the API key that you defined in the REST connection.
```
export DECODABLE_API_KEY_BASE64=$(echo -n '<API key plaintext>' | base64)
```
Be careful using this on a shared system, where other users might be able to see the environment variables you’ve set, or examine your shell history. For production keys, best practices are to load them from a secure secret store.

Send your records through the REST connection. Use the following example as a template for your request, where <url> is the derived URL property shown in the Configuration tab on your REST connection; the URL will look like https://<number>.api.data.decodable.co/v1alpha2/connections/<connection_id>/events. Once you’ve set your API key and created a payload.json file containing your JSON records, you are ready to submit the events to Decodable.

curl -X POST <url>                                       \
  -H "Content-Type: application/json"                    \
  -H "Accept: application/json"                          \
  -H "Authorization: Bearer ${DECODABLE_API_KEY_BASE64}" \
  -d @payload.json

The payload.json file must be in the following format, and the records must have the same schema as the schema that you defined when configuring your REST source connection.

{
  "events": [
    {
      "<fieldName>": <fieldValue>
    }
    ...
  ]
}

A response displaying how many records were written is shown. Records are written atomically to the stream, meaning all records in the POST body are written or none are. Although rare, if an error occurs after successful writing (for example a network partition or failure of the client or server), the client will need to retry, possibly resulting in duplicates. Records are guaranteed to be written in the order they appear in the events element of the POST body.