REST source connector

Features

Use the REST Connector to send data to Decodable over HTTPS. After you’ve created a REST connection, you can send data to Decodable using the /v1alpha2/connections/<connection-id>/events endpoint.

Connector name

rest

Delivery guarantee

At least once

Supported task sizes

S, M, L

Supported stream types

xref:streams.adoc#append

Configuration properties

Property Description Required Default

Property	Description	Required	Default
`url`	The URL where your application or service will send HTTP POST requests, using the encoded `api-key` as the bearer token.	Yes
`api-key`	The ID of the secret whose value will be used to authenticate writes to the REST connection via the Decodable API. To write events to this connection, base64 encode the plaintext value of this secret and use it as a bearer token on POST requests to the location provided in the `url` property.	Yes
`parse-error-policy`	Error handling policy. Must be one of the following: `FAIL`: When set to FAIL, Decodable stops the connection if any validation errors are encountered in the incoming data stream. `IGNORE`: When set to IGNORE, Decodable ignores all invalid records. All validated records are sent. With this policy, the connection isn’t stopped nor will any errors or warnings be shown if there is an invalid record. A total successful `count` will be returned in the response body.	—	`FAIL`
`format`	Must be set to JSON.	—	`JSON`

url

The URL where your application or service will send HTTP POST requests, using the encoded api-key as the bearer token.

Yes

api-key

The ID of the secret whose value will be used to authenticate writes to the REST connection via the Decodable API.

To write events to this connection, base64 encode the plaintext value of this secret and use it as a bearer token on POST requests to the location provided in the url property.

Yes

parse-error-policy

Error handling policy. Must be one of the following:

FAIL: When set to FAIL, Decodable stops the connection if any validation errors are encountered in the incoming data stream.
IGNORE: When set to IGNORE, Decodable ignores all invalid records. All validated records are sent. With this policy, the connection isn’t stopped nor will any errors or warnings be shown if there is an invalid record. A total successful count will be returned in the response body.

—

FAIL

format

Must be set to JSON.

—

JSON

Example

Base64-encode your API key and store it in an environment variable in your shell. This is the API key that you defined in the REST connection.
```
export DECODABLE_API_KEY_BASE64=$(echo -n '<API key plaintext>' | base64)
```
Be careful using this on a shared system, where other users might be able to see the environment variables you’ve set, or examine your shell history. For production keys, best practices are to load them from a secure secret store.

Send your records through the REST connection. Use the following example as a template for your request, where <url> is the URL property shown in the connection details once you’ve set your API key and payload.json is a file containing the records you want to submit.

curl -X POST <url>                                       \
  -H "Content-Type: application/json"                    \
  -H "Accept: application/json"                          \
  -H "Authorization: Bearer ${DECODABLE_API_KEY_BASE64}" \
  -d @payload.json

The payload.json file must be in the following format, and the records must have the same schema as the schema that you defined in the connection.

{
  "events": [
    {
      "<fieldName>": <fieldValue>
    }
    ...
  ]
}

A response displaying how many records were written is shown. Records are written atomically to the stream, meaning all records in the POST body are written or none are. Although rare, if an error occurs after successful writing (for example a network partition or failure of the client or server), the client will need to retry, possibly resulting in duplicates. Records are guaranteed to be written in the order they appear in the events element of the POST body.