REST source connector

Use the REST Connector to send data to Decodable over HTTPs. After you’ve created a REST connection, you’re able to send data to Decodable using the /v1alpha2/connections/<connection-id>/eventsendpoint.

Features

Delivery guarantee

At least once

Steps: Create a REST connection

If you want to use the Decodable CLI or API to create the connection, you can refer to the Property Name column for information about what the underlying property names are. The connector name is rest.

From the Connections page, select the REST connector and complete the following fields.

UI Field Property Name Description

UI Field	Property Name	Description
URL	`url`	The URL where your application or service will send HTTP POST requests, using the encoded `api-key` as the bearer token. See the below example for details: Send data to Decodable using the REST APIs.
Endpoint	`endpoint`	DEPRECATED: The `endpoint` property is removed from the UI in favor of the `url` property as soon as your connection is assigned an `api-key`. The location where your application or service will need to send HTTP POST requests to. The complete URL will be in the format `https://<your_account>.api.decodable.co/v1alpha2/connections/<connection_id>/events`. Once the connection has been created, you can find the connection id by double-clicking on the name of the connector. The id is shown directly below the connector name.
Value Format	`format`	Must be set to JSON.
Parse error policy	`parse-error-policy`	Optional. Select the error handling policy. Must be one of the following: `FAIL`: When set to FAIL, Decodable stops the connection if any validation errors are encountered in the incoming data stream. `IGNORE`: When set to IGNORE, Decodable ignores all invalid records. All validated records are sent. With this policy, the connection is not stopped nor will any errors or warnings be shown if there is an invalid record. A total successful `count` will be returned in the response body. Defaults to `FAIL`.
API Key	`api-key`	The ID of the secret whose value will be used to authenticate writes to the REST connection via the Decodable API. To write events to this connection, you will base64 encode the plaintext value of this secret and use it as a bearer token on POST requests to the location provided in the `url` property. See the below example for details: Send data to Decodable using the REST APIs. We recommend generating a complex, unique value for your API key and storing it securely for later use. If you are using the Decodable CLI, run `decodable secret --help` for help with creating a new secret. Once created, you can reuse your API key secret on multiple REST connections. Note: For security purposes, Decodable will never display secret values in plaintext. You can manage which users have permissions to create, delete, or modify secrets in the Access Control management view. For more information, see Roles, groups, and permissions.

URL

url

The URL where your application or service will send HTTP POST requests, using the encoded api-key as the bearer token. See the below example for details: Send data to Decodable using the REST APIs.

Endpoint

endpoint

DEPRECATED: The endpoint property is removed from the UI in favor of the url property as soon as your connection is assigned an api-key.

The location where your application or service will need to send HTTP POST requests to. The complete URL will be in the format https://<your_account>.api.decodable.co/v1alpha2/connections/<connection_id>/events.

Once the connection has been created, you can find the connection id by double-clicking on the name of the connector. The id is shown directly below the connector name.

Value Format

format

Must be set to JSON.

Parse error policy

parse-error-policy

Optional. Select the error handling policy. Must be one of the following:

FAIL: When set to FAIL, Decodable stops the connection if any validation errors are encountered in the incoming data stream.
IGNORE: When set to IGNORE, Decodable ignores all invalid records. All validated records are sent. With this policy, the connection is not stopped nor will any errors or warnings be shown if there is an invalid record. A total successful count will be returned in the response body.

Defaults to FAIL.

API Key

api-key

The ID of the secret whose value will be used to authenticate writes to the REST connection via the Decodable API. To write events to this connection, you will base64 encode the plaintext value of this secret and use it as a bearer token on POST requests to the location provided in the url property. See the below example for details: Send data to Decodable using the REST APIs.

We recommend generating a complex, unique value for your API key and storing it securely for later use. If you are using the Decodable CLI, run decodable secret --help for help with creating a new secret. Once created, you can reuse your API key secret on multiple REST connections.

Note: For security purposes, Decodable will never display secret values in plaintext. You can manage which users have permissions to create, delete, or modify secrets in the Access Control management view. For more information, see Roles, groups, and permissions.

Select the stream that you’d like to connect to this connector. This will be the stream that receives events using REST. Then, select Next.
Define the connection’s schema. Select New Schema to manually enter the fields and field types present or Import Schema if you want to paste the schema in Avro or JSON format.
1. The stream’s schema must match the schema of the data that you plan on sending through this connection.
2. For more information about creating a stream or defining the stream schema, see Create and manage streams.
Select Next when you are finished defining the connection’s schema.
Give the newly created connection a Name and Description and select Save.

Your Decodable stream can now receive data from HTTP clients using the REST Connector. Next, configure your HTTP clients to send data to Decodable by following the instructions in the next section.

Send data to Decodable using the REST APIs

To send data through the REST connection you created earlier, follow these steps.

Base64-encode your API key and store it in an environment variable in your shell. This is the API key that you defined in the REST connection.
```
export DECODABLE_API_KEY_BASE64=$(echo -n '<API key plaintext>' | base64)
```
Be careful using this on a shared system, where other users might be able to see the environment variables you’ve set, or examine your shell history. For production keys, best practices are to load them from a secure secret store.
Send your records through the REST connection. Use the following example as a template for your request, where <url> is the url property shown in the connection details once you’ve set your API key and payload.jsonis a file containing the record(s) you want to submit.
```
curl -X POST <url> \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer ${DECODABLE_API_KEY_BASE64}" \
  -d @payload.json
```
Notes:
- The payload.json file must be in the following format, and the records must have the same schema as the schema that you defined in the connection.
  { "events": [ { "<fieldName>": <fieldValue> } ... ] }

A response displaying how many records were written is shown. Records are written atomically to the stream, meaning all records in the POST body are written or none are. Although rare, if an error occurs after successful writing (for example a network partition or failure of the client or server), the client will need to retry, possibly resulting in duplicates. Records are guaranteed to be written in the order they appear in the events element of the POST body.