OpenSearch sink connector

Use the OpenSearch connector to send data from Decodable to OpenSearch, an open source search and analytics suite.

Features

Delivery Guarantee

at least once

Steps

If you want to use the Decodable CLI or API to create the connection, you can refer to the Property Name column for information about what the underlying property names are. The connector name is opensearch.

From the Connections page, select the OpenSearch connector and complete the following fields.

UI Field Property Name Description

UI Field	Property Name	Description
Hosts	`hosts`	A semicolon-delimited list of OpenSearch hosts to connect to. For example: `http://host_name:9092;http://host_name:9093`.
Index	`index`	The name of the OpenSearch index to send data to. If the index doesn’t exist, Decodable will create it for you. Decodable supports two different methods for sending data to an OpenSearch index: Send data to a specific index: Send data to a single target index by specifying a plain string index name. For example: `users`. Dynamically route data to different indexes: Send data to specific indexes based on a field value present in the records themselves. To use this option, include the field name, enclosed in brackets, that you want to route data based on. You can also include a date format patterns with the field name. See DateTimeFormatter for supported format patterns. For example: `index-for-user-\{user\}` where `user` is a field present in the stream schema. `users-at-{ts\|yyyy-MM-dd}` where `ts` is a timestamp field present in the stream schema and `yyyy-MM-dd` is an optional date format pattern.
Username	`username`	The username to connect to your OpenSearch cluster.
Password	`password`	The authentication token associated with your OpenSearch password. If you are using the Decodable CLI, this is the ID of a secret resource in your account. Run `decodable secret list` to view available secrets or `decodable secret --help` for help with creating a new secret. Note: For security purposes, Decodable will never display secret values in plaintext. You can manage which users have permissions to create, delete, or modify secrets in the Access Control management view. See Roles, groups, and permissions for more information.

Hosts

hosts

A semicolon-delimited list of OpenSearch hosts to connect to. For example: http://host_name:9092;http://host_name:9093.

Index

index

The name of the OpenSearch index to send data to. If the index doesn’t exist, Decodable will create it for you. Decodable supports two different methods for sending data to an OpenSearch index:

Send data to a specific index: Send data to a single target index by specifying a plain string index name. For example: users.

Dynamically route data to different indexes: Send data to specific indexes based on a field value present in the records themselves. To use this option, include the field name, enclosed in brackets, that you want to route data based on. You can also include a date format patterns with the field name. See DateTimeFormatter for supported format patterns. For example:

index-for-user-\{user\} where user is a field present in the stream schema. users-at-{ts|yyyy-MM-dd} where ts is a timestamp field present in the stream schema and yyyy-MM-dd is an optional date format pattern.

Username

username

The username to connect to your OpenSearch cluster.

Password

password

The authentication token associated with your OpenSearch password. If you are using the Decodable CLI, this is the ID of a secret resource in your account. Run decodable secret list to view available secrets or decodable secret --help for help with creating a new secret.

Note: For security purposes, Decodable will never display secret values in plaintext. You can manage which users have permissions to create, delete, or modify secrets in the Access Control management view. See Roles, groups, and permissions for more information.

Select which stream contains the records that you’d like to send to OpenSearch. Then, select Next.
Give the newly created connection a Name and Description and select Save.

You can now start the connection to send data from Decodable into your OpenSearch cluster.