OpenSearch sink connector Features Connector name opensearch Compatibility OpenSearch 1.x OpenSearch 2.x Delivery guarantee At least once Supported task sizes S, M, L Multiplex capability A single instance of this connector can write a single Decodable stream to one or more OpenSearch indices. Supported stream types Append stream Change stream Configuration properties Property Description Required Default hosts A semicolon-delimited list of OpenSearch hosts to connect to. For example: http://node-1:9092;http://node-2:9093 Yes index The name of the OpenSearch index to send data to. If the index doesn’t exist, Decodable will create it for you. Decodable supports static and dynamic index creation. See Types of Index. Yes username The username to connect to your OpenSearch cluster. — password The authentication token associated with your OpenSearch password. This must be provided as a secret resource. — Advanced sink.bulk-flush.max-actions Maximum number of buffered records per request. — 1000 sink.bulk-flush.max-size Maximum size of buffered records per request (in MB) — 2 sink.bulk-flush.interval The interval to flush buffered records (in seconds). — 1 Types of Index Static Index A single target index that all records will be continuously written into represented by a plain string index name. For example, users. Dynamic Index A dynamic index includes {field_name} in the index name to reference a field value and dynamically generate a target index for each record. The dynamic index name can also be of the form {field_name|date_format_string}, where date_format_string is a format pattern compatible with Java’s DateTimeFormatter. For example: users-at-{ts|yyyy-MM-dd}, where ts is a timestamp field in the connector schema and yyyy-MM-dd is an optional date format pattern. index-for-user-{user} When using a dynamic index generated by the current system time with a change stream, there is no guarantee that the records with the same primary key can generate the same index name. Therefore, if you are using a dynamic index based on the system time you should only use an append stream. Connector starting state and offsets A new sink connection will start reading from the Latest point in the source Decodable stream. This means that only data that’s written to the stream when the connection has started will be sent to the external system. You can override this when you start the connection to Earliest if you want to send all the existing data on the source stream to the target system, along with all new data that arrives on the stream. When you restart a sink connection it will continue to read data from the point it most recently stored in the checkpoint before the connection stopped. You can also opt to discard the connection’s state and restart it afresh from Earliest or Latest as described above. Learn more about starting state here.