Amazon Kinesis sink connector Use the Amazon Kinesis Connector to send data from Decodable into Amazon Kinesis. If you are looking for information about how to get data from Amazon Kinesis into Decodable, see Amazon Kinesis source connector in the Connect to a data source chapter. Features Delivery guarantee At least once Prerequisites Access to your AWS resources Decodable interacts with resources in AWS on your behalf. To do this you need an IAM role configured with a trust policy that allows access from Decodable’s AWS account, and a permission policy as detailed below. For more details on how this works, how to configure the trust policy, and example steps to follow see here. To use this connector you must associate a permissions policy with the IAM role. This policy must have the following permissions: Access to your Kinesis stream. For more information, see Controlling Access to Amazon Kinesis Data Streams Resources Using IAM. You can also restrict the permissions for specific Kinesis streams by specifying a Kinesis ARN instead of the wildcard *. Sample Permission Policy { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "kinesis:*", "Resource": "*" } ] } Steps If you want to use the Decodable CLI or API to create the connection, you can refer to the Property Name column for information about what the underlying property names are. The connector name is kinesis. From the Connections page, select Amazon Kinesis and complete the following fields. UI Field Property Name Description Connection Type N/A Select Source to use this connector to get data into Decodable. Stream Name stream The name of the stream that you want to receive data from. AWS Region aws.region The region that the stream is located in. IAM Role ARN aws.credentials.role.arn The AWS ARN of the IAM Role that you configured as part of the prerequisites. Value Format format The format of the data in the stream. Must be json. Select the stream that you’d like to connect to this connector. Then, select Next. Define the connection’s schema. Select New Schema to manually enter the fields and field types present or Import Schema if you want to paste the schema in the form of an Avro or JSON array. The stream’s schema must match the schema of the data that you plan on sending through this connection. For more information about creating a stream or defining the stream schema, see Create and manage Streams. Select Next when you are finished providing defining the connection’s schema. Give the newly created connection a Name and Description and select Save. Connector starting state and offsets A new sink connection will start reading from the Latest point in the source Decodable stream. This means that only data that’s written to the stream when the connection has started will be sent to the external system. You can override this when you start the connection to Earliest if you want to send all the existing data on the source stream to the target system, along with all new data that arrives on the stream. When you restart a sink connection it will continue to read data from the point it most recently stored in the checkpoint before the connection stopped. You can also opt to discard the connection’s state and restart it afresh from Earliest or Latest as described above. Learn more about starting state here.