Amazon Kinesis source connector Features Connector name kinesis Delivery guarantee At least once Supported task sizes S, M, L Multiplex capability A single instance of this connector can read from one Kinesis stream Supported stream types Append stream Configuration properties Property Description Required Default aws.region The region in which the stream is located. Yes aws.credentials.role.arn The AWS ARN of the IAM Role that has the necessary permissions to access the Kinesis stream. Yes stream The name of the stream from which you want to receive data. Yes format The format of the data in the stream. Must be json — json Prerequisites Access to your AWS resources Decodable interacts with resources in AWS on your behalf. To do this you need an IAM role configured with a trust policy that allows access from Decodable’s AWS account, and a permission policy as detailed below. For more details on how this works, how to configure the trust policy, and example steps to follow see here. To use this connector you must associate a permissions policy with the IAM role. This policy must have the following permissions: Access to your Kinesis stream. For more information, see Controlling Access to Amazon Kinesis Data Streams Resources Using IAM. You can also restrict the permissions for specific Kinesis streams by specifying a Kinesis ARN instead of the wildcard *. Sample Permission Policy { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "kinesis:*", "Resource": "*" } ] } Metadata fields When consuming records from Amazon Kinesis into Decodable, you can access specific metadata fields associated with each Kinesis record. A metadata field is identified by a string-based key and an associated data type. For example, the Amazon Kinesis connector has a metadata field with the key timestamp and data type TIMESTAMP_LTZ(3). A metadata field has its type formed as: {datatype} METADATA [FROM '{key}'] Thus, a metadata field is indicated by the METADATA keyword, with an optional FROM to provide the key. If the key isn’t provided explicitly with FROM, it defaults to the name of the field. For example: TIMESTAMP_LTZ(3) METADATA FROM 'timestamp' Available metadata Key Data Type Description timestamp TIMESTAMP_LTZ(3) NOT NULL The approximate time when the record was inserted into the stream. shard-id VARCHAR(128) NOT NULL The unique identifier of the shard within the stream from which the record was read. sequence-number VARCHAR(128) NOT NULL The unique identifier of the record within its shard.