Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. Customers of all sizes and industries can use Amazon S3 to store and protect any amount of data for a range of use cases, such as data lakes, websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics. Amazon S3 provides management features so that you can optimize, organize, and configure access to your data to meet your specific business, organizational, and compliance requirements.

• Storage and access management
• Storage logging and monitoring
• Analytics and insights
• Strong consistency

Getting Started

Sending a data stream from S3 to Decodable is accomplished in two stages, first by populating an AWS Kinesis stream via an AWS lambda function, and then by adding and configuring a Kinesis source connector in Decodable.

Create Kinesis Data Source

To capture events for S3, an AWS lambda function is used to populate an AWS Kinesis stream.

1. Configure S3 read permissions
2. Create a Kinesis stream
3. Configure Kinesis write permissions
4. Create the AWS lambda function
5. Create a notification trigger

For more detailed information, please refer to the S3 Lambda example in the Decodable GitHub repository.

Configure A Kinesis Source

Sign in to the Decodable Web Console and follow the configuration steps provided for the AWS Kinesis Connector to create a source connector. For examples of using the command line tools or scripting, see the How To guides.

Reference

Additional S3 Lambda Support
If you are interested in a direct Decodable Connector for S3 via an AWS lambda function, please contact [email protected] or join our Slack community and let us know!