StarTree sink integration

This is a supported integration which requires manual configuration.

Contact Decodable support if you are interested in native support with a Decodable connector.

Sending a Decodable data stream to StarTree is accomplished in two stages:

  1. Creating a sink connector from Decodable to a data source that’s supported by StarTree

  2. Adding that data source to your StarTree configuration.

Decodable and StarTree mutually support several technologies, including the following:

  • Amazon S3

  • Apache Kafka

Add a Kafka sink connector

Follow the configuration steps provided for the Apache Kafka sink connector.

Create Kafka data source

The StarTree Data Manager can ingest messages from your own Kafka cluster or hosted Kafka services like Confluent Cloud or Amazon Managed Streaming for Apache Kafka (MSK).

  1. Create a dataset.

  2. For the data source, select Streaming and then Kafka.

  3. For the data modeling, select the topic that you’d like to connect to Pinot and its format.

  4. Optionally, add a primary key to enable upserts to your real-time table.

  5. Review dataset configuration and submit to create.

For more detailed information, refer to StarTree’s Kafka documentation.