StarTree sink connector

StarTree Cloud provides managed hosting for Apache Pinot on all major cloud platforms, including AWS, GCP and Azure. StarTree Cloud enables developers to provision Pinot clusters of different sizes, ingest data from real-time and batch data sources, and run analytics workloads with ultra-low latency. Meanwhile, StarTree manages the underlying infrastructure for you, allowing you to have timely insights from a diverse set of data and make informed business decisions. StarTree Cloud consists of tools that give you a better developer experience, including StarTree Dataset Manager (for data ingestion) and StarTree ThirdEye (for Anomaly Detection).

Additional StarTree Support
If you are interested in a direct Decodable Connector for StarTree, contact support@decodable.co or join our Slack community and let us know!

Getting started

Sending a Decodable data stream to StarTree is accomplished in two stages, first by creating a sink connector to a data source that is supported by StarTree, and then by adding that data source to your StarTree configuration. Decodable and StarTree mutually support several technologies, including the following:

  • Amazon S3

  • Apache Kafka

Configure as a sink

This example demonstrates using Kafka as the sink from Decodable and the source for StarTree. Sign in to Decodable Web and follow the configuration steps provided in the Apache Kafka sink connector topic to create a sink connector. For examples of using the command line tools or scripting, see the How To guides.

Create Kafka data source

The StarTree Data Manager can ingest messages from your own Kafka cluster or hosted Kafka services like Confluent Cloud or Amazon MSK.

  1. Create a dataset.

  2. For the data source, select Streaming and then Kafka.

  3. For the data modeling, select the topic that you’d like to connect to Pinot and its format.

  4. Optionally, add a primary key to enable upserts to your real-time table.

  5. Review dataset configuration and submit to create.

For more detailed information, refer to StarTree’s Kafka documentation.

Reference

sink