StreamNative sink connector

StreamNative Cloud provides an easy-to-use and fully-managed Apache Pulsar service in the public cloud. It’s API-driven for easy integration with continuous-delivery pipelines. Securely connect to clusters based on OAuth 2.0 and TLS. Automated cluster upgrades mean zero downtime for your enterprise. You can sign up for StreamNative Cloud through the StreamNative website to create and manage StreamNative Cloud resources and Pulsar components. You can also use the StreamNative Cloud CLI tool for managing StreamNative Cloud resources.

If you are interested in a direct Decodable Connector for StreamNative, contact support@decodable.co or join our Slack community and let us know!

Getting started

Sending a Decodable data stream to StreamNative is accomplished in two stages, first by creating a sink connector to a data source that is supported by StreamNative, and then by adding that data source to your StreamNative configuration. Decodable and StreamNative mutually support several technologies, including the following:

  • Amazon Kinesis

  • Apache Kafka

Configure as a sink

This example demonstrates using Kafka as the sink from Decodable and the source for StreamNative. Sign in to Decodable Web and follow the configuration steps provided in the Apache Kafka sink connector topic to create a sink connector. For examples of using the command line tools or scripting, see the How To guides.

Create Kafka data source

The Kafka source connector pulls messages from Kafka topics and persists the messages to Pulsar topics hosted in StreamNative Cloud. Before using the Kafka source connector, you need to create a configuration file using JSON or YAML. As an example of using the Kafka source connector, you can use the following steps:

  1. Download a Kafka client and a Kafka connector.

  2. Create a network.

  3. Pull a ZooKeeper image and start ZooKeeper.

  4. Pull a Kafka image and start Kafka.

  5. Pull a Pulsar image and start Pulsar standalone.

  6. Create a producer file kafka-producer.py.

  7. Create a consumer file pulsar-client.py.

  8. Copy the configuration files to Pulsar.

  9. Open a new terminal window and start the Kafka source connector in local run mode.

  10. Open a new terminal window and run the consumer.

For more detailed information, see StreamNative’s Kafka documentation.