Datastax Astra Streaming source connector Features Connector name astra-streaming Compatibility Pulsar 2.9.x or above Delivery guarantee At least once Supported task sizes S, M, L Multiplex capability A single instance of this connector can read from a single topic. Supported stream types Append Configuration properties Property Description Required Default Connection service-url The URL to connect to your Astra Streaming broker. For example, pulsar+ssl://broker.example.com:6651. Yes admin-url The URL to connect to your Astra Streaming admin endpoint. For example, <https://broker.example.com>. Yes Authentication token The authentication token associated with your Astra stream. This must be provided as a secret resource. Yes Data topic The fully qualified name of the topic. For example, persistent://stream/namespace/topic-name. Yes key.fields A list of fields, delimited by semicolons, that comprise the partition key. For example: field1;field2. — key.format The format used to serialize and deserialize the partition key. Must be one of the following: JSON Avro Raw — — format The format for data in the topic. Must be one of the following: JSON Avro Raw Debezium (JSON) If you want to send CDC data through this connector, then you must select Debezium (JSON). — — value.fields-include If set to ALL then the partition key columns will be included in the payload. Set to EXCEPT_KEY if you don’t want the partition key columns to be included in the payload. For an example of how the key.fields, key.format, and value.fields-include arguments work together, see the examples in the Key and Value Formats section in the Apache Flink documentation. — ALL scan.startup.mode Specifies where in the topic to start reading data when the connection is first started, or when it’s restarted with the state discarded. latest: Start reading data from the latest available point in the stream. earliest: Start reading data from the earliest available point in the stream. — latest Schema Registry Decodable can automatically populate the connection’s schema using Pulsar’s schema registry. To do this you need to create the connection through the Decodable Web UI. If you would like to manually enter the schema in the Decodable Web UI, select New Schema or Import Schema. If you want to automatically populate the connection schema using Pulsar’s schema registry, you’ll also need to provide the name of your Pulsar tenant, the namespace where your topic is, and the version of the schema that you’d like to use. Select Next when you are finished providing defining the connection’s schema. Give the newly created connection a Name and Description and select Save. Connector starting state and offsets When you create a connection, or restart it and discard state, it will read from the position in the topic as defined by scan startup mode. By default this is latest and will therefore read from the end of the topic. Learn more about starting state here.