Data Generator source connector

The Datagen connector can be used as a source to generate sample data based on a specified data.type. Use the Datagen connector when you want to test pipeline SQL statements using random data without setting up connections to external systems.

Features

Delivery guarantee

At most once

Use the Datagen Connector

If you want to use the Decodable CLI or API to create the connection, you can refer to the Property Name column for information about what the underlying property names are. The connector name is datagen.
  1. From the Connections page, select the Datagen connector and complete the following fields.

    UI Field Property Name Description

    Data Type

    data.type

    The type of sample data that you want this connection to emit. Must be one of the following:

    - envoy: Emits records formatted in Envoy’s default layout pattern. See Default format pattern in the Envoy documentation for an example of what the emitted record looks like.

    - counter: Emits records with counter and time fields. The counter field is a monotonically increasing bigintand the time field is a timestamp encoded as a string.

    Delay

    delay

    Specifies the delay in milliseconds to wait between records. Must be greater than 100.

    Defaults to 100.

    Value Format

    value.format

    Must be set to JSON.

  2. Select the stream that you’d like to connect to this connector. Then, select Next.

  3. On the "Define this Connection’s schema" page, select Next. You cannot modify the schema for the Datagen connector.

  4. Give the newly created connection a Name and Description and select Save.