Use the ClickHouse Connector to get data from Decodable into a ClickHouse table.

Overview

The ClickHouse Connector connects to a ClickHouse table through HTTP requests. The ClickHouse Connector can materialize data from both append and change streams. However, a ClickHouse connection handling change streams, particular change streams with frequent update and delete records, may operate significantly slower than pure append streams.

Prerequisites

Before you can get data from Clickhouse, the following requirements must be met:

• Your ClickHouse instance must be publicly accessible. Decodable uses the username and password provided during connection creation to authenticate to the database.
• A ClickHouse cluster, database, and table to send data to. Decodable does not create any of these entities for you, so they must already exist in your ClickHouse instance for Decodable to send data to them.

Steps

Follow these steps to get data from Decodable into ClickHouse. These steps assume that you are using Decodable Web. However, if you want to use the Decodable CLI to create the connection, you can refer to the Property Name column for information about what the underlying property names are.

1. From the Connections page, select ClickHouse and complete the following fields.

2. Select which stream contains the records that you’d like to send to ClickHouse. You can also choose to create a new stream. Then, select Next.
3. If you decided to create a new stream in the previous step, you must define its schema. Select New Schema to manually enter the fields and field types present or Import Schema if you want to paste the schema in the form of an Avro or JSON array.
   1. The stream's schema must match the schema of the data that you plan on sending through this connection. For information on how ClickHouse types map to Decodable types, see Data Type Mappings.
   2. If you want to send Change Data Capture (CDC) records to ClickHouse, then you must specify one or more field(s) to use as a primary key. To do this, you must first explicitly tell Decodable that the type is not null explicitly by entering: <type> NOT NULL. For example: BIGINT NOT NULL. Then, you are able to specify the not null field as a primary key field.
4. Select Next when you are finished providing defining the connection’s schema.
5. Give the newly created connection a Name and Description and select Save.

Data Type Mappings

The following table describes the mapping of Decodable data types to their ClickHouse data type counterparts. See the documentation for more info on ClickHouse's types, and Decodable's supported types.