MySQL CDC

Use the MySQL CDC (Change Data Capture) Connector to get data from MySQL into Decodable. The MySQL CDC Connector is powered by Debezium and able to extract and send change events (INSERTS, UPDATES, and DELETES) through Decodable.

Overview

Connector namemysql-cdc
Typesource
Delivery guaranteeexactly once

Getting Started

Prerequisites

Before you can get data from MySQL, the following requirements must be met:

  • Your MySQL database must be publicly accessible. Decodable uses the username and password provided during connection creation to authenticate to the database.
  • Your MySQL instance must be configured to send change records. See Setting up MySQL from the Debezium documentation.
    • Note: If you are using AWS RDS, Automated Backups must be enabled. Otherwise, binlog will be disabled regardless of any other settings. See How To: Set up a MySQL CDC connection on AWS RDS for detailed instructions on how to configure MySQL on AWS RDS.
  • Your MySQL user must have sufficient permissions for CDC replication.

Steps

Follow these steps to get data from MySQL into Decodable. These steps assume that you are using the Decodable web interface. However, if you want to use the Decodable CLI to create the connection, you can refer to the Property Name column for information about what the underlying property names are.

  1. From the Connections page, select MySQL CDC and complete the following fields.
UI FieldProperty Name in the Decodable CLIDescription
Connection TypeN/ASelect Source to use this connector to get data into Decodable.
HosthostnameThe hostname for your MySQL database. For example, mysql-server.
PortportOptional. The port number to use when connecting to the host.

Defaults to 3306.
Databasedatabase-nameThe name of the database containing the table that you want to ingest data from.
Table Nametable-nameThe name of the table that you want to capture CDC records from.
UsernameusernameThe username to use to authenticate to MySQL.
PasswordpasswordThe password associated with the username. This must be provided as a secret resource. If you are using the Decodable CLI, run decodable secret list to view available secrets or decodable secret --help for help with creating a new secret.

Note: For security purposes, Decodable will never display secret values in plaintext. You can manage which users have permissions to create, delete, or modify secrets in the Access Control management view. See Manage Roles, Groups, and Permissions for more information.
  1. Select the stream that you’d like to connect to this connector. Then, select Next.
  2. Define the connection’s schema. Select New Schema to manually enter the fields and field types present or Import Schema if you want to paste the schema in the form of an Avro or JSON array.
    1. The stream's schema must match the schema of the data that you plan on sending through this connection.
    2. You must designate one or more fields to use as a primary key. A primary key is a field that contains a value that can be used to uniquely identify each row in a table. To specify a primary key, you must first explicitly tell Decodable that the type is not null explicitly by entering: <type> NOT NULL. For example: BIGINT NOT NULL.
    3. For more information about Change Data Capture, change streams, or creating a stream, see the following pages:
  3. Select Next when you are finished providing defining the connection’s schema.
  4. Give the newly created connection a Name and Description and select Save.