How To: Set up SASL Authentication with Apache Kafka

A common method for securing client connections to Apache Kafka brokers is through SASL. Decodable supports a number of SASL authentication mechanisms, both with and without SSL/TLS encryption.

Consult the Apache Kafka source connector page for full details and configuration reference.

Self-signed certificates aren’t supported

When using SASL authentication with SSL/TLS encryption (SASL_SSL), the broker must be configured to use a trusted certificate.

This page will explain how to set up SASL authentication for your connection from Kafka to Decodable.

We’ll assume here that you already have a Decodable account and have gotten started with the Decodable CLI. If you haven’t done that yet, see The Decodable CLI to learn how to install and setup the Decodable CLI.

Create a Stream

decodable stream create --name kafka_sasl_in            \
  --description "input stream"                          \
  --field value=string

Create a connection

When configuring a SASL connection, there are several required configuration properties.

Here is an example of creating a source Kafka connection using SASL/PLAIN:

decodable connection create --connector kafka --type source          \
  --name kafka-sasl-source                                           \
  --description "Kafka source connection with SASL/SSL"              \
  --stream-id=<stream-id>                                            \
  --field value=STRING                                               \
  --prop value.format=raw                                            \
  --prop bootstrap.servers=<broker_list>                             \
  --prop topic=source_raw                                            \
  --prop security.protocol=SASL_SSL                                  \
  --prop sasl.mechanism=PLAIN                                        \
  --prop sasl.username="<username>"                                  \
  --prop sasl.password="<secret id>"

sasl.password must be supplied as a secret.

Test the connection

The quickest way to test the connection is to activate it and run a preview job. After activation, we can verify that the connection is activated successfully by checking the actual state.

Note that it can take up to 1 minute for the state to update.

Activate the connection

$ decodable connection activate <connection_id>

$ decodable connection get <connection_id>

kafka-sasl-source
  id                       a3ead34d
  description
  connector                kafka
  type                     source
  stream id                abbcfccd
  schema
    0  value                 STRING
  properties
    bootstrap.servers        <broker_list>
    value.format             raw
    sasl.mechanism           PLAIN
    sasl.username            <username>
    security.protocol        SASL_SSL
    topic                    source_raw
  target state             RUNNING
  actual state             RUNNING
  create time              2021-11-23T15:15:02Z
  update time              2021-11-23T15:15:02Z

Create a preview job

Run a preview to read from the stream the source Kafka connection writes into. If you produce raw strings to the source_raw topic, you should see sample data coming out from the preview command output.

Note that it can take up to 1 minute for data to appear.

decodable pipeline preview "SELECT * FROM kafka_sasl_in"