A common method for securing client connections to Kafka brokers is through SASL (ref)
Decodable supports a number of SASL authentication mechanisms, both with and without SSL/TLS encryption.
We'll assume here that you already have a Decodable account and have gotten started with the Decodable CLI. If you haven't done that yet, refer to the Setup doc.
decodable stream create --name kafka_sasl_in \ --description "input stream" \ --field value=string
When configuring a SASL connection, the required properties are:
security-protocol (one of
sasl.mechanism (one of
sasl.username (the provided username or API key)
sasl.password (the provided password or API secret key)
Note that when using SASL authentication with SSL/TLS encryption (
SASL_SSL), the broker must be configured to use a trusted certificate. Self-signed certificates are not supported at this time.
These can be used as Decodable connection properties, e.g.
decodable connection create --connector kafka --type source \ --name kafka-sasl-source \ --description "Kafka source connection with SASL/SSL" \ --stream-id=<stream-id> \ --field value=STRING \ --prop value.format=raw \ --prop bootstrap.servers=<broker_list> \ --prop topic=source_raw \ --prop security.protocol=SASL_SSL \ --prop sasl.mechanism=PLAIN \ --prop sasl.username="<username>" \ --prop sasl.password="<password>"
The quickest way to test the connection is to activate it and run a preview job. After activation, we can verify that the connection is activated successfully by checking the
Note that it may take up to 1 minute for the state to update.
decodable connection activate <connection_id> decodable connection get <connection_id> #sample output kafka-sasl-source id a3ead34d description connector kafka type source stream id abbcfccd schema 0 value STRING properties bootstrap.servers <broker_list> value.format raw sasl.mechanism PLAIN sasl.username <username> security.protocol SASL_SSL topic source_raw target state RUNNING actual state RUNNING create time 2021-11-23T15:15:02Z update time 2021-11-23T15:15:02Z
Run a preview to read from the stream the source Kafka connection writes into. If you produce raw strings to the
source_raw topic, you should see sample data coming out from the preview command output.
Note that it may take up to 1 minute for the data to show up.
decodable pipeline preview "SELECT * FROM kafka_sasl_in"
Updated about 1 year ago