Editing a connection Connections in Decodable ingest data from a data source or send data out to a data sink. You can find a full list of available connectors here, or in the navigation menu on the left. There are four ways to manage connections in Decodable: the Decodable web app, the CLI, and the API. Within the CLI you may use an imperative approach, or a declarative YAML-based declarative approach. Web App CLI Declarative Resource Management API Navigate to the Connections page. On the Connections page, select the row that corresponds to the connection that you want to edit. The Overview page for the connection opens. If the connection is running, stop the connection. You can’t edit a running connection. Update the connection as desired. The tabs that you see next to Overview depend on whether the connection supports connecting to multiple streams or not. If your connection supports reading from or writing to multiple streams: Select Details to update the configuration of the connection. You must confirm your stream mappings after making any changes to the connector configuration. Select stream Mappings to view or update the mappings between external tables and Decodable streams. Select Edit to add or remove mappings. An updated list of available tables and streams appears, and you can select or remove the mappings that you want managed by the connection. If you need to modify the schema of one of the streams, navigate to the streams page instead. See Schema migrations to learn about best practices when performing schema migrations. For a programmatic approach to this step, see declarative resource management and the connection scan command provided by the CLI. If your connection only supports reading from or writing to a single stream: Select Details to update the configuration of the connection. Select Schema to update the schema of records that are flowing through this connection. Remember that the connection schema must be compatible with the schema of the stream. You can also use the Schema page to update the fields used as a primary or partition key, or update a field that’s enabled as a watermark. See Schema migrations to learn about best practices when performing schema migrations. Select Save to save your changes. Select Start to resume the connection. A modal appears with some additional configuration. Configuration option Description Task Size The amount of resources, such as how much CPU or Memory, to allocate for streaming data. Task Count The number of connection workers that perform data collection or processing. Discard State Select whether you want to discard state. By selecting this option, you are reactivating the connection without attempting to restore state from a save point or checkpoint. If you want data ingestion to start from the earliest offset, select this option. Note: When supported, the configuration for where to resume data ingestion upon activating a source connection is managed by the Scan Startup Mode connection setting. For example, for the Confluent Cloud connector, you specify the point at which data ingestion should resume by configuring the connector’s Scan Startup Mode. Earliest/Latest Where to resume data ingestion when a sink connection is activated. Select Earliest to start processing data from the earliest record available or Latest to start processing data from the latest record available. Only applicable for sink connections. When you restart the connection you can choose where it should start processing from. See Connection starting state and offsets. To use the CLI, download it and then login to your account Use decodable connection list to see all connections: $ decodable connection list id name connector type state create time update time 64ae6213 my-datagen-connection datagen source RUNNING 2024-09-26T12:49:09Z 2024-09-26T12:49:09Z Use decodable connection update to amend the properties on a connection. For example, to change the delay property in the above connection run: decodable connection update 64ae6213 --prop delay=2000 To find out more about managing connections with the CLI consult the relevant documentation or run decodable connection --help to see the list of available subcommands. You can use YAML to define your resources, including connections, and manage them in a declarative way. This approach integrates seamlessly with CI/CD tools and allows for management of large-scale and complex resources. To learn more about using declarative resource management consult the documentation pages. For connectors that support multiple streams the decodable scan command is an easy way to generate YAML files. You can also use decodable query to return the definitions of existing resources in YAML as a starting point for creating new ones. The Decodable CLI provides access to the declarative resource management commands. Before using the examples below, download the Decodable CLI and login to your account. By definition, the process of editing a connection with declarative resource management is almost identical to creating one. YAML is used to define the connection resource, and then decodable apply makes the state of the resource in your Decodable account match. How you edit or update resource definitions depends on what you’re using as the source of truth for your resources: If local or git-resident YAML files, edit those, then decodable apply them. This would be the case if you’re following a CI/CD based process If the Decodable service, first fetch them via decodable query, edit them locally, then decodable apply them. You can edit resource YAML in an editor manually, or via an automated tool like yq. For example, to get the resource definition for a connection called my-datagen-connection run the following. $ decodable query --name my-datagen-connection --export > my-datagen-connection.yaml Edit the file as required per the changes you want to make to the connection. Re-apply the resource definition: $ decodable apply my-datagen-connection.yaml --- kind: connection name: my-datagen-connection id: b4e5ad46 result: updated Use a PATCH HTTP call to the /connections/{id} Decodable control plane API endpoint. The URL of the endpoint is based on your account name: https://<account name>.api.decodable.co/v1alpha2/connections/ Below is an example of editing a connection with id 0abe42bf under the account called acme-01 and using curl to make the HTTP call. You will need an access token in order to authenticate your REST call to the Decodable API endpoint. You can obtain this using the Decodable CLI's decodable token access command. For more details see here. curl -X PATCH "https://acme-01.api.decodable.co/v1alpha2/connections/0abe42bf" \ -H "Authorization: Bearer $(decodable token access)" \ -H 'Content-Type: application/json' \ -d '{ "properties": { "delay": "2000" } }' Consult the connector’s documentation page for details of the available connector properties.