Listing connections

Connections in Decodable ingest data from a data source or send data out to a data sink. You can find a full list of available connectors here, or in the navigation menu on the left.

There are four ways to manage connections in Decodable: the Decodable web app, the CLI, and the API. Within the CLI you may use an imperative approach, or a declarative YAML-based declarative approach.

  • Web App

  • CLI

  • Declarative Resource Management

  • API

  1. Navigate to the Connections page to view a list of connections.

    list connections

  2. From the list of connections shown, click on the connection for which you want to view details.

    connection details

  3. Use the tabs to switch between Overview, Schema, and Properties details for the connection.

To use the CLI, download it and then login to your account

Use decodable connection list to see all connections:

$ decodable connection list
id        name                               connector     type    state    create time           update time
64ae6213  my-datagen-connection              datagen       source  RUNNING  2024-09-26T12:49:09Z  2024-09-26T12:49:09Z

or decodable connection get to inspect a particular connection

$ decodable connection get 64ae6213
my-datagen-connection
  id                       64ae6213
  description              -
  connector                datagen
  type                     source
  stream id                ce4a6658
  fields
    0  message             STRING
  primary key fields       -
  properties
    data.type              envoy
    delay                  1000
    format                 json
  target state             RUNNING
  actual state             RUNNING
  requested tasks          1
  actual tasks             1
  requested task size      M
  actual task size         M
  create time              2024-09-26T12:49:09Z
  update time              2024-09-26T12:49:09Z
  last runtime error       -
  metrics

To find out more about managing connections with the CLI consult the relevant documentation or run decodable connection --help to see the list of available subcommands.

You can use YAML to define your resources, including connections, and manage them in a declarative way. This approach integrates seamlessly with CI/CD tools and allows for management of large-scale and complex resources. To learn more about using declarative resource management consult the documentation pages.

For connectors that support multiple streams the decodable scan command is an easy way to generate YAML files. You can also use decodable query to return the definitions of existing resources in YAML as a starting point for creating new ones.

The Decodable CLI provides access to the declarative resource management commands. Before using the examples below, download the Decodable CLI and login to your account.

To list connections using declarative resource management use the decodable query command in conjunction with a suitable filter to identify the connections you want to list.

For example, to list all connections:

$ decodable query --kind connection
---
kind: connection
metadata:
    name: my-datagen-connection
spec_version: v2
spec:
    connector: datagen
    type: source
    stream_name: my-datagen-connection_source
    schema_v2:
        fields:
            - kind: physical
              name: message
              type: STRING
        constraints: {}
    properties:
        delay: "2000"
        format: json
        data.type: envoy
    execution:
        active: true
        task_count: 1
        task_size: M
status:
    create_time: 2024-09-30T15:24:05.506+00:00
    update_time: 2024-09-30T15:26:19.999+00:00
    last_activated_time: 2024-09-30T15:26:19.999+00:00
    execution:
        state: RUNNING

To return a minimal view of the data use --metadata-only (or -M):

$ decodable query --kind connection --metadata-only
---
kind: connection
metadata:
    name: my-datagen-connection

You can combine the output of the decodable query command with the yq utility to filter the data further. To list only connections on your Decodable account using the datagen connector run:

decodable query --kind connection | \
        yq -r eval-all 'select(.spec.connector=="datagen") | .metadata.name '
my-datagen-connection
---
my-datagen-connection-01
---
my-datagen-connection-02

For more details and options, see the decodable query documentation.

Use a GET HTTP call to the connections Decodable control plane API endpoint.

You can also use a GET HTTP call to the connections/{id} Decodable control plane API endpoint to get the specific information for a given connection id.

The URL of the endpoint is based on your account name:

https://<account name>.api.decodable.co/v1alpha2/connections/

Below is an example of listing all connections under the account called acme-01 and using curl to make the HTTP call and jq to format the output.

You will need an access token in order to authenticate your REST call to the Decodable API endpoint. You can obtain this using the Decodable CLI's decodable token access command. For more details see here.

 
curl -X GET -s "https://acme-01.api.decodable.co/v1alpha2/connections/" \
     -H "Authorization: Bearer $(decodable token access)"  | jq '.'
{
  "id": "0abe42bf",
  "name": "my-datagen-connection",
  "description": null,
  "connector": "datagen",
  "type": "source",
  "stream_mappings": null,
  "stream_id": "ce4a6658",
  "create_time": "2024-09-26T13:16:51.244+00:00",
  "update_time": "2024-09-26T13:16:51.244+00:00",
  "schema": [
    {
      "name": "message",
      "type": "STRING"
    }
  ],
  "schema_v2": null,
  "schema_mapping": {},
  "properties": {
    "format": "json",
    "delay": "1000",
    "data.type": "envoy"
  },
  "target_state": "RUNNING",
  "actual_state": "RUNNING",
  "requested_tasks": 1,
  "actual_tasks": 1,
  "requested_task_size": "M",
  "actual_task_size": "M",
  "last_runtime_error": {
    "message": "",
    "raw_exception": "",
    "timestamp": null
  },
  "last_activated_time": "2024-09-26T13:16:51.244+00:00",
  "metrics": null
}