Tips and tricks

Abbreviations and aliases

While the CLI has descriptive names for commands such as connection and pipeline, it can become verbose once you’re accustomed to working with Decodable. Most commands support abbreviations and even aliases to make your life a little easier. All of these are listed in the help of each command. For example:

decodable connection -h
# Output:
# Manage connections
#
# Usage:
#   decodable connection [command]
#
# Aliases:
#   connection, conn
#
# ...

Note that conn is an alias for connection. Here are a few others:

Command or Subcommand Aliases

config

conf

connection

conn

pipeline

pl

list

ls

create

new

delete

rm

Using standard in/out

A few commands - especially those that deal with SQL statements and other long strings - support reading from and/or writing to standard in and out, respectively. This makes it easy to script complex operations in a "Unix-like" way.

For example, the pipeline command uses standard in. While you can provide the SQL directly on the command line,

# Providing SQL on the command line to `create`:
decodable pipeline create "INSERT INTO ... SELECT ..."

you can also use the literal string - (a single dash) to tell the CLI to read from standard in instead.

# A single dash tells the CLI to read from stdin.
cat somefile.sql | decodable pipeline create - --name my-pipeline

The config setup command is another example. Instead of writing the generated configuration to its default location, use -o - to write it to stdout instead!

decodable config setup my-account -o -

# Output:
# # Decodable configuration file.
# #
# # Generated by esammer, 2021-08-07T18:27:18-07:00
#
# version: 1.0.0
# active-profile: default
#
# profiles:
#   default:
#     account: my-account

Using multiple profiles

Working with multiple Decodable accounts is easy. The CLI supports storing and switching between profiles, each with their own configurations, including the account. You can also use the -p <profile> option to specify a profile for almost any command.

You can list all profiles and see which one is currently active (that’s used by default), using the config profile list command.

decodable config profile list

# Output:
# profile name         account              active
# dev                  my-account-dev       true
# prod                 my-account           false

You can edit the YAML configuration file with your favorite editor. It’s in ~/.decodable/config by default.

# Decodable configuration file.
#
# Generated by esammer, 2021-04-30T15:20:17-07:00

version: 1.0.0
active-profile: dev

profiles:
  dev:
    account: my-account-dev
  prod:
    account: my-account

By default, commands use whichever profile is marked as the active-profile, but you can override that using the -p <profile> argument to most commands.

# Uses the currently active profile (dev).
decodable pipeline create ...

# Use the prod profile instead.
decodable pipeline create -p prod ...

Metrics

Want to know how your job is doing? Pipelines and connections will output throughput metrics telling you how many records and bytes they’re currently processing, as well as how many they have processed in their lifetime of being activated.

decodable pipeline get 37c0ea06
# test
#   id                       37c0ea06
#   version                  1
#   is latest                true
#   target state             RUNNING
#   actual state             RUNNING
#   requested tasks          1
#   actual tasks             1
#   description              INSERT INTO envoy_processed
#   create time              2022-01-10T18:54:47Z
#   last runtime error       <none>
#   input metrics
#     envoy_raw              0.5   records / 106.8  B per second |   8.2 K records /   1.7 MB total
#   output metrics
#     envoy_processed        0.5   records / 106.8  B per second |   8.2 K records /   1.7 MB total
#
#   SELECT * FROM envoy_raw

For pipelines, metrics are available for each input or output stream. For connections, a single set of metrics is available for their throughput to their associated stream.