System events and metrics

Decodable generates data about the health, state, and throughput of its resources that’s recorded into the _events and _metrics streams. You can use these streams to monitor the user activity and health of Decodable.

Events

Events generated by Decodable include:

  • Audit events of CRUD operations on your resources (connections, streams, and pipelines)

  • State transitions of active resources

  • Errors encountered by your active resources

Events are available in the _events stream and have the following schema.

Name

Type

Description

id

STRING

A unique identifier for the event.

account_id

STRING

The account of the event.

resource_type

STRING

The type of resource that the event relates to. Values are: connection, stream, pipeline, or table.

resource_id

STRING

A unique identifier for the resource the event relates to.

timestamp

BIGINT

The epoch timestamp of the event.

audit_event_type

STRING

The category of audit event.

message

STRING

A human-readable message for the event.

properties

MAP<STRING, STRING>

Additional key-value properties for metadata associated with the audit event type.

A sample event

{
  "id": "f6a77bdb-eb41-4a18-b300-05450194dd0b",
  "account_id": "12345678",
  "resource_type": "stream",
  "resource_id": "14c78d3e",
  "timestamp": 1664213138882,
  "audit_event_type": "api_stream_create",
  "message": "Stream created",
  "properties": {
    "user_id": "google-oauth2|102883480266440686107"
  }
}

Metrics

Metrics for active connections and pipelines are available in the _metrics stream and have the following schema.

Name Type Description

timestamp

TIMESTAMP(3)

The UTC time that the metric was emitted.

cloud

STRING

The cloud provider where the resource is located. For example: "AWS".

region

STRING

The region where the resource is located. For example: "us-east-1."

cell

STRING

The Decodable cell where the resource is located. Each Decodable account belongs to one cell.

account_id

STRING

The Decodable account that the resource belongs to.

resource_type

STRING

The type of resource the metric relates to. For example: "pipeline."

resource_id

STRING

The unique identifier of the resource the metric relates to.

resource_name

STRING

The name of the resource that the metric relates to.

metric_name

STRING

The name of the metric.

metric_value

DOUBLE

The value of the metric. If this value is non-numeric or undefined, then the metric_value is null.

properties

MAP<STRING, STRING>

Key-valued properties for additional metric dimensions. Keys are listed in the "Metric properties" table below.

Available metrics

The following table describes the metrics that are available in the _metrics stream.

Metric Name Description

bytes_in_total

The total bytes received.

bytes_in_per_second

The average number of bytes received per second.

bytes_out_total

The total bytes sent.

bytes_out_per_second

The average number of bytes sent per second.

parse_errors_per_second

The average number of events which couldn’t be parsed per second. Applicable to source connections only.

parse_errors_total

The total number of errors which couldn’t be parsed.

records_in_per_second

The average number of events received per second.

records_in_total

The total events received.

records_lag_max

The highest number of messages currently unconsumed in a single partition.

Given the following stream:

Partition

Unconsumed messages

P0

10

P1

15

P2

12

The value of records_lag_max would be 15.

records_lag_total

The total number of messages currently unconsumed across all partitions.

In the above records_lag_max example, the corresponding records_lag_total value would be 37.

records_out_per_second

The average number of events sent per second.

records_out_total

The total events sent.

Metric properties

The following table describes the properties that are available in a metric.

Property Key Present When Description

connection_direction

resource_type is connection

Whether the connection is a source or sink.

connector

resource_type is connection

The type of the connector, for example kafka or postgres.

pipeline_version

resource_type is pipeline

The version of the pipeline.

stream_direction

metric relates to I/O against a single stream

Whether the stream is being used as a source or a sink by the resource.

stream_id

metric relates to I/O against a single stream

The id of the stream.

stream_name

metric relates to I/O against a single stream

The name of the stream.

stream_type

metric is attributable to a single stream

The type of the stream. This can be either "append" or "change."

task_index

metric is attributable to a single task

The task number of the task producing the metric. If only one task is used, then this number is "0."

All of the `*_total metrics (such as bytes_in_total) reset each time a connection or pipeline restarts. This means you shouldn’t use these metrics to compare the total number of records or bytes processed with external systems or across Decodable resources.

A sample metric

{
  "account_id": "00000000",
  "resource_type": "pipeline",
  "resource_id": "b672bf30",
  "resource_name": "pipeline_name",
  "metric_name": "bytes_in_per_second",
  "metric_value": 83.00278357689632,
  "timestamp": "2023-01-25 23:26:46.816",
  "cloud": "AWS",
  "region": "us-east-1",
  "cell": "2",
  "properties": {
    "pipeline_version": "1",
    "stream_name": "envoy_raw",
    "stream_id": "1c724825",
    "task_index": "0",
    "stream_direction": "source",
    "stream_type": "append"
  }
}