System Events and Metrics

Decodable generates data about the health, state, and throughput of its resources that is recorded into the _events and _metrics streams. You can use these streams to monitor the user activity and health of Decodable.

Events

Events generated by Decodable include:

  • Audit events of CRUD operations on your resources (connections, streams, and pipelines)
  • State transitions of active resources
  • Errors encountered by your active resources

Events are available in the _events stream and have the following schema.

NameTypeDescription
idSTRINGA unique identifier for the event.
account_idSTRINGThe account of the event.
resource_typeSTRINGThe type of resource that the event relates to.

Values are: connection, stream, pipeline, or table.
resource_idSTRINGA unique identifier for the resource the event relates to.
timestampBIGINTThe epoch timestamp of the event.
audit_event_typeSTRINGThe category of audit event.
messageSTRINGA human-readable message for the event.
propertiesMAP<STRING, STRING>Additional key-value properties for metadata associated with the audit event type.

A sample event

{
  "id": "f6a77bdb-eb41-4a18-b300-05450194dd0b",
  "account_id": "12345678",
  "resource_type": "stream",
  "resource_id": "14c78d3e",
  "timestamp": 1664213138882,
  "audit_event_type": "api_stream_create",
  "message": "Stream created",
  "properties": {
    "user_id": "google-oauth2|102883480266440686107"
  }
}

Metrics

Metrics for active connections and pipelines are available in the _metrics stream and have the following schema.

NameTypeDescription
timestampTIMESTAMP(3)The UTC time that the metric was emitted.
cloudSTRINGThe cloud provider where the resource is located (e.g. "aws").
regionSTRINGThe region where the resource is located (e.g. "us-east-1").
cellSTRINGThe Decodable cell where the resource is located. Each Decodable account belongs to one cell.
account_idSTRINGThe Decodable account that the resource belongs to.
resource_typeSTRINGThe type of resource the metric relates to (e.g. "pipeline").
resource_idSTRINGThe unique identifier of the resource the metric relates to.
resource_nameSTRINGThe name of the resource that the metric relates to.
metric_nameSTRINGThe name of the metric.
metric_valueDOUBLEThe value of the metric. If this value is non-numeric or undefined, then the metric_value is null.
propertiesMAP<STRING, STRING>Key-valued properties for additional metric dimensions. Keys are listed in the "Metric properties" table below.

Metrics are currently not supported for custom pipelines.

Available metrics

The following table describes the metrics that are available in the _metrics stream.

Metric NameDescription
bytes_in_totalThe total bytes received.
bytes_in_per_secondThe average number of bytes received per second.
bytes_out_totalThe total bytes sent.
bytes_out_per_secondThe average number of bytes sent per second.
parse_errors_per_secondThe average number of events which could not be parsed per second. Applicable to source connections only.
parse_errors_totalThe total number of errors which could not be parsed.
records_in_per_secondThe average number of events received per second.
records_in_totalThe total events received.
records_lag_maxThe number of records that are ready for processing, but have not been processed.
records_lag_totalThe total number of unconsumed records across all partitions.
records_out_per_secondThe average number of events sent per second.
records_out_totalThe total events sent.

Metric properties

The following table describes the properties that are available in a metric.

Property KeyPresent WhenDescription
connection_directionresource_type is connectionWhether the connection is a source or sink.
connectorresource_type is connectionThe type of the connector, for example "kafka" or "postgres".
pipeline_versionresource_type is pipelineThe version of the pipeline.
stream_directionmetric relates to I/O against a single streamWhether the stream is being used as a source or a sink by the resource.
stream_idmetric relates to I/O against a single streamThe id of the stream.
stream_namemetric relates to I/O against a single streamThe name of the stream.
stream_typemetric is attributable to a single streamThe type of the stream. This can be either "append" or "change".
task_indexmetric is attributable to a single taskThe task number of the task producing the metric. If only one task is used, then this number is "0".

๐Ÿ“˜

Metrics reset on connection or pipeline restarts

All of the *_totalmetrics (such as bytes_in_total) reset each time a connection or pipeline restarts. This means you should not use these metrics to compare the total number of records or bytes processed with external systems or across Decodable resources.

A sample metric

{
  "account_id": "00000000",
  "resource_type": "pipeline",
  "resource_id": "b672bf30",
  "resource_name": "pipeline_name",
  "metric_name": "bytes_in_per_second",
  "metric_value": 83.00278357689632,
  "timestamp": "2023-01-25 23:26:46.816",
  "cloud": "aws",
  "region": "us-east-1",
  "cell": "2",
  "properties": {
    "pipeline_version": "1",
    "stream_name": "envoy_raw",
    "stream_id": "1c724825",
    "task_index": "0",
    "stream_direction": "source",
    "stream_type": "append"
  }
}