Datadog sink connector

Features

Connector name

datadog

Delivery guarantee

At least once

Supported task sizes

S, M, L

Multiplex capability

A single instance of this connector can write to multiple Datadog metrics. The metrics must share the same structure in the source Decodable stream. That is, the name and value fields must be the same for all metrics in the stream.

Supported stream types

Use the Datadog Connector to send data from Decodable to Datadog for downstream analytics.

You can use this connector to send data from the Decodable _metrics stream into Datadog in order to monitor the state and health of your Decodable connections and pipelines. For information about the type of metrics that Decodable tracks, see System Events and Metrics.

Configuration properties

Property Description Required Default

api-key

Your Datadog API key.

This must be provided as a secret resource.

You can generate an API key in Datadog by navigating to the Organization Settings under your account menu, and then choosing API Keys

Yes

site

The site to which you want to send data.

Must be one of the following values:

  • datadoghq.com

  • us3.datadoghq.com

  • us5.datadoghq.com

  • datadoghq.eu

  • ddog-gov.com

See Getting Started with Datadog Sites in the Datadog documentation for more information about sites.

Yes

metric-name-column

The field name in the input stream containing the metric name.

If you want to send data from the _metrics stream, enter metric_name.

Yes

metric-value-column

The field name in the input stream containing the metric value.

If you want to send data from the _metrics stream, enter metric_value.

Yes

timestamp-column

The field name in the input stream containing the timestamp.

If you want to send data from the _metrics stream, enter timestamp.

Datadog will reject data with a timestamp that’s older than one hour, or more than ten minutes in the future.

Decodable will still count these as having been processed. In this situation you will observe a difference between the number of records sent from Decodable and that of those loaded into Datadog.

Yes

tag-columns

Enter any additional field names that you’d like to include as tags on the metrics. If you want to send data from the _metrics stream, see Metrics for the field names that you can include.

static-tags

A comma-separated list of static key:value pairs that will be included as tags on the metrics.

metric-name-prefix

A prefix to add to the names of your metrics.

For example, for the metric records-in-total and a prefix myaccount_, the metric name you would see in Datadog would be myaccount_records-in-total.

Connector starting state and offsets

A new sink connection will start reading from the Latest point in the source Decodable stream. This means that only data that’s written to the stream when the connection has started will be sent to the external system. You can override this when you start the connection to Earliest if you want to send all the existing data on the source stream to the target system, along with all new data that arrives on the stream.

When you restart a sink connection it will continue to read data from the point it most recently stored in the checkpoint before the connection stopped. You can also opt to discard the connection’s state and restart it afresh from Earliest or Latest as described above.

Learn more about starting state here.