MongoDB sink connector

Use the MongoDB Connector to send data from Decodable to MongoDB. If you are looking for information about how to create a connection to get data from MongoDB into Decodable, see MongoDB source connector.

Features

Delivery guarantee

Exactly once

Supported stream types

append, change

Steps

If you want to use the Decodable CLI or API to create the connection, you can refer to the Property Name column for information about what the underlying property names are. The connector name is mongodb.
  1. From the Connections page, select the MongoDB Connector and complete the following fields.

    UI Field Property Name Description

    Connection String

    connection-string

    The connection string for your MongoDB cluster without your username and password. Decodable supports both the Standard Connection String and DNS Seed List Connection String formats.

    For example: mongodb(+srv)://my-cluster.mongodb.net/?retryWrites=true&w=majority

    Database

    database

    The name of the database containing the collection that you want to send data to.

    Collection

    collection

    The name of the collection to send data to.

    Username

    username

    Your MongoDB username.

    Password

    password

    The secret containing your MongoDB password.

  2. Select the stream that you’d like to connect to this connector. This will be the stream that sends records from Decodable into MongoDB. Then, select Next.

  3. Define the connection’s schema. Select New Schema to manually enter the fields and field types present or Import Schema if you want to paste the schema in the form of an Avro or JSON array.

    1. The stream’s schema must match the schema of the data that you plan on sending through this connection.

    2. For more information about creating a stream or defining the stream schema, see Create and manage Streams.

  4. Select Next when you are finished providing defining the connection’s schema.

  5. Give the newly created connection a Name and Description and select Save.

Start the connection and observe data flow into your MongoDB instance.

Connector starting state and offsets

A new sink connection will start reading from the Latest point in the source Decodable stream. This means that only data that’s written to the stream when the connection has started will be sent to the external system. You can override this when you start the connection to Earliest if you want to send all the existing data on the source stream to the target system, along with all new data that arrives on the stream.

When you restart a sink connection it will continue to read data from the point it most recently stored in the checkpoint before the connection stopped. You can also opt to discard the connection’s state and restart it afresh from Earliest or Latest as described above.

Learn more about starting state here.