Use the Google Cloud Pub/Sub (GCP Pub/Sub) to get data from Google Cloud into Decodable. If you are looking for instructions on how to send data from Decodable into Google Cloud, see Google Cloud Platform Pub/Sub in the Connect to a data destination chapter.

Overview


Connector name	`gcp-pubsub`
Type	`source`, `sink`
Delivery guarantee	at least once

Create a connection to Google Cloud Pub/Sub

Prerequisites

You must have the following.

A GCP service account key that can be used for authentication. You can create and download the key by browsing to the Keys tab under the Service Account details in GCP IAM.
A GCP subscription with the Pull delivery type.
The following GCP roles assigned to the service account principal:

GCP Resource Role 1 Role 2
Subscription Pub/Sub Viewer Pub/Sub Subscriber

GCP Resource	Role 1	Role 2
Subscription	Pub/Sub Viewer	Pub/Sub Subscriber

Steps

Follow these steps to send data from Google Cloud Pub/Sub into Decodable. These steps assume that you are using Decodable Web. However, if you want to use the Decodable CLI to create the connection, you can refer to the Property Name column for information about what the underlying property names are.

From the Connections page, select the GCP Pub/Sub connector and complete the following fields.

UI Field	Property Name in the Decodable CLI	Description
Connection Type	N/A	Set to Source to create a connection that gets data from GCP Pub/Sub into Decodable.
Project	project	The project name that the Pub/Sub resource is located in.
Subscription	subscription	The name of the subscription to read data from.
Service Key JSON Secret	service-key-json	The secret associated with your service account. If you are using the Decodable CLI, this is the ID of a secret resource in your account. Run `decodable secret list` to view available secrets or `decodable secret --help` for help with creating a new secret. Note: For security purposes, Decodable will never display secret values in plaintext. You can manage which users have permissions to create, delete, or modify secrets in the Access Control management view. See Manage Roles, Groups, and Permissions for more information.
Value Format	format	The format of the data in the stream. Must be `json`.
N/A	subscription.max-messages-per-pull	The maximum number of pulled messages per pull request. This property is only configurable through the Decodable CLI. Defaults to 100.
N/A	subscription.per-request-timeout-sec	Optional. The amount of time, in seconds, to wait before timing out. This property is only configurable through the Decodable CLI. Defaults to 15 seconds.
N/A	subscription.retries	Optional. The maximum number of subscription retry attempts. This property is only configurable through the Decodable CLI. Defaults to 3.

📘
Subscription Deadlines
Messages are acknowledged to Pub/Sub approximately every 10 seconds. Best practices are to set the subscription's acknowledgement deadline to be much larger, or else the messages may be processed multiple times.

Select the stream that you’d like to connect to this connector. Then, select Next.
Define the connection’s schema. Select New Schema to manually enter the fields and field types present or Structured Schema Definition if you want to paste the schema in the form of an Avro or JSON array.
1. The stream's schema must match the schema of the data that you plan on sending through this connection.
2. For more information about creating a stream or defining the stream schema, see Create a Stream.
Select Next when you are finished providing defining the connection’s schema.
Give the newly created connection a Name and Description and select Save.