BYOC setup

Request BYOC from Decodable Support

New Decodable accounts are configured to be Fully Managed by default. Contact Decodable support to convert your account to BYOC.

Once your account has been converted you will be able to access the BYOC configuration page in the web app.

Decide how users will connect to the API

Decodable BYOC deployments include a REST API which is accessed by Decodable clients (for example, the web app and CLI) to configure connections and preview data.

There are several ways to configure connectivity from clients to the API.

On the internet

The API has built-in authentication, and can be exposed to the Internet. If the API is on the Internet you must configure a well-known DNS record and obtain a TLS certificate to secure the connection.

On a private VPC

The API can be configured to use a private LB within a VPC where your users have access via a VPN or similar tunnel. If the API is on a private VPC you must configure a well-known DNS record and obtain a TLS certificate to secure the connection.

Using localhost port-forwarding

In this deployment mode only users who are running the port-forward will be able to use certain Decodable features.

For proof-of-concept deployments it may be simpler to use kubectl port-forward or an SSH tunnel to connect to the API. The benefit of this deployment mode is that no TLS certificate or DNS record is required to get started.

Configuring DNS and TLS certificates

The specific steps to add a DNS record will vary depending on your organization.

To configure a TLS certificate one option is to use Amazon Certificate Manager and obtain a public certificate by doing a DNS or email challenge. The certificate ARN can be added to the Kubernetes annotations for the Ingress or Service.

Follow the setup process in the web UI

The Decodable Data Plane authenticates to the control plane using an mTLS key pair.

Sign in to the web UI and follow the instructions to:

  • Generate a new TLS key and Certificate Signing Request (CSR).

  • Create a Kubernetes secret containing the TLS key.

  • Upload the CSR file to the web UI and receive a signed certificate.

  • Download the reference Terraform and Helm charts.

Create cloud resources

Today AWS is the only cloud provider supported by Decodable BYOC. Support for other cloud providers will be added in the future.

Decodable BYOC deployments require a Kubernetes cluster, Apache Kafka and object storage. Decodable provides a list of necessary cloud resources and a reference Terraform implementation for each cloud provider.

Deploy these cloud resources and make note of their identifiers.

Configure the Helm values file

Configure the values file using the cloud resources and mTLS credentials created above.

An example values.yaml file:

global:
  namespace: "decodable"
  cloud:
    # The cloud region where the cluster is deployed
    region: "us-west-2"
  kafka:
    # The Kafka brokers IAM bootstrap string
    brokers: ""
    # The SASL/SCRAM bootstrap strings
    saslScramBrokers: ''
    # The ARN of the secret for SASL/SCRAM
    vectorSecretId: ''
    dataPlaneControllerSecretId: ''

  dataPlane:
    # The data plane ID from the Decodable web UI
    id: ""

  s3:
    # The names of the buckets where data and logs will be stored
    productBucket: ""
    productDebugBucket: ""
    vectorKubernetesLogBucket: ""

data-plane-controller:
  # The watchdog URL from the Decodable web UI
  controlPlaneURL: ''

  # The signed mTLS certificate provided by Decodable support
  controlPlaneMTLSCert: |

  # The ARN of the IAM role for the data-plane-controller
  serviceAccount:
    annotations:
      'eks.amazonaws.com/role-arn': ''

  # The ARN of the IAM role for Flink Java and Python pipelines
  flinkCustomPipelinesRole: ''

# The following values are the ARNs of AWS IAM roles
data-plane-api:
  serviceAcccount:
    annotations:
      'eks.amazonaws.com/role-arn': ''

flink-preview:
  serviceAccount:
    annotations:
      'eks.amazonaws.com/role-arn': ''

flink-app:
  serviceAccount:
    annotations:
      'eks.amazonaws.com/role-arn': ''

datagen:
  serviceAccount:
    annotations:
      'eks.amazonaws.com/role-arn': ''

vector:
  serviceAccount:
    annotations:
      'eks.amazonaws.com/role-arn': ''

Deploy the Helm charts

Install the Flink Kubernetes Operator Helm chart:

helm -n decodable create flink-kubernetes-operator flink-kubernetes-operator.tgz

Install the Decodable Data Plane Helm chart using the values file configured above:

helm -n decodable create decodable-data-plane decodable-data-plane.tgz -f values.yaml

Configure the Data Plane host

In the web UI configure the DNS address for the API. The UI will validate that the address can be resolved, and that the connection uses TLS.