Helm chart reference

The reference for the data-plane Helm chart. If you are installing for the first time, follow Install the data plane on Kubernetes instead — it walks the one path that works. This page is what you reach for when the wizard’s output is not enough: bringing your own Kafka, schema registry or PostgreSQL, tuning resources, or managing secrets out of band. The chart is published as an OCI artifact:

oci://ghcr.io/popsink/charts/data-plane

A full, annotated list of every value (with @param blocks) ships inside the chart’s own values.yaml, and the control-plane wizard links the reference values file for the version it serves you.

Components

Component	Purpose	Default	Replaceable by
`data-plane`	Main API + UI. Talks to the control plane, launches connector workers as pods	enabled	—
`kodansu`	Stateless Kafka-compatible broker, object-store backed	enabled	`defaultKafka.*`
`kora`	Confluent-compatible schema registry	enabled	`schemaRegistry.*`
`kotatsu`	Optional topic / schema browser over the broker’s S3 objects	disabled	—
`postgresql`	Bitnami sub-chart, in-cluster Postgres for the data-plane database	enabled	`externalDatabase.*`

Connector workers are not part of the chart: the data plane creates them as pods in its own namespace at runtime.

kodansu was previously called tansu, and the chart still reads a tansu: block for backwards compatibility (anything under kodansu: wins). New values files should use kodansu:.Two components that appeared in earlier versions of this page — karapace and kafka-ui — no longer exist in the chart. They were replaced by kora and kotatsu respectively. A values.yaml that still sets karapace.* or kafkaUi.* is silently ignored.

Identity

These come from the control plane (Deployments → New deployment → Self-hosted) and must be used verbatim — they are how the data plane authenticates itself.

controlPlaneUrl:         https://control-plane-api.popsink.com/api
controlPlaneFrontendUrl: https://control-plane.popsink.com
deploymentMode: SELF_HOSTED       # or STANDALONE — no control plane at all
deploymentId:   <uuid>
deploymentJwtToken:
  token: <jwt>                    # an object, not a string

ingressUrl: https://popsink.your-company.com

deploymentMode: STANDALONE disables every control-plane call, including the heartbeat, and requires no deploymentJwtToken. It is the mode behind air-gapped installs — see Standalone for what you give up.

ingressUrl must match the public URL the control plane redirects users to — it is also used for OAuth-style callbacks. A mismatch shows up as a login redirect loop.

Secrets you must generate

Beyond the values handed by the control plane, four secrets are generated locally. The wizard mints three of them in your browser; installing by hand means producing them yourself.

Secret	Format	How to generate
`adminCredentials.username` / `password`	any	Pick a strong password
`jwt.secret`	random string ≥ 32 chars	`openssl rand -base64 48`
`connectorConfigEncryptionKey.key`	URL-safe base64-encoded 32-byte Fernet key	`python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"`
`postgresql.auth.password` (or `externalDatabase.password`)	any	`openssl rand -base64 24`

These four secrets encrypt connector credentials, sign user sessions and protect the admin login. Lose them and you lose access to your stored connector configs. Back them up in your secret manager.connectorConfigEncryptionKey.key must never be rotated: every stored connector configuration is encrypted with it and becomes unreadable if it changes. Each deployment gets its own — never share one across deployments.

Managing secrets with `existingSecret`

Any sensitive value can come from an existing Kubernetes Secret instead of a literal. Set the matching existingSecret field and leave the literal unset — this works with Vault, Sealed Secrets and the External Secrets Operator, and is the recommended shape in production. Available for deploymentJwtToken, jwt, connectorConfigEncryptionKey, adminCredentials, kora.database, kodansu.storage.aws, externalDatabase and postgresql.auth. The sources are mutually exclusive: an existingSecret combined with a literal password or url is rejected at render time.

Image pull

All images are published to a private GAR registry:

europe-west1-docker.pkg.dev/popsink-common-438615/onprem

Popsink provides a service-account JSON token to pull them.

Chart-managed pull secret
Bring your own pull secret

imagePullSecret:
  create: true
  registry: europe-west1-docker.pkg.dev/popsink-common-438615/onprem
  token: |
    <base64-encoded service account JSON, or the JSON itself>

global:
  imagePullSecrets:
    - <release-name>-regcred

global.imagePullSecrets is required alongside imagePullSecret.create. The chart creates the secret but does not attach it to any pod, so without it every image fails with ImagePullBackOff. This is a known chart defect.

kubectl create secret docker-registry popsink-registry \
  --docker-server=europe-west1-docker.pkg.dev \
  --docker-username=_json_key \
  --docker-password="$(cat key.json)" \
  -n popsink

global:
  imagePullSecrets:
    - popsink-registry

Data-plane database

In-cluster (default)
External (recommended in production)

postgresql:
  enabled: true
  auth:
    password: <strong-password>
  primary:
    persistence:
      size: 50Gi
      storageClass: <your-ssd-class>

Fine for evaluation. In production, prefer a managed instance with snapshots.

postgresql:
  enabled: false

externalDatabase:
  host: popsink.abc123.eu-west-2.rds.amazonaws.com
  port: 5432
  user: popsink
  database: popsink
  password: <strong-password>

externalDatabase.user and externalDatabase.database are not guarded by the chart: left empty they render, install, and the data plane then crash-loops on an empty DB_USER. The database must exist before the install — the data plane runs its own migrations on startup.

Schema registry (Kora)

Kora is Popsink’s schema registry: every connector registers the structure of what it reads there, and every target resolves schemas from it. It is enabled by default and it requires its own PostgreSQL role and database, which the chart does not create for you.

This is the step most first installs miss. The bundled PostgreSQL provisions the data-plane’s own role and database (postgresql.auth.*) and nothing else, so leaving kora.database unset points Kora at a role that exists nowhere. From chart 0.1.0-alpha.162 the install aborts with the instructions below; on earlier charts it installs a pod that crash-loops on:

failed to connect to database: Backend("error returned from database:
password authentication failed for user \"kora\"")

Dedicated database (recommended)
Share the data-plane database
Bring your own registry

Run once against your PostgreSQL instance — managed services such as RDS or Cloud SQL included:

CREATE ROLE kora LOGIN PASSWORD 'a-strong-password';
CREATE DATABASE kora OWNER kora;

kora:
  database:
    host: "popsink.abc123.eu-west-2.rds.amazonaws.com"   # bundled: "<release>-postgresql"
    port: 5432
    user: "kora"
    database: "kora"
    password: "a-strong-password"

No SQL to run — point Kora at the same role and database as the data plane. This keeps a self-hosted install to a single database to operate, and is what the control-plane wizard generates.

kora:
  database:
    host: "<same as externalDatabase.host, or <release>-postgresql>"
    port: 5432
    user: "popsink"      # same as externalDatabase.user / postgresql.auth.username
    database: "popsink"  # same as externalDatabase.database / postgresql.auth.database
    password: "<same password>"

If you already operate a Confluent-compatible schema registry:

kora:
  enabled: false

schemaRegistry:
  url: "https://schema-registry.example.com"
  username: "popsink"
  password: "<secret>"
  # or: existingSecret + secretKeys.{username,password}

Credentials can also come from a secret rather than a literal — use kora.database.existingSecret with secretKeys.password, or secretKeys.url for a full connection URL. Kora also supports an Oracle backend: kora.database.backend: oracle, port 1521, and database set to the service name (e.g. FREEPDB1). The chart selects the matching image automatically.

Broker storage (Kodansu)

Kodansu is a stateless Kafka broker that stores log segments in an object store. kodansu.storage.engine decides where records live, and both it and kodansu.storage.aws.region are required by the chart — the region on every path, including the ones that never read it.

kodansu:
  storage:
    engine: "s3://my-popsink-bucket/kafka/"
    aws:
      region: eu-west-1
      irsaRoleArn:    arn:aws:iam::123456789012:role/popsink-kodansu
      assumedRoleArn: arn:aws:iam::123456789012:role/popsink-kodansu

kodansu:
  storage:
    engine: "s3://my-popsink-bucket/kafka/"
    aws:
      region: eu-west-1
      accessKeyId: AKIA…
      secretAccessKey: <secret>

kodansu:
  storage:
    engine: "s3://my-popsink-bucket/kafka/"
    aws:
      region: us-east-1
      endpoint: http://minio.minio.svc.cluster.local:9000
      allowHttp: true
      accessKeyId: minio
      secretAccessKey: minio123

kodansu:
  storage:
    engine: "abfss://popsink@mystorage.dfs.core.windows.net/kafka/"
    aws:
      # Required by the chart even here, and never read on this path.
      region: us-east-1
  podLabels:
    azure.workload.identity/use: "true"
  serviceAccount:
    annotations:
      azure.workload.identity/client-id: <managed-identity-client-id>

The bucket must exist, be writable by the configured identity, and ideally have object versioning enabled — on S3. On ADLS Gen2 versioning is the opposite instruction; see below.

There is no PostgreSQL retention. The broker has been object-store only since Kodansu 0.7.0-beta.11, and a postgres:// storage engine is now refused as an unsupported storage URL rather than ignored — the pods do not start. The self-hosted wizard still offers the choice and still generates that URL; do not take it. postgresql.* and externalDatabase.* are a different thing entirely: the data plane’s own database, which every install needs.

Azure (ADLS Gen2)

Supported from Kodansu 1.0.0-alpha.17. The install recipe, with the credential variants side by side, is on the Kubernetes install page; what follows is what the chart cannot express.

Accepted URL forms

Form	When
`abfss://<container>@<account>.dfs.core.windows.net/<prefix>`	Canonical. What an ADLS Gen2 user already writes, and it carries the account
`az://<container>/<prefix>`	Short form; the account comes from `AZURE_STORAGE_ACCOUNT_NAME`
`abfs://<container>@<account>.dfs.core.windows.net/<prefix>`	The same Hadoop convention over HTTP

wasbs://, adl://, azure:// and a raw https://<account>.blob.core.windows.net/ are refused, even though the underlying object-store parser understands them: every extra alias is another spelling a deployment can drift onto, and wasbs:// is the legacy Blob scheme — it should fail loudly rather than be treated quietly as Gen2. A dfs host is resolved to <account>.blob.core.windows.net: the broker speaks the Blob endpoint against an ADLS Gen2 account, never the DFS one. Every kodansu.storage.tuning knob is scheme-independent and applies unchanged.

Credentials

The broker reads them from the environment only, via the object-store client’s own resolution. The chart has no kodansu.storage.azure block, so anything other than workload identity goes in through kodansu.extraEnvVarsSecret, extraEnvVarsCM or extraEnvVars.

Mechanism	Variables
Workload identity federation	Injected by the AKS webhook — `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_AUTHORITY_HOST`, `AZURE_FEDERATED_TOKEN_FILE`
Account key	`AZURE_STORAGE_ACCOUNT_NAME`, `AZURE_STORAGE_ACCOUNT_KEY`
Client secret	`AZURE_STORAGE_CLIENT_ID`, `AZURE_STORAGE_CLIENT_SECRET`, `AZURE_STORAGE_TENANT_ID`
Managed identity	`AZURE_STORAGE_CLIENT_ID` (optional; IMDS otherwise)

Whichever you use, the identity needs Storage Blob Data Contributor on the account or container. Owner is an ARM role and grants no data-plane access whatsoever; it fails as a 403 AuthorizationFailure that names no role.

Account settings are part of the contract

The object layout is create-only and immutable, and retention is implemented by delete. Four account settings therefore are not preferences:

Setting	Required	Why
Blob versioning	off	Every superseded write is retained and billed. The broker keeps working, so this fails as a bill, not an error
Blob soft delete	off, or minimal	Supported under hierarchical namespace, but a retention delete then frees no space
Immutability / legal hold	absent	It breaks the delete path outright
Access tier	hot	Cool and cold add per-GB read charges this access pattern pays constantly

A freshly created StorageV2 account with nothing but --enable-hierarchical-namespace true --access-tier Hot already satisfies all four. Read the table as “do not turn these on” rather than “remember to turn these off”: the deployment that gets it wrong is the one that hardened an existing data-lake account. None of the four is detectable from the data plane, so there is no startup warning for any of them. Hierarchical namespace is the target. Deleting every blob under a prefix leaves the directories behind — not billed for capacity, absent from the listings the broker issues, untidy and nothing more. Flat Blob Storage is untested.

Private endpoints

Azure’s guidance for an ADLS Gen2 account is a private endpoint for both the blob and dfs sub-resources, because managing ACLs and creating or deleting directories need the DFS one. The broker does none of those three, so blob alone is sufficient. The failure worth knowing is the other direction: a network team that provisioned dfs only — the usual default for a data-lake account — leaves the broker unable to connect, with nothing in the error naming the sub-resource. If abfss:// times out or is refused while the account resolves, check this first.

Cost and Kotatsu

At the shape of a measured production S3 fleet, ADLS Gen2 comes to roughly 1.11× the S3 request bill, almost entirely the ~30 % hierarchical-namespace write premium. Two differences worth knowing: a list is billed as a read here (≈9× cheaper than on S3, where LIST is priced with PUT), and read/write transactions are metered per 4 MiB — so every 16 MiB merged segment write bills four times. Deletes are free, exactly as on S3. Kotatsu has no ADLS client: it reads S3 objects directly. Keep kotatsu.enabled: false on this path.

Bring your own Kafka

kodansu: { enabled: false }
kora:    { enabled: false }

defaultKafka:
  bootstrapServer: kafka.example.com:9093
  securityProtocol: SASL_SSL
  saslMechanism: SCRAM-SHA-512
  saslUsername: popsink
  saslPassword: <secret>
  caCert: |-
    -----BEGIN CERTIFICATE-----
    …
  cert: ""
  key:  ""

schemaRegistry:
  url: https://schema-registry.example.com
  username: popsink
  password: <secret>

Stream browser (Kotatsu)

Kotatsu is optional and disabled by default. It is a read-only browser over the objects the broker wrote to S3, and it backs two things in the UI:

search, pagination and the timestamp column on the datamodel Stream tab;
the per-subscription target lag rows on the pipeline latency view.

Without it, the Stream tab falls back to a live Kafka read (no search, no pagination) and the target-lag rows are simply absent. Pipelines, connectors, subscriptions and delivery are unaffected — it is safe to leave disabled in production. Because it reads the bucket the broker writes, it only means anything on an s3:// engine — it has no ADLS client. To enable it:

kotatsu:
  enabled: true

That is the entire configuration. Bucket, cluster id, region, endpoint and credentials all default to the broker’s own (kodansu.storage.*, kodansu.clusterId) — the only combination that can work, since the two read and write the same objects. Set kotatsu.s3.* only to point Kotatsu at a replica of the bucket or at a separate read-only credential. On EKS with IRSA, leave the keys empty and annotate kotatsu.serviceAccount.annotations with the role ARN.

On charts before 0.1.0-alpha.162 Kotatsu defaulted to enabled, so an install that had not opted into S3 aborted with kotatsu.s3.bucket must be provided when kotatsu is enabled. Setting kotatsu.enabled: false is the correct workaround on those versions.

Ingress

ingress.enabled is off by default and we recommend leaving it that way: ingress in production usually needs company-specific annotations (cert-manager, WAF, allow-lists). Both shapes — chart-rendered and bring-your-own — are covered in step 4 of the install guide.

Value	What it does
`ingress.enabled`	Render an `Ingress` resource (default `false`)
`ingress.hostname`	Host to serve on
`ingress.ingressClassName`	`nginx`, `traefik`, `istio`…
`ingress.tls`	Enable TLS on the rendered resource
`ingress.annotations`	Passed through — cert-manager issuer, WAF, allow-lists
`ingress.extraHosts` / `extraTls`	Additional hosts and TLS blocks
`ingress.path`	Path prefix (default `/`)
`ingress.existingSecret`	Use an existing TLS secret rather than issuing one

The chart never installs an ingress controller. Bringing your own means pointing it at the <release>-data-plane service on port 80.

Resources and availability

Value	Default	What it does
`replicaCount`	`2`	Data-plane API replicas
`resources.requests` / `.limits`	small	Per-component CPU and memory — tune for your workload
`kodansu.replicaCount`	`3`	Broker replicas
`pdb.create`	`true`	PodDisruptionBudget — keeps ≥ 1 pod during drains
`autoscaling.hpa.*`	off	Horizontal pod autoscaling
`autoscaling.vpa.*`	off	Vertical pod autoscaling
`image.tag`	chart	Pin the data-plane image — never deploy `latest`

Safety switches

Value	Default	What it does
`allowDesignLogin`	`false`	Dev-only. Leave off.
`pipelineMode`	`false`	Dev-only. Leave off.

A complete `values.yaml`

values.yaml

# ───── Identity (from the control-plane wizard) ─────
controlPlaneUrl:         https://control-plane-api.popsink.com/api
controlPlaneFrontendUrl: https://control-plane.popsink.com
deploymentMode: SELF_HOSTED
deploymentId:   <uuid-from-wizard>
deploymentJwtToken:
  token: <jwt-from-wizard>

# ───── Public URL of this data plane ─────
ingressUrl: https://popsink.your-company.com

# ───── Image pull ─────
imagePullSecret:
  create: true
  registry: europe-west1-docker.pkg.dev/popsink-common-438615/onprem
  token: <gar-service-account-json>
global:
  imagePullSecrets:
    - <release-name>-regcred

# ───── Secrets you generated ─────
adminCredentials:
  username: admin
  password: <strong-password>
jwt:
  secret: <openssl rand -base64 48>
connectorConfigEncryptionKey:
  key: <fernet-key>

# ───── In-cluster Postgres (default) ─────
postgresql:
  enabled: true
  auth:
    password: <strong-password>
  primary:
    persistence:
      size: 50Gi
      storageClass: <your-ssd-class>

# ───── Schema registry database — NOT created by the chart ─────
# Run once:  CREATE ROLE kora LOGIN PASSWORD '<kora-password>';
#            CREATE DATABASE kora OWNER kora;
kora:
  database:
    host: <release>-postgresql        # or your RDS / Cloud SQL endpoint
    port: 5432
    user: kora
    database: kora
    password: <kora-password>

# ───── Broker + S3 ─────
kodansu:
  storage:
    engine: "s3://my-popsink-bucket/kafka/"
    aws:
      region: eu-west-1
      irsaRoleArn:    arn:aws:iam::123:role/popsink-kodansu
      assumedRoleArn: arn:aws:iam::123:role/popsink-kodansu

# ───── Optional stream browser (s3:// engines only) ─────
kotatsu:
  enabled: true

Most useful values

Path	What it does
`controlPlaneUrl` / `controlPlaneFrontendUrl`	Control plane API and UI URLs
`deploymentId`, `deploymentJwtToken.token`	Identity vs. the control plane
`deploymentMode`	`SELF_HOSTED` or `STANDALONE`
`ingressUrl`	Public URL of this data plane
`image.tag`	Pin the data-plane image version
`replicaCount`	Data-plane API replicas (default 2)
`resources`	Data-plane CPU / memory requests and limits
`imagePullSecret.*` / `global.imagePullSecrets`	How to authenticate to the GAR registry
`adminCredentials.*`	First admin login
`jwt.secret`	Signs user session tokens
`connectorConfigEncryptionKey.key`	Encrypts connector credentials at rest
`kodansu.enabled`, `kodansu.storage.*`	In-cluster broker or BYO Kafka; object-store backend
`kora.enabled` / `schemaRegistry.*`	In-cluster schema registry or BYO
`kora.database.*`	Required. Kora’s own PostgreSQL role and database
`kotatsu.enabled`	Optional stream browser (default `false`)
`postgresql.enabled` / `externalDatabase.*`	In-cluster Postgres or BYO
`pdb.create`	PodDisruptionBudget (default `true`)
`autoscaling.hpa.`, `autoscaling.vpa.`	HPA / VPA — disabled by default

Install the data plane on Kubernetes

The guided path, from the control-plane wizard to a Live deployment.

Troubleshooting

Symptom-to-cause table for the install and the first boot.

Kubernetes requirements

What the cluster, the network and your Popsink account must provide first.

Deployments and environments

How to map deployments onto your regions, networks and staging tiers.

Welcome

Platform Overview

Security

Deployment

Source Connectors

Target Connectors

Integrations

API

Transformation

Changelog

Components

Identity

Secrets you must generate

Managing secrets with `existingSecret`

Image pull

Data-plane database

Schema registry (Kora)

Broker storage (Kodansu)

Azure (ADLS Gen2)

Accepted URL forms

Credentials

Account settings are part of the contract

Private endpoints

Cost and Kotatsu

Bring your own Kafka

Stream browser (Kotatsu)

Ingress

Resources and availability

Safety switches

A complete `values.yaml`

Most useful values

Further reading

Install the data plane on Kubernetes

Troubleshooting

Kubernetes requirements

Deployments and environments

​Components

​Identity

​Secrets you must generate

​Managing secrets with existingSecret

​Image pull

​Data-plane database

​Schema registry (Kora)

​Broker storage (Kodansu)

​Azure (ADLS Gen2)

​Accepted URL forms

​Credentials

​Account settings are part of the contract

​Private endpoints

​Cost and Kotatsu

​Bring your own Kafka

​Stream browser (Kotatsu)

​Ingress

​Resources and availability

​Safety switches

​A complete values.yaml

​Most useful values

​Further reading

Install the data plane on Kubernetes

Troubleshooting

Kubernetes requirements

Deployments and environments

Components

Identity

Secrets you must generate

Managing secrets with `existingSecret`

Image pull

Data-plane database

Schema registry (Kora)

Broker storage (Kodansu)

Azure (ADLS Gen2)

Accepted URL forms

Credentials

Account settings are part of the contract

Private endpoints

Cost and Kotatsu

Bring your own Kafka

Stream browser (Kotatsu)

Ingress

Resources and availability

Safety switches

A complete `values.yaml`

Most useful values

Further reading