Tunnels and private connectivity

Popsink connectors are always the client. A connector worker pod opens an outbound TCP connection to your database and keeps it open; nothing in Popsink ever listens for a connection coming from your source. So the only networking question that ever matters is:

Can a pod in the Popsink namespace open a TCP socket to the source host and port?

If the answer is yes, there is nothing to configure. If the answer is no — because the database lives in a data centre, behind a corporate firewall, or in a VPC that is not the cluster’s — you need a tunnel. This page covers the three shapes we see in practice. This is a data plane concern only. The control plane never reaches into your network at all; see Control plane and data plane for why.

Choosing a shape

	IPsec site-to-site	SSH tunnel	Overlay network
Configured in	Your network infrastructure	The Popsink connector form	Your network infrastructure
Scope	Every pod, every connector, whole subnets	One connector, one TCP port	Every pod on the tailnet
Operated by	Your network team	Whoever creates the connector	Platform / infra team
Source stays private	Yes	Yes	Yes
Inbound firewall rule needed	Yes, on the peer gateway	Yes, to the bastion	None — both ends dial out
Typical time to set up	Days — change control, two teams	Minutes	Hours
Good for	Production, many sources, on-prem data centres	A single database behind an existing bastion	Networks you cannot re-architect, or where no inbound rule will ever be approved

A rule of thumb: IPsec if the source is a data centre and your network team is willing, SSH if the source is a single database with a bastion already in front of it, an overlay if no inbound rule is ever going to be approved. They also compose — an IPsec tunnel to the data centre plus an SSH tunnel to one stubborn host inside it is a perfectly normal end state. Whichever you pick, the tunnel only has to carry the database port. Popsink needs no inbound path, no reverse tunnel and no callback.

IPsec site-to-site VPN

The classic enterprise answer: a permanent encrypted tunnel between the network the Popsink data plane runs in and the network the source lives in, terminated by a gateway on each side. Once it is up, the source is simply routable and connectors are configured with its private address as if it were local.

┌───────────────────────────┐                      ┌──────────────────────────┐
│  Popsink data plane       │                      │  Your data centre        │
│  VPC / cluster            │   IKEv2 + ESP        │                          │
│  10.20.0.0/16             │◄════════════════════►│  10.90.0.0/16            │
│    connector worker pods  │   UDP 500 / 4500     │    db.internal:1521      │
└───────────────────────────┘                      └──────────────────────────┘

What to agree on before anything is built

IPsec fails at the first mismatched parameter, and the error messages are famously unhelpful. Fill this table in with your counterpart before touching a gateway.

Parameter	Notes
Peer public IPs	One per side, static. Dynamic IPs need IKEv2 with a FQDN identity
IKE version	IKEv2. Only use IKEv1 if the remote appliance genuinely cannot do v2
Phase 1 (IKE) proposal	e.g. `aes256gcm16-prfsha384-ecp384`; a common conservative fallback is `aes256-sha256-modp2048`
Phase 2 (ESP) proposal	e.g. `aes256gcm16-ecp384`; enable PFS
Lifetimes	Phase 1 typically 8 h, phase 2 typically 1 h. They must match, or you get periodic silent drops
Authentication	Pre-shared key or certificates. If PSK, exchange it out of band, never in a ticket
Local traffic selector	The CIDR Popsink egresses from — see the routing gotcha
Remote traffic selector	The source subnet, or a single `/32` if the remote team prefers to scope tightly
DPD	Dead Peer Detection on, `restart` action, so the tunnel re-establishes itself

Keep the traffic selectors as narrow as both sides can live with. A /32 on the remote side and a single egress address on ours is a much easier security review than two /16s.

The routing gotcha

This is the step that goes wrong on almost every first attempt. Connector workers are pods, so their source address is a pod IP from the cluster’s CNI range — not a node IP and not something the remote side has ever heard of. Two consequences:

If you declare the pod CIDR as the local traffic selector, the remote firewall team has to accept a range that means nothing to them, and overlapping RFC 1918 space becomes likely.
If you declare the node or subnet CIDR instead, traffic must be SNAT’d to that address on the way out, or it will leave with a pod IP that the tunnel’s policy drops.

The clean answer on managed Kubernetes is to give the cluster a stable egress address and present that to the tunnel:

AWS
Azure
GCP
Self-managed (strongSwan)

Put the node group in private subnets whose route table sends 0.0.0.0/0 at a NAT gateway with a fixed Elastic IP, and attach the Site-to-Site VPN to the same VPC. The remote selector is then the NAT gateway’s EIP or the private subnet CIDR, and traffic is SNAT’d for you.

Use a NAT Gateway with a static public IP on the AKS node subnet, or the userAssignedNATGateway outbound type, then terminate the connection on a VPN Gateway in the same VNet.

Run strongSwan on a dedicated gateway VM in the same network as the cluster, route the source subnet at it, and MASQUERADE on the way out:

iptables -t nat -A POSTROUTING -s <pod-cidr> -d 10.90.0.0/16 \
  -j SNAT --to-source <gateway-private-ip>

Check for CIDR overlap first. If your data centre and the cluster VPC both use 10.0.0.0/16, no amount of IPsec configuration will fix it — one side has to renumber or you have to NAT the remote subnet into a spare range. Confirm this on day one, not after the change window.

MTU

ESP encapsulation adds overhead, and a database connection that completes its handshake then hangs on the first large result set is almost always a fragmentation problem. Clamp TCP MSS on the gateway:

iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN \
  -j TCPMSS --clamp-mss-to-pmtu

Cloud-managed gateways do this themselves; self-managed strongSwan does not.

Verifying

From inside the cluster, before you create the connector:

kubectl -n popsink run nettest --rm -it --restart=Never \
  --image=nicolaka/netshoot -- \
  nc -vz db.internal 1521

A succeeded here means every connector in that namespace will be able to connect. A timeout means routing or firewall; a connection refused means the tunnel is fine and the database is not listening where you think it is.

SSH tunnel

The only pattern Popsink implements itself. When the source sits behind a bastion (jump host) that you already expose, the connector opens an SSH session to the bastion and forwards the database port through it — no change to your network, no gateway, no ticket to the network team. It is configured per connector, in the connector form, and is supported by:

PostgreSQL

MySQL

Oracle

SQL Server

IBM i (DB2)

Configuration

Field	Required	Description
SSH Host	Yes	Hostname or IP of the bastion host
SSH Port	Yes	SSH port (default `22`)
SSH User	Yes	User to authenticate as on the bastion
SSH Private Key	Yes	Base64-encoded private key for the SSH user

The Host and Port of the connector itself stay as the database sees them from the bastion — usually a private address such as 10.90.4.12 or db.internal, not something resolvable from the cluster.

Preparing the key

Generate a key pair dedicated to Popsink — never reuse an operator’s personal key — and leave it without a passphrase, since no one is present to type one when a connector restarts.

# 1. Dedicated key pair, no passphrase
ssh-keygen -t ed25519 -N "" -C "popsink-connector" -f ./popsink_tunnel

# 2. Install the public key on the bastion
ssh-copy-id -i ./popsink_tunnel.pub popsink@bastion.example.com

# 3. Base64-encode the private key for the connector form
base64 -w0 ./popsink_tunnel        # Linux
base64 -i  ./popsink_tunnel        # macOS

Paste the single-line output of step 3 into SSH Private Key. It is stored encrypted at rest with your deployment’s connectorConfigEncryptionKey, like every other connector credential.

Hardening the bastion account

The Popsink account needs to forward one port and nothing else. Express that in authorized_keys rather than trusting it to behave — this account cannot then open a shell, forward anything but the database, or be used to pivot:

restrict,port-forwarding,permitopen="10.90.4.12:1521",from="203.0.113.7" ssh-ed25519 AAAA... popsink-connector

restrict turns everything off — PTY, agent forwarding, X11, ~/.ssh/rc, and all port forwarding — and keeps doing so for any option OpenSSH adds later;
port-forwarding puts back only forwarding, which restrict had just removed. Omitting it is the usual mistake: permitopen narrows forwarding, it does not re-enable it, so the tunnel fails with administratively prohibited;
permitopen then narrows that forwarding to exactly one destination;
from pins the cluster’s egress address (see the NAT gateway discussion above — you need a stable egress IP for this to be usable).

Give the account no shell either (/usr/sbin/nologin) so the key is good for the forward and nothing else.

Limitations

One connector, one tunnel

Each connector establishes its own SSH session. Ten connectors behind the same bastion means ten concurrent sessions from the same account — make sure MaxSessions and MaxStartups in sshd_config are not set low enough to start refusing them.

Single hop only

The connector forwards through one bastion. A chain of two jump hosts is not expressible in the form; put a ProxyJump on the bastion side, or use an IPsec tunnel instead.

Passphrase-protected keys do not work

Connector workers restart unattended — on upgrade, on node drain, on failure. A key that needs a human is a connector that stops replicating at 3 a.m.

Key rotation is a connector edit

Rotating the key pair means updating the connector configuration. Plan for a short pause on that connector; replication resumes from its last offset, so nothing is lost.

Overlay and gateway services — Tailscale and ngrok

These sidestep the network team entirely: both ends dial out to a coordination service and the source becomes privately routable on a virtual network. No inbound firewall rule, no gateway appliance, no static IP, and in both cases the source is never published on the public internet. That makes them the practical answer when the source sits in a network you cannot re-architect — a customer’s data centre, an acquisition still on its own infrastructure, anywhere an inbound rule would take a quarter to approve.

Tailscale (WireGuard mesh)

The usable pattern is a subnet router: one host inside the source network advertises the database subnet to the tailnet, and the Popsink cluster joins the same tailnet and accepts those routes.

Advertise the source subnet

On a small Linux host inside the source network:

tailscale up \
  --advertise-routes=10.90.0.0/16 \
  --advertise-tags=tag:db-subnet-router \
  --authkey="${TS_AUTHKEY}"

Approve the advertised route in the Tailscale admin console, or set autoApprovers in the ACL so it does not need a human.

Join the cluster to the tailnet

Install the Tailscale Kubernetes operator in the cluster and expose the source as a Service that connectors resolve normally. An ephemeral, tagged auth key is the right credential here — not a personal one, which disappears when its owner leaves.

Lock it down with ACLs

The default tailnet policy is allow-all. Replace it so the Popsink tag can reach the database port and nothing else:

{
  "acls": [
    {
      "action": "accept",
      "src":    ["tag:popsink"],
      "dst":    ["10.90.4.12:1521"]
    }
  ]
}

Point the connector at the private address

10.90.4.12:1521 — the same address the subnet router sees. Verify with the netshoot pod from the IPsec section before creating the connector.

Tailscale keys expire. A node whose key expires drops off the tailnet and every connector through it stalls. Use tagged nodes (whose keys do not expire by default) or put the renewal in your calendar.

ngrok (Gateway, internal endpoints)

Use ngrok’s site-to-site connectivity, not the public ngrok tcp endpoint you may have seen in tutorials. The database is published as an internal endpoint — a .internal address that is not addressable from the public internet at all and can only be reached through a forward-internal traffic policy — and the cluster consumes it through a Kubernetes-bound endpoint that the ngrok Operator projects as an ordinary Service. The result is that a connector talks to a normal Kubernetes DNS name, and nothing is ever exposed publicly. Both ends only make outbound connections on TCP 443.

Publish the database as an internal endpoint

Run the agent next to the database, in the source network. The .internal suffix is what makes the endpoint private — internal domains are namespaced to your account, so the name is yours alone:

# ngrok.yml, on a host in the source network
version: 3
agent:
  authtoken: <AGENT_AUTHTOKEN>
endpoints:
  - name: Popsink source database
    url: tcp://source-db.acme.internal
    upstream:
      url: 1521

ngrok start --all

Install the ngrok Kubernetes Operator

In the cluster running the Popsink data plane. It dials out to ngrok on TCP 443 and needs no inbound rule.

Bind the endpoint into the cluster

A CloudEndpoint bound to kubernetes forwards to the internal endpoint. Name it for the namespace the connectors run in — the URL’s host and namespace parts become the Service:

apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: CloudEndpoint
metadata:
  name: source-db
  namespace: popsink
spec:
  url: tcp://source-db.popsink:1521
  bindings:
    - kubernetes
  trafficPolicy:
    policy:
      on_tcp_connect:
        - actions:
            - type: forward-internal
              config:
                url: tcp://source-db.acme.internal:1521

The operator projects an ExternalName Service called source-db in the popsink namespace pointing at its own forwarder.

Point the connector at the Service

Host source-db.popsink, port 1521 — the connector resolves it through cluster DNS like any in-cluster service, with no SSH tunnel and no VPN fields to fill in. Verify with the netshoot pod from the IPsec section first.

Because the binding is kubernetes, the endpoint is reachable only from inside clusters where the operator is installed — it is not public, and it is not reachable from elsewhere in your ngrok account either. Traffic policy also sees conn.k8s.pod.* metadata, so access can be scoped to specific workloads.

The trade-off to weigh at review time is not exposure, it is dependency: the data path runs through ngrok’s cloud, so ngrok availability becomes replication availability, and you are trusting their transport. Connections are end-to-end encrypted between the two agents, and enabling TLS on the database itself keeps the credentials and rows opaque to the transport regardless.

Firewall rules, by pattern

Pattern	Direction	Rule
IPsec	Popsink ↔ peer gateway	UDP 500 (IKE), UDP 4500 (NAT-T), ESP (IP protocol 50)
IPsec	Pod → source	Database port only, over the tunnel
SSH tunnel	Cluster egress → bastion	TCP 22 (or your SSH port), from the cluster’s egress IP
SSH tunnel	Bastion → database	Database port
Tailscale	Both ends → internet	TCP 443 to `controlplane.tailscale.com`, UDP 41641 outbound for direct connections
ngrok	Agent and operator → internet	TCP 443 outbound only, to `connect.ngrok-agent.com`

Note that the last two patterns need no inbound rule at all on either side — that is the whole reason to reach for them. In every row, the data plane still needs its own outbound HTTPS to the control plane — that requirement is unchanged and independent of the tunnel.

Troubleshooting

Symptom	Usual cause
Credential check times out	No route. Test with the `netshoot` pod; if that times out too, it is the tunnel, not Popsink
`connection refused`	Route works; the database is not listening on that address/port, or is bound to `127.0.0.1`
Works, then stalls on large tables	MTU / MSS clamping on the IPsec gateway
Connector drops every hour, to the minute	Mismatched phase 2 lifetimes, or DPD not set to `restart`
SSH tunnel fails right after it worked	Key rotated on the bastion, key expired, or `MaxStartups` refusing concurrent sessions
Everything dies at once, no config change	Tailscale key expiry, or the ngrok agent in the source network having stopped
ngrok Service resolves but connections hang	The `CloudEndpoint` traffic policy points at an internal endpoint whose agent is offline

Deployments and environments

One deployment per isolated network — why a tunnel is often the alternative to a second deployment.

Welcome

Platform Overview

Deployment

Source Connectors

Target Connectors

On-Premises & Integrations

Transformation

Changelog

Choosing a shape

IPsec site-to-site VPN

What to agree on before anything is built

The routing gotcha

MTU

Verifying

SSH tunnel

PostgreSQL

MySQL

Oracle

SQL Server

IBM i (DB2)

Configuration

Preparing the key

Hardening the bastion account

Limitations

Overlay and gateway services — Tailscale and ngrok

Tailscale (WireGuard mesh)

ngrok (Gateway, internal endpoints)

Firewall rules, by pattern

Troubleshooting

Deployments and environments

​Choosing a shape

​IPsec site-to-site VPN

​What to agree on before anything is built

​The routing gotcha

​MTU

​Verifying

​SSH tunnel

PostgreSQL

MySQL

Oracle

SQL Server

IBM i (DB2)

​Configuration

​Preparing the key

​Hardening the bastion account

​Limitations

​Overlay and gateway services — Tailscale and ngrok

​Tailscale (WireGuard mesh)

​ngrok (Gateway, internal endpoints)

​Firewall rules, by pattern

​Troubleshooting

Deployments and environments

Choosing a shape

IPsec site-to-site VPN

What to agree on before anything is built

The routing gotcha

MTU

Verifying

SSH tunnel

Configuration

Preparing the key

Hardening the bastion account

Limitations

Overlay and gateway services — Tailscale and ngrok

Tailscale (WireGuard mesh)

ngrok (Gateway, internal endpoints)

Firewall rules, by pattern

Troubleshooting