Single VM Deployment

​

Overview

• Proof-of-concept and evaluation — get Popsink running in your environment quickly, without managing a distributed infrastructure.
• Lightweight workloads — small teams or low-traffic environments where horizontal scalability is not a requirement.

This deployment model is intentionally simple. For high-availability or large-scale production use, see the Kubernetes deployment instead.

​

VM Platform

​

Prerequisites

​

Hardware

Hypervisors that pre-allocate the full sparse image size require significantly more datastore space. Confirm the allocation behaviour of your target hypervisor before import.

​

External Dependencies

​

Supported Hypervisors

​

Getting the VM Image

sha256sum -c popsink-deploy-vm-<version>.<format>.sha256

​

Network Configuration

​

Inbound

​

Outbound

​

Outbound Proxy

​

Deployment

​

KVM/QEMU

1. Build the seed.iso configuration image first — see Cloud-Init below for step-by-step instructions.
2. Import the image and attach the seed.iso as a virtual CD-ROM in a single command. The example below uses virt-install; virsh and virt-manager are equally valid:
   virt-install \
     --name popsink \
     --memory 16384 \
     --vcpus 4 \
     --disk path=popsink-deploy-vm-<version>.qcow2,format=qcow2 \
     --disk path=seed.iso,device=cdrom,readonly=yes \
     --import \
     --os-variant generic \
     --noautoconsole
   
3. Boot the VM. Cloud-init reads the ISO on first boot and configures the data-plane automatically.

​

VMware vSphere

1. Build the seed.iso configuration image first — see Cloud-Init below for step-by-step instructions.
2. In the vSphere UI, right-click the target datacenter or cluster → Deploy OVF Template.
3. Follow the wizard:
   • At the Select networks step, map the VM’s network adapter to a port group that satisfies the inbound and outbound requirements listed above.
   • Confirm the hardware summary: 4 vCPU, 16 GB RAM, 20 GiB disk.
   • At the Select storage step, choose thin provisioning unless thick is required by your policy.
4. After the deploy task completes, do not boot the VM yet. Upload seed.iso to your vSphere datastore:
   • In the vSphere Client, navigate to the target datastore → Files tab → Upload.
   • Upload the seed.iso file you generated.
5. Attach seed.iso as a CD-ROM drive on the VM:
   • Right-click the VM → Edit Settings.
   • Click Add Other Device → CD/DVD Drive.
   • Select Datastore ISO File and browse to the uploaded seed.iso.
   • Ensure Connect at Power On is checked, then click OK.
6. Boot the VM. Cloud-init reads the ISO on first boot and configures the data-plane automatically.

​

Configuration

​

Cloud-Init

• meta-data — minimal instance identity (hostname and ID).
• user-data — the actual configuration in #cloud-config format.

​

Step 1 — Create a working directory

mkdir popsink-cloudinit
cd popsink-cloudinit

​

Step 2 — Create meta-data

instance-id: popsink-vm
local-hostname: popsink-vm

​

Step 3 — Create user-data

#cloud-config
write_files:
  - path: /etc/popsink/config.env
    owner: root:root
    permissions: '0600'
    content: |
      PG_HOST=db.example.com
      PG_PORT=5432
      PG_DATABASE=popsink
      PG_USER=popsink
      PG_PASSWORD=REPLACE_WITH_SECRET

      KORA_DB_HOST=db.example.com
      KORA_DB_PORT=5432
      KORA_DB_DATABASE=kora
      KORA_DB_USER=kora
      KORA_DB_PASSWORD=REPLACE_WITH_SECRET

      S3_BUCKET=s3://my-bucket/kafka/
      S3_REGION=eu-west-1
      S3_ACCESS_KEY_ID=REPLACE_WITH_KEY_ID
      S3_SECRET_ACCESS_KEY=REPLACE_WITH_SECRET

      DEPLOYMENT_ID=REPLACE_WITH_UUID
      DEPLOYMENT_JWT_TOKEN=REPLACE_WITH_SECRET
      INGRESS_URL=http://vm.example.com

      ADMIN_PANEL_USERNAME=admin
      ADMIN_PANEL_PASSWORD=REPLACE_WITH_SECRET

      JWT_SECRET=REPLACE_WITH_SECRET
      CONNECTOR_CONFIG_ENCRYPTION_KEY=REPLACE_WITH_SECRET

Security: This file contains secrets. Do not commit it to version control with real credentials. Keep it in a secure location and delete it once the VM is configured.

SSH access (optional): No SSH keys are pre-provisioned in the image. To enable SSH, add an ssh_authorized_keys stanza to your user-data alongside the write_files block:

ssh_authorized_keys:
  - ssh-rsa AAAA...your-public-key...

Without this, the VM is only accessible via the hypervisor console.

​

Static IP (optional)

version: 2
ethernets:
  eth0:
    addresses:
      - 192.168.1.50/24
    gateway4: 192.168.1.1
    nameservers:
      addresses: [192.168.1.1]

​

Step 4 — Generate seed.iso

# Install on Ubuntu/Debian
sudo apt-get install cloud-image-utils

# Without static IP
cloud-localds seed.iso user-data meta-data

# With static IP (if you created a network-config file)
cloud-localds seed.iso user-data meta-data --network-config network-config

# Install on Ubuntu/Debian
sudo apt-get install genisoimage

# Without static IP
genisoimage -output seed.iso -volid cidata -joliet -rock user-data meta-data

# With static IP (if you created a network-config file)
genisoimage -output seed.iso -volid cidata -joliet -rock user-data meta-data network-config

# Install on RHEL/Fedora
sudo dnf install genisoimage

# Without static IP
mkisofs -output seed.iso -volid cidata -joliet -rock user-data meta-data

# With static IP
mkisofs -output seed.iso -volid cidata -joliet -rock user-data meta-data network-config

The -volid cidata flag is mandatory. Cloud-init identifies the NoCloud datasource by looking for a volume labelled cidata — any other label will be ignored.

hdiutil makehybrid -o seed.iso -hfs -joliet -iso -default-volume-name cidata popsink-cloudinit/

​

Step 5 — Attach as CD-ROM and boot

• KVM/QEMU: pass seed.iso as a virtual CD-ROM in the virt-install command (see the KVM/QEMU deployment section above).
• VMware vSphere: upload seed.iso to your datastore and attach it as a CD/DVD drive before booting (see the VMware vSphere deployment section above).

​

Manual Configuration

1. Create the configuration file:
   sudo install -m 0600 -o root -g root /dev/null /etc/popsink/config.env
   sudo tee /etc/popsink/config.env <<'EOF'
   PG_HOST=db.example.com
   # ... (fill in all required parameters — see table below)
   EOF
   
2. Start the configure service:
   sudo systemctl start popsink-configure.service
   
3. Verify success:
   sudo journalctl -u popsink-configure.service
   
   A successful run ends with:
   Popsink cloud-init configuration applied successfully. Data-plane is running.
   

​

Re-applying Configuration

1. Edit /etc/popsink/config.env with the new values.
2. Remove the marker and restart the service:
   sudo rm /etc/popsink/.configured
   sudo systemctl restart popsink-configure.service
   
3. Verify success:
   sudo journalctl -u popsink-configure.service
   

​

Parameter Reference

​

PostgreSQL (required)

​

Kora Schema Registry Database (required)

​

S3-Compatible Storage (required)

​

Popsink Platform (required)

​

Admin Credentials (required)

​

Security Keys (required)

​

Outbound Proxy (optional)

​

Health Verification

​

Layer 1 — Data-Plane (network reachability)

curl http://<vm-ip>/api/readyz

​

Layer 2 — k3s Runtime

systemctl status k3s

​

Layer 3 — Configuration Service

journalctl -u popsink-configure.service

​

Layer 4 — Boot Check

journalctl -u popsink-boot-check.service

• No output: configuration was present at boot — this layer is healthy.
• Output present: no configuration was found when the VM booted. Deliver the configuration file and re-run the configure service:
  sudo rm -f /etc/popsink/.configured
  sudo systemctl start popsink-configure.service
  

​

Troubleshooting

​

Accessing Popsink

http://<vm-ip>/

​

Updates

1. Boot the new VM with the same configuration.
2. Confirm curl http://<new-vm-ip>/api/readyz returns HTTP 200.
3. Run a test pipeline end-to-end on the new VM.
4. Pause any active pipelines on the old VM, then switch network routing to the new VM.
5. Resume pipelines and decommission the old VM.

The old and new VM can run against the same database and S3 bucket simultaneously — state is in the external systems, and concurrent access is safe. Do not serve production traffic from both VMs at the same time, as pipelines may execute in duplicate.

​

Limitations