Five Proxmox nodes, a ZFS NAS, and a stack of repurposed laptops — carrying ~40 production containers, a GPU-bursting media pipeline, and the AI agent fleet that builds everything on the products page. Public services reach the internet through Cloudflare tunnels; nothing is port-forwarded, and every secret lives in a vault.
Proxmox VE across five nodes, each with a role:
The ZFS workhorse: snapshots, NFS artifact stores for the media pipelines, and ~26 production containers — the full *arr media stack with Plex and Jellyfin, Immich photo library, Vaultwarden, shared Postgres and Valkey, and the OK Folio gallery on its own writable dataset.
One Maestro daemon supervises nine repositories around the clock: it labels-in work, spawns coding agents in git worktrees, gates merges behind CI and automated review, and deploys on merge. A local proxy pools multiple LLM subscriptions with deterministic model-routing and automatic failover, so the fleet keeps working when any one backend cools down.
Stem separation and Whisper transcription run on serverless GPUs (RunPod, vast.ai) rented by the minute under rolling spend caps — cents per job, receipts tracked in-app. Audio moves through Cloudflare R2; the lab never pays for an idle GPU.
Infisical manages machine-identity-scoped secrets for every service; Vaultwarden covers the humans. Public endpoints (*.oklabs.uk, kossoy.com) are published exclusively through Cloudflare tunnels behind Nginx Proxy Manager. Panoptikon — built here, hosted here — watches the network itself.
Docker lifecycles go through Dockhand (no ad-hoc docker compose by policy), releases are SemVer-tagged with auto-deploy, health-check, and rollback, backups run on an isolated PBS node, and Uptime-Kuma + ntfy page me when something actually needs a human. A Windows DAW machine and three Linux laptops serve as native test benches for OK Player.
The lab isn't the hobby — it's the leverage. Owning the full path from idea to production means every product here ships with real infrastructure behind it: monitored, backed up, secret-scoped, and rebuilt by agents while I sleep.