rootnode-academy/docs/PRODUCTION_DEPLOYMENT.md

Production Deployment

Step-by-step, for when the real Hetzner host exists. See docs/PHASE4_PRODUCTION_INFRASTRUCTURE.md for the architecture this executes.

0. One-time, manual, not automated (deliberately)

1. Install Proxmox VE on the bare-metal host — infra/proxmox/README.md.
2. Inspect the real disks, pick a ZFS layout — docs/PHASE4_PRODUCTION_INFRASTRUCTURE.md §4, docs/RUNBOOK_DISK_FAILURE.md's inspection checklist.
3. Build the cloud-init VM template Terraform will clone — infra/proxmox/cloud-init/user-data.yaml + infra/proxmox/README.md.
4. Fill in infra/terraform/terraform.tfvars (copy from .tfvars.example) with the real Proxmox API endpoint, node name, storage/datastore names, and chosen VM sizes.
5. Fill in infra/ansible/inventory/production.ini with the real VM IPs, and infra/ansible/group_vars/production.yml with platform_ip, lab_host_ip, management_cidr, proxmox_mgmt_cidr.
6. Copy infra/ansible/group_vars/production.vault.yml.template → production.vault.yml, fill in every vault_* secret with real generated values, then ansible-vault encrypt group_vars/production.vault.yml. Never commit the plaintext.

1. Create the VMs

cd infra/terraform
tofu init
tofu plan    # review before applying against real hardware
tofu apply

Creates rn-platform-01 and rn-lab-host-01 from the cloud-init template. Confirm both boot and are SSH-reachable from the management network before continuing.

2. Generate TLS certificates (offline, once)

export ROOTNODE_TLS_BUNDLE_DIR=/secure/offline/path   # never inside the repo
./infra/scripts/docker-tls/generate-docker-tls.sh
./infra/scripts/docker-tls/verify-docker-tls.sh

Generates FOUR leaf certs under one CA (Phase 4.1 extended this from two): the CONTROL-plane pair (server-*/provisioner-client-*, for Provisioner↔dockerd) and the DATA-plane pair (router-server-*/gateway-client-*, for Gateway↔Lab Host Router — see docs/PRODUCTION_NETWORK_SECURITY.md's "control plane vs data plane" section). The CA private key stays in ROOTNODE_TLS_BUNDLE_DIR and never touches either VM — only the CA cert (public) and each host's own leaf cert+key go to each host, both pairs deployed by the same docker-tls Ansible role into two separate directories (/etc/rootnode/docker-tls/ and /etc/rootnode/lab-router-tls/).

3. Configure both hosts

cd infra/ansible
ansible-playbook -i inventory/production.ini site.yml --ask-vault-pass

Role order matters and is fixed in site.yml: common → firewall → docker-tls → platform/lab-host → monitoring-agent → backup. Firewall and TLS land before anything that depends on them.

4. First application deploy

./infra/scripts/deploy-production.sh <image-tag>

Pulls the tagged images, runs the compose stack, applies Postgres migrations (automatic, on Platform API/Provisioner startup — see docs/RUNBOOK_FAILED_MIGRATION.md if one fails).

5. Verify

./infra/scripts/production-smoke-test.sh

Curls /healthz on the Gateway and Platform API, confirms Postgres is reachable, confirms Prometheus/Grafana/Alertmanager are up. Then walk docs/PRODUCTION_VALIDATION_CHECKLIST.md in full before calling the deployment live.

Normal (non-first) deploys

docs/RUNBOOK_NORMAL_DEPLOY.md. Rollback: docs/RUNBOOK_ROLLBACK.md.

What this document does NOT cover

DNS/Cloudflare setup for the public origin (a one-time manual step outside this repo's scope — point the A/AAAA record at the real host and confirm Caddy issues its certificate). Which storage layout to pick (decision framework only, §4 of the master doc — the real answer depends on the real disks).