rootnode-academy/docs/PRODUCTION_NETWORK_SECURITY.md

Production Network & Security

The Platform VM / Lab Host VM boundary is the core production control. This document is where every network decision that touches it gets written down. Nothing here weakens it.

Topology

Internet ──HTTPS(443)──▶ Caddy (rn-platform-01) ──▶ Gateway (internal bridge)
                                                        │
management_cidr ──SSH──▶ rn-platform-01                │
management_cidr ──SSH──▶ rn-lab-host-01                 │
proxmox_mgmt_cidr ──22/8006──▶ Proxmox host             │
                                                        │
rn-platform-01 (Provisioner) ──TLS:2376──▶ rn-lab-host-01 (dockerd)
rn-lab-host-01 (promtail) ──3100──▶ rn-platform-01 (Loki)
rn-lab-host-01 (node-exporter/cadvisor) ◀──scrape── rn-platform-01 (Prometheus)

Firewall source of truth: infra/firewall/{proxmox-host,rn-platform-01,rn-lab-host-01}.rules (iptables-restore format, deployed by the firewall Ansible role). Validate with infra/firewall/validate-firewall.sh.

What is explicitly denied

• Student Lab → another student Lab (no student-to-student traffic, ever).
• Student Lab → Proxmox management (both the host firewall and the lab_subnet_range DENY rule in rn-lab-host-01.rules).
• Student Lab → Platform Postgres / Platform API / Gateway internals (logged DENY in rn-platform-01.rules for anything from lab_host_ip to ports 5432/4100).
• rn-lab-host-01 → Postgres (same rule — the Lab Host itself has no legitimate reason to reach Postgres; only the Provisioner on rn-platform-01 does, and it talks to Postgres over the internal Docker bridge, never across the VM boundary).
• Any student egress except through the approved egress proxy (enforced by the Provisioner's existing per-Lab DOCKER-USER chain manipulation from Phase 1 — unchanged by this phase).
• Direct public exposure of the Provisioner, Postgres, Docker TLS port, Proxmox UI, or Grafana (Grafana is reachable only via Caddy's /grafana/* path, itself restricted to management_cidr by a caddy matcher — see infra/docker/platform/Caddyfile.j2).

Docker TLS (Provisioner ↔ Lab Host)

Mutual TLS on port 2376, restricted at the firewall to platform_ip only. Application support already existed since Phase 1 (platform/provisioner/src/docker.js's buildClient()); this phase added the operational half: infra/scripts/docker-tls/generate-docker-tls.sh (CA + server + client cert generation), the docker-tls Ansible role (deploys certs, configures dockerd's daemon.json), and docs/RUNBOOK_DOCKER_TLS_ROTATION.md for renewal. The CA private key is generated offline and never committed or deployed to either VM — only the CA cert (to verify the peer) and each host's own leaf cert+key.

Public ingress

One origin, Caddy, automatic HTTPS. reverse_proxy gateway:4200 — Caddy talks only to the Gateway; nothing else is proxied. WebSocket upgrade works automatically (Caddy's default reverse_proxy behavior, no special config needed for the terminal WebSocket path). Whether to front this with Cloudflare or expose the origin directly is left as a deployment-time decision (both work with Caddy's ACME flow; Cloudflare adds DDoS absorption at the cost of Cloudflare being able to see plaintext unless "Full (strict)" mode is used) — not decided here since it doesn't affect anything else in this document.

Known gap: Gateway-to-Lab reachability across two real VMs

This is the most important item in this document.

The Gateway currently reaches a running Lab container via env.internal_ip — a Docker bridge-local address recorded by the Provisioner when it creates the container (platform/gateway/src/server.js, platform/provisioner, built in Phase 2A). This works when Gateway and the Lab share a Docker daemon, which has been true in every setup so far, including dev and any single-host deployment. It stops being true once Labs run on a genuinely separate rn-lab-host-01 — a Docker bridge-internal IP on one host is not routable from a different host by default.

What's built now (network-policy half only): a firewall allowance in infra/firewall/rn-lab-host-01.rules for platform_ip → lab_gateway_port_range (30000:32767 by default, configurable), so that once Lab containers publish ports on the Lab Host's own interface, the Gateway VM is permitted to reach them.

What's not yet built (application half — explicitly flagged, not silently decided here): the Provisioner needs to either (a) publish each Lab container's relevant port(s) on the Lab Host's interface and record a reachable external_ip/external_port alongside (or instead of) internal_ip/internal_port, or (b) route Gateway→Lab traffic through a proxy/tunnel that lives on the Lab Host and forwards to the container's internal address. Option (a) is simpler and consistent with how the egress proxy and cadvisor are already handled in this phase's compose files; option (b) avoids publishing a wide port range but adds a moving part. This is a small, reviewable, application-level change — not a network-policy change — and should land as its own reviewed commit before real two-VM production traffic depends on it. Until it lands, Gateway↔Lab terminal/scenario traffic will not work correctly across two separate VMs, even though every other production networking, firewall, and TLS boundary is in place and correct.

Cookie / auth production settings

infra/environments/production/*.env.template: COOKIE_SECURE=1, TRUST_PROXY=1. Dev's insecure defaults (COOKIE_SECURE unset, no proxy trust) are never inherited — production env files are separate files, not a dev file with overrides layered on.