Files
ced4568 f3c356951c test(preprod): two-host routing simulation, drill suite, CI security scanning
- infra/scripts/simulate-two-host/: real two-Docker-network simulation
  (Gateway genuinely isolated from any Lab bridge network) proving the
  Lab Host Router fix - 13/13 real assertions (HTTP+WebSocket through the
  Router, Alice/Bob isolation, garbage/destroyed tokens, mTLS-only Router
  access).
- infra/scripts/drills/: Lab Host loss/rebuild (9/9), deployment rollback,
  CI-fails-closed (6 required failure modes), fake cohort dress rehearsal
  (14/14, one continuous narrative through the real app).
- infra/ci/gitea-workflows/build-and-test.yml: new security-scan job
  (npm audit, gitleaks, hadolint, shellcheck, trivy - all free/open-source),
  severity policy in docs/SECURITY_SCANNING.md (CRITICAL/HIGH blocks,
  MEDIUM/LOW reported only). Gates build-images alongside regression.

Found and fixed two real pre-existing bugs via the CI-fails-closed drill:
staging-smoke-test.sh and production-smoke-test.sh both had a classic
set -e gotcha (curl ... && echo "OK" as one statement - a failing command
that isn't last in an && chain doesn't trigger -e) that let an
unreachable/broken endpoint fall through silently instead of failing the
smoke test.

platform/test/e2e-phase3a.js updated for the new paginated
/api/admin/audit-events response shape (Phase 4.1 pagination work).
2026-08-24 20:16:00 -05:00

2.9 KiB

Two-Host Routing Simulation

Proves, with real Docker network isolation (not mocks), that the Gateway-to-Lab data path described in docs/PHASE4_1_PREPRODUCTION_HARDENING.md §1 actually works before the real Hetzner/Proxmox hardware exists. See that document for the full design; this is the test harness.

Run it

cd infra/scripts/simulate-two-host
PROVISIONER_SHARED_SECRET=... ./run-simulation.sh          # builds, runs, tears down
PROVISIONER_SHARED_SECRET=... ./run-simulation.sh --keep-up # leaves the stack up for debugging

Confirmed passing, 13/13, as of this phase's implementation — see the Phase 4.1 report for the full run transcript.

What this genuinely proves (TESTED LOCALLY, with real Docker networking)

  • The gateway container, attached ONLY to an isolated bridge network with no route to any Lab's per-environment bridge network, cannot open a TCP connection to a Lab container's Docker-internal IP — verified via a raw connect attempt run from inside the actual gateway container's own network namespace, not asserted from the test driver's (irrelevant) vantage point.
  • The Gateway does reach the Lab correctly through the Lab Host Router, over real mTLS, for both a plain HTTP request and a full WebSocket terminal session — including a genuine kubectl get nodes executed inside the Lab and its real output read back.
  • Alice/Bob isolation (own lab ALLOW, other's lab DENY, including over WebSocket), garbage tokens, anonymous requests, and destroyed-environment tokens are all rejected through the real code path — the same checkOwnership/resolveTarget logic production will run.
  • The Lab Host Router refuses a connection with no client certificate at all — mTLS is the actual authentication mechanism, not a documented intention.

What this does NOT prove (SIMULATED PRODUCTION BOUNDARY, not identical to real hardware)

  • Real cross-VM latency, MTU, or anything about two genuinely separate kernels — this is one Docker daemon with two isolated networks, a faithful stand-in for network reachability rules but not a physical network.
  • The actual Hetzner/Proxmox firewall enforcement (infra/firewall/*.rules) — this harness doesn't touch iptables at all; Docker's own network isolation is what's under test here.
  • Provisioner/API/Postgres reachability across the VM boundary — deliberately kept host-networked and unchanged from docker-compose.dev.yml in this harness, because that control-plane path was already proven separately (Phase 4's real mTLS handshake test against the Docker Engine TLS port) and isn't what changed this phase. Only the Gateway's isolation is new here.

REQUIRES REAL HARDWARE (tracked in docs/PRODUCTION_VALIDATION_CHECKLIST.md, not here)

Running this exact flow against the real rn-platform-01/rn-lab-host-01 VMs, through the real firewall, with real network latency between two physical/virtual hosts.