rootnode-academy/docs/LAB_ARCHITECTURE.md

# RootNode Academy — Ephemeral Lab Architecture

Status: **Phase 1 (platform foundation) + Phase 2A (Gateway + student flow) + Phase 2B
(Platform API split, Postgres academic data, server sessions, admin portal) + Phase 3A
(persistent sessions, per-instructor accounts, RBAC, audit log) + Phase 3B (learning/assessment
engine: attempt telemetry, progressive hints, curriculum rules, exam + capstone engine,
achievements) IMPLEMENTED, 2026-08-18** — see `docs/PHASE1_PLATFORM_FOUNDATION.md`,
`docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md`, `docs/PHASE2B_ADMIN_AND_PLATFORM_DATA.md`,
`docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md`, and
`docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` for concrete schema/API/test results, and
`platform/README.md` for how to run the stack. This document remains the architecture record
for the full ephemeral-lab direction; production OIDC/SSO (an adapter interface exists, not a
working integration) and production Proxmox deployment are still future work (Phase 4 — see the
Phase 3B doc §21). Supersedes the namespace-isolation
(`shared_namespace`/`ephemeral_cluster`) approach that was in flight before this redirect — see
`scenarios/SCHEMA.md`'s `isolation_type` note for the pointer back here.

Committed direction: **one fully isolated, disposable environment per student. No shared
Kubernetes control plane, ever.** This is the long-term architecture, not a stopgap.

**Decisions locked 2026-08-18** (resolves the four open questions this doc originally ended
with — see §2–§5 below for where each lands, and `docs/PHASE1_PLATFORM_FOUNDATION.md` for the
Phase 1 build plan against these decisions):
1. **Egress:** restricted, explicit allowlist. No unrestricted internet egress, ever. No route
   to trusted RootNode management/CDAA/home/business networks, under any allowlist entry.
2. **Platform DB:** PostgreSQL. The Platform control plane is not built around SQLite. The
   existing single-container local dev mode keeps using its own local SQLite as today —
   unaffected, unrelated database.
3. **Suspend/resume:** experimental only, not load-bearing. The guaranteed lifecycle is
   `provision → run → destroy → recreate`. Persistent academic state lives outside the Lab
   specifically so destroying one never loses student progress — suspend/resume is a possible
   future optimization, not something correctness depends on.
4. **Resource sizing:** not hard-coded. Quotas are policy/config-driven. Target host class is
   ~16 cores/32 threads, 128GB RAM, SSD/NVMe — actual per-lab limits and concurrency get set
   by benchmarking against that class, not guessed up front.

**Corrections locked 2026-08-18** (round 2 — architectural fixes required before Phase 1
implementation begins; folded into §1, §4, §7, §9 below, not just listed here):

- **A. VM boundary.** Privileged Lab containers must never share an OS/VM boundary with the
  Platform (Postgres, control-plane) in production — separate VMs under Proxmox, connected by
  a Provisioner-initiated control channel only. The Lab Host is a *configured target*, not an
  assumption that Docker is local. Single-machine is still fine for Phase 1 development — see §1/§9.
- **B. Egress is a firewall control, not an app convention.** `HTTP_PROXY`/`HTTPS_PROXY`
  configure well-behaved apps for convenience; they enforce nothing. The firewall is the
  control, and rule *ordering* matters — required infrastructure (proxy, DNS) must be
  explicitly permitted before the blanket RFC1918 drops, or those drops shadow the allow
  rules and nothing works. See §4 — this also corrects a real ordering bug in the original
  design (explained there).
- **C. `student_id` is opaque in Phase 1.** No real foreign key to a `students` table —
  students/progress stay on SQLite until a later, separate Platform DB migration. Documented,
  not silently assumed. See §3/§9.
- **D. Lab host is an abstraction from day one**, even with exactly one configured host in
  Phase 1 — the Provisioner is never allowed to hard-code "Docker is local." See §4.

---

## 0. What already exists (grounding, not invention)

The current repo (`~/Dev/Gitea/ced4568/rootnode-academy`) already documents almost this exact
intent in its own README, just never built out:

> "The upstream project assumes one learner, one container. RootNode Academy runs it across
> three delivery modes... One container **per student** — never share a cluster between
> learners... Put authentication in front (proxy-level)... Idle-timeout + auto-teardown."

And the existing image already **is** a complete single-student lab: nginx + React frontend +
Node/Express backend + `node-pty` WebSocket terminal + a single-node k3s cluster, all in one
`--privileged` container (confirmed required — k3s needs kernel namespaces/cgroups, per
`README.md:52`). That container is the unit this design multiplies, not something it replaces.

What changes: today that one container also owns identity/progress (SQLite at
`/data/progress.db`, one volume per student, mounted by hand). This design pulls that
ownership out into a persistent Platform, and makes the container itself fully disposable.

---

## 1. Proposed architecture

Two roles — and in production, on **two separate VM/OS boundaries**, not just two logical
components sharing a box. Correction A drives this: `--privileged` Docker containers are not
a security boundary from their own Docker host, so the boundary has to be a real VM boundary,
not a process boundary.

**Production topology:**

```
Proxmox VE (or equivalent hypervisor)
│
├── RootNode Platform VM   (trusted — never runs student code)
│   │
│   │  Platform API   roster · login · progress · curriculum · exam records ·
│   │                 environment registry · instructor controls
│   │  Platform DB    PostgreSQL — environments + audit log now;
│   │                 students/progress join later (§3/§9, correction C)
│   │  Provisioner    talks OUT to a *configured* Lab Host endpoint —
│   │                 never assumes Docker is local (§4, correction D)
│   │  Gateway        opaque-token routing to student lab containers;
│   │                 only component with a leg in both worlds
│   │
│   ▼ (Provisioner-initiated control channel only — TCP+TLS to the Lab
│      Host's Docker API. Nothing on the Lab Host VM ever initiates a
│      connection back. See §4/§7 for why this direction matters.)
│
└── RootNode Lab Host VM   (untrusted — this is where privileged student
    │                       containers actually run; holds nothing the
    │                       Platform needs protected)
    │
    ├── Student A's LAB ── own private network, own nested k3s, no identity
    ├── Student B's LAB ── own private network, own nested k3s, no identity
    └── Student C's LAB ── own private network, own nested k3s, no identity
```

Each Lab is **today's `Dockerfile` and `entrypoint.sh`, functionally unchanged** — still runs
k3s + the terminal + scenario setup/validate/teardown exactly as now. The only change inside
it: it stops being the system of record for who's using it or what they've completed (§3, §9).

A compromised student container's worst case is the Lab Host VM — never Postgres, never the
Platform API, because there is no network path from that VM back to the Platform VM at all
(§4, §7). That containment is the actual point of the two-VM split, not a deployment nicety.

**Phase 1 development exception:** running the Platform and a Lab container on one machine is
fine for local testing (§9) — the Provisioner just gets configured with a single `lab_host`
pointing at the local Docker socket instead of a remote TLS endpoint. The *code* never
special-cases "local" vs "remote" — same abstraction either way (§4/§9 of
`docs/PHASE1_PLATFORM_FOUNDATION.md`).

---

## 2. Environment lifecycle

State machine per environment, tracked in the Platform DB (`environments` table, §3):

**Guaranteed path** (what correctness depends on): `requested → provisioning → running →
resetting → running` (reset loop, same route) and `(provisioning|running|error) → destroying
→ destroyed`. `provisioning → error` and `running → error` cover failed/lost environments;
`error → destroying → destroyed` (or `→ resetting → running`, when recoverable) is the way out.

```
requested → provisioning → running ─┬─→ resetting → running (clean slate, same route)
                                     └─→ destroying → destroyed
provisioning → error (failed to reach ready state)
running → error (health check fails mid-session)
error → destroying → destroyed  (or → resetting → running, if recoverable)

[EXPERIMENTAL, not relied on for correctness — see decision #3]
running ──suspend──→ suspended ──resume──→ running (or → error, if resume fails)
```

- **provision** — Provisioner picks a host with capacity (§5), creates a private network for
  this student (§4), starts a container from the pinned Lab image, generates a random
  high-entropy routing token, waits for the existing readiness signal (k3s API ready + backend
  `/api/health` OK — same check the current entrypoint already does), writes the `environments`
  row, hands the route back to the Platform API.
- **start** — used only for `provisioning → running` (starting a freshly created container).
  There is no supported path that starts a previously-stopped container in the guaranteed
  lifecycle — see suspend/resume below.
- **suspend/resume — EXPERIMENTAL, decision #3.** `docker stop`/`docker start` on a k3s-in-container
  Lab is *not assumed to work* and is not built as part of the guaranteed lifecycle. It's
  worth testing (see the Phase 1 test plan) because if it turns out reliable it's a cheap
  latency win later — but nothing in the platform depends on it succeeding. The only supported
  way to bring an environment back is `destroy` + `provision` (or `reset`, which is that same
  pair against one `environments` row/routing token). This is also *why* academic state living
  outside the Lab (§3) isn't optional — it's what makes "just destroy and recreate" a fully
  safe default instead of a data-loss risk.
- **reset** — destroy and recreate the container from the pinned image, **same routing token
  and same `environments` row**, so the student's "return to my lab" link doesn't change.
  This is the instructor-facing "reset student environment" control.
- **destroy** — stop + remove container, remove its private network, release the routing
  token, mark the `environments` row `destroyed` with a timestamp. Progress already lives in
  the Platform DB (§3), so destroying an environment never loses academic data — only
  in-terminal, not-yet-validated work.
- **Idle reaper** (new, required — not optional) — background job in the Provisioner checks
  `last_active_at` per running environment (heartbeat from the Gateway or from Lab `/api/health`
  polling) and auto-suspends past an idle threshold, auto-destroys past a longer abandoned
  threshold. Without this, "ephemeral by default" is just a description, not a guarantee —
  a closed browser tab does not stop a running k3s cluster.

---

## 3. Data model changes

Everything below lives in the **Platform DB**, not inside any Lab container.

Already built (previous slice, unaffected by this redirect — still correct under this
architecture):
- `students(student_id, display_name, cohort, created_at, last_active_at)`
- `progress(student_id, scenario_id, status, attempts, ...)` — composite key, already
  per-student.
- `sessions` / `exam_progress` — exam records, already student-attributed.

New:
- `environments(environment_id UUID PK, student_id, status, routing_token, container_id,
  lab_host_id FK, internal_ip, cpu_limit, mem_limit_mb, storage_limit_mb, image_ref, created_at,
  started_at, last_active_at, destroy_at, error_reason)` — **`student_id` is an opaque external
  identifier here (correction C), not a foreign key.** Students/progress stay on the existing
  SQLite in Phase 1 (see §9); Postgres has no `students` table yet, so there is nothing to key
  against. The Provisioner trusts whatever `student_id` its caller passes — it does not
  validate it against a roster. A real FK gets added when students/progress actually migrate
  onto this Postgres instance, not before. `lab_host_id`, by contrast, *is* a real FK — both
  tables live in the same Provisioner-owned Postgres DB (correction D, full schema in
  `docs/PHASE1_PLATFORM_FOUNDATION.md`).
- `lab_hosts(lab_host_id PK, endpoint, enabled, ...)` — the abstraction correction D requires:
  which Docker endpoint a given environment's container actually lives on. Phase 1 ships with
  exactly one row (pointing at either a local socket in dev or the Lab Host VM's TLS endpoint
  in production) — the table exists so environment creation is never coupled to "Docker is
  local" as an assumption, even though there's no scheduling logic yet.
- `environment_events(id, environment_id, event_type, actor, detail, created_at)` — audit
  trail for provision/start/stop/reset/destroy/error, needed for both incident response and
  ordinary "why did my lab disappear" support.

**Removed from the Lab image's responsibility:** the `students` and roster-facing parts of the
schema I added to `backend/server.js` in the previous slice move to be Platform-only. The Lab
image's own SQLite either goes away entirely or is kept only as scratch/ephemeral state that's
allowed to die with the container — it must not be treated as authoritative for anything.

**Decision #2: PostgreSQL, not SQLite.** The Platform control plane — the environment registry
specifically, and the target for progress/roster once they migrate over — is Postgres. The
existing single-container local dev mode keeps its own local SQLite exactly as today; that's a
different, unrelated database serving a different purpose (single-tenant local testing), not
something this decision touches. See `docs/PHASE1_PLATFORM_FOUNDATION.md` for the exact schema
being stood up first.

---

## 4. Networking model

- **One private Docker network per student environment**, not one shared network for all
  labs. Docker's default per-network isolation means Student A's container has no L2/L3 path
  to Student B's container at all — this is stronger and simpler than trying to firewall
  students apart on a shared network.
- **No published ports on Lab containers.** Today's image publishes `-p 7554:80`; under this
  design that stops — the only thing that can reach a Lab container is the Gateway, over that
  student's private network.

### Egress — correction B: the firewall is the control, not the proxy env vars

`HTTP_PROXY`/`HTTPS_PROXY` are set on each Lab container so well-behaved processes (containerd
pulling scenario images) route through the allowlisting proxy cleanly. **That is configuration,
not enforcement** — a process that ignores those env vars and dials out directly must still be
blocked, by something that doesn't depend on the container's cooperation. That something is a
host firewall rule, evaluated on every packet regardless of what any process inside the
container does.

**Ordering bug in the original draft, corrected here:** the previous version of this section
listed proxy/DNS `ACCEPT` rules before the RFC1918 `DROP` rules in prose, but specified
installing them with `iptables -I` (insert-at-head) in that same top-to-bottom order — which
actually reverses the result, since each `-I` without a rule number pushes the *new* rule to
position 1. Run in that order, the RFC1918 drops end up evaluated *before* the proxy/DNS
accepts, silently blocking the proxy itself if it happens to sit on an RFC1918 address (it
will — it's Platform-side infrastructure). Corrected mechanism (exact rules in
`docs/PHASE1_PLATFORM_FOUNDATION.md` §5): each student network gets its own dedicated iptables
chain, built with `-A` (append, so rules land in the order they're added — no head-insertion
trap), containing explicit `ACCEPT` rules for the egress proxy, DNS, and any specifically
required Platform endpoint *first*, then the RFC1918 `DROP`s, then a final default-`DROP`. That
chain is reached from `DOCKER-USER` (the chain Docker guarantees it won't silently rewrite) via
one jump rule matching that student's subnet.

**What no student lab may ever reach, regardless of allowlist state:**
- RootNode management networks
- the Platform database (Postgres)
- Docker/Proxmox management interfaces (including the Lab Host VM's own Docker API endpoint —
  a Lab container talking back to the thing that provisioned it is exactly the escalation path
  this design exists to prevent)
- CDAA infrastructure
- home/trusted networks
- other students' lab networks (already unreachable via Docker's own per-network isolation
  above; the RFC1918 drop is a deliberate second layer, not redundant — it also catches any
  case where two student networks' address ranges could otherwise see each other at the host
  routing level)

**Decision #1, restated with the ordering fix:** restricted egress, explicit allowlist, no
unrestricted internet access ever, enforced at the firewall layer with required infrastructure
explicitly permitted *before* the blanket denies — not the reverse, and not left implicit.

### VM-to-VM boundary — correction A

The Lab Host VM has **no route back to the Platform VM at all**, in either direction, except
the one the Provisioner itself opens. The Provisioner (on the Platform VM) connects *out* to
the Lab Host's Docker API over TCP+TLS with a client certificate only the Provisioner holds;
nothing on the Lab Host VM — student containers or the VM's own OS — can open a connection back
toward the Platform VM. This is enforced at the hypervisor/network level (Proxmox VLAN/firewall
rules between the two VMs), the same category of control as the CDAA/home boundary below, not
something Docker networking alone provides. A compromised student container's blast radius
stops at the Lab Host VM.

**No route to trusted CDAA/home/business networks, full stop.** This is a host/hypervisor
boundary, not something Docker networking alone can guarantee — the box(es) these containers
run on need to sit on their own segment with no bridge/route to those networks, consistent with
the existing `RootNode_Academy_Business_Pack/06_Technology/01_Technical_Platform_Plan.md`
("dedicated student network/DMZ separated from trusted networks"). This design assumes that
host-level segmentation is a prerequisite, not something the app layer can create on its own.

### Lab host abstraction — correction D

The Provisioner never talks to "the local Docker socket" as a hard-coded assumption — it talks
to whichever `lab_hosts` row it's configured to use (§3), resolved to a Docker endpoint that
may be `unix:///var/run/docker.sock` (Phase 1 dev, single machine) or a remote
`tcp://<lab-host-vm>:2376` with TLS (production, per the topology in §1). Phase 1 ships exactly
one enabled `lab_host` and zero scheduling logic — the abstraction exists so adding a second
host later is a config row, not a rewrite.

- **Gateway** is the only component with a leg in both the public-facing network and every
  student's private network. It resolves `routing_token → container internal IP:port` by
  querying the Platform API (or a shared registry cache) per request/connection, then proxies
  both HTTPS (scenario API, static frontend) and the WSS terminal connection.

---

## 5. Resource model

- Per-environment caps applied at container-create time: `--cpus`, `--memory` (with
  `--memory-swap` pinned equal to `--memory` so nothing swaps), `--pids-limit` (fork-bomb
  guard). Storage: needs a size-capped volume — exact mechanism (`--storage-opt size=`, a
  loopback-backed bind mount, or a tmpfs cap) depends on the host's actual Docker storage
  driver, which I haven't inspected — needs to be confirmed against the real deploy target
  before picking one.
- **Decision #4: policy-driven, not hard-coded.** Limits come from config, not constants in
  code — a resource-policy file the Provisioner reads at startup (and can reload), not numbers
  baked into the create-container call. No specific numbers are chosen yet. Target host class
  for planning purposes: ~16 cores/32 threads, 128GB RAM, SSD/NVMe. Actual per-lab limits and
  how many environments run concurrently on that class get set by benchmarking (Phase 1's test
  plan includes the first real measurements), not by guessing backward from the spec sheet.
- **Admission control:** Provisioner refuses to `provision` past the point where already-running
  environments' summed limits would exceed host capacity. Simple counter/gate to start; can
  become real bin-packing later if there's ever more than one host.

---

## 6. Routing/auth model

1. Student logs in at the Platform (existing roster/pick-your-name flow, unchanged).
2. Frontend asks the Platform for "my lab." If none running, Platform calls the Provisioner to
   `provision` (or `start`, if one's suspended).
3. Platform returns an **opaque, high-entropy routing token** scoped to
   `(student_id, environment_id)`, short TTL, revoked on `destroy`.
4. Frontend connects to the Gateway using that token — `https://labs.<domain>/e/<token>/...`
   and `wss://labs.<domain>/e/<token>/shell-ws`. The Lab container itself never sees a
   student identity; it's purely an execution surface addressed by an opaque route.
5. Gateway validates the token maps to a currently `running` environment before proxying —
   rejects stale tokens from destroyed/reset environments outright, so there's no window where
   an old link reaches a different (possibly now-different-student's) container.
6. **Instructor auth** extends the shared-passcode model already used for the roster-add
   endpoint: instructor-only calls (`list environments`, `reset`, `terminate`, `resource usage`)
   go through the Platform API under that same gate, which then calls the Provisioner
   internally. Students and the public internet never talk to the Provisioner or the Docker
   socket directly — only the Platform backend does.

---

## 7. Security boundaries (defense in depth)

1. **Browser ↔ Gateway** — public-facing, TLS, opaque routing tokens only. Nothing
   cluster-admin-shaped is reachable from here.
2. **Gateway ↔ Lab container** — private per-student Docker network only, no published ports,
   Gateway is the sole entry point.
3. **Inside the Lab container** — `--privileged` is required for k3s (confirmed, not
   avoidable — this isn't a place to try to "hold the line" without breaking k3s). The
   mitigation is containment, not avoidance: this privileged container has no reachable
   network path to anything except the Gateway, runs as a throwaway that gets destroyed/reset
   routinely, and cluster-admin *inside a student's own disposable cluster* is expected and
   fine — that's the entire point of full isolation, a student doing CKS-tier cluster-admin
   work only ever affects their own throwaway cluster.
4. **Lab container ↔ its own Docker host (correction A — the boundary that actually matters
   here)** — `--privileged` is explicitly **not treated as a sufficient isolation boundary from
   its own Docker host.** In production the Lab Host VM holding these containers is a separate
   VM from the Platform VM, with no route back to it at all (§4). If a student ever escapes
   their container, the ceiling is the Lab Host VM — never Postgres, never the Platform API,
   never another student's environment.
5. **Provisioner ↔ Lab host's Docker API** — only the Provisioner can invoke it, over a
   connection *it* initiates (TCP+TLS, client-cert auth, per §4's lab-host abstraction). The
   Docker endpoint is never student-reachable, never internet-reachable, and nothing on the Lab
   Host VM can initiate a connection back toward the Provisioner or the Platform VM.
6. **Platform VM ↔ other networks** — dedicated segment, no bridge to home/business/CDAA
   trusted infrastructure, matching the existing Technical Platform Plan doc. Prerequisite, not
   something this design builds. Same standard applies to the Lab Host VM independently — it
   isn't exempted just because it's "the untrusted one."
7. **Routing tokens** — CSPRNG-generated, short TTL, invalidated on any lifecycle transition
   out of `running`.

---

## 8. Failure/recovery behavior

- **Provisioning fails** (host at capacity, image pull fails, k3s doesn't reach ready state in
  time) — environment marked `error`, student gets an explicit "your lab failed to start, try
  again" message (never a silent hang), one automatic retry, then surfaced to the instructor
  dashboard (future) as a flagged failure.
- **Mid-session crash** — Gateway's health check detects the upstream is unreachable, marks
  the environment `error`, offers the student a "restart my lab" action (→ `reset`, since a
  crashed in-container k3s is unlikely to resume cleanly). Because progress/identity/exam
  records live in the Platform DB, this never loses the student's academic record — worst case
  is losing in-progress, not-yet-validated terminal work.
- **Host failure / reboot** — every environment on that host becomes unreachable. On
  Provisioner restart, it must **reconcile its registry against actual running containers**
  rather than trust stale DB state — some containers may auto-restart via Docker's restart
  policy, most won't; anything the registry claims is `running` that isn't actually running
  gets marked `error`, not silently left wrong.
- **Idle/abandoned environments** — the reaper (§2) bounds resource leakage from crashed tabs
  or students who just walked away, independent of any other failure path.
- **Core resilience property of this whole design:** because the Platform DB — not the
  disposable environment — is the system of record, every failure mode above is recoverable
  without academic data loss. That's the actual payoff of the split in §1/§3, not just an
  architectural nicety.

---

## 9. Migration path from the current single shared container

- **Phase 0 (already true today, keep it):** running the existing image standalone —
  `docker run --privileged -p 7554:80 ... rootnode-academy:latest` — is exactly the
  single-user local dev/test mode you asked to keep available. No change needed to support
  this; it already works and stays supported as-is.
- **Phase 1:** Build Provisioner + `environments`/`environment_events`/`lab_hosts` registry
  around the **existing, unmodified** Lab image, on Postgres, with the egress firewall/proxy
  from §4 in place from the start (not bolted on later). Development exception (correction A):
  running Platform + Provisioner + one Lab container all on a single machine is fine here — the
  Provisioner is configured with one `lab_host` pointing at the local Docker socket instead of
  a remote TLS endpoint, same code path either way. Prove two lab environments can run side by
  side, fully isolated from each other, before any production traffic or a second VM enters
  the picture. `students`/`progress` stay on the existing SQLite through this phase
  (correction C) — `environments.student_id` is opaque, not a foreign key, until a later,
  separate migration actually moves roster/progress onto Postgres.
- **Phase 2 (the one real code change inside the Lab image):** narrow `backend/server.js`
  inside the Lab image down to a "dumb executor" — scenario setup/validate/teardown/answer and
  the WS terminal stay, but it stops owning `students`/roster and stops being the source of
  truth for progress. Either it calls back to the Platform API to report validation results,
  or — cleaner — the Platform API becomes the only thing the frontend talks to for
  progress/curriculum/identity, and it calls the Lab container (via the route the Provisioner
  already tracks) purely to execute against that student's cluster and get a pass/fail back.
- **Phase 3:** Wire instructor controls (list/reset/terminate/resource-usage) against the new
  registry — this is also where the "no auth on roster-add" gap from the previous slice gets
  closed for real, since instructor auth is now a load-bearing security boundary, not just
  roster hygiene.
- **Phase 4 (cutover):** point the real domain at the Gateway instead of a single container.
  Keep the Phase 0 single-container path available under local/dev tooling, per your explicit
  requirement.
- **Rollback safety:** at every phase through Phase 3, the current single-shared-container path
  keeps working completely unmodified — nothing about this migration risks the working
  baseline until the Phase 4 cutover, which is also the only step that's meaningfully
  reversible-with-effort rather than trivially reversible.

---

## 10. Exact files/components this touches (once approved — no code written yet)

**New, none of it touching the existing Lab image's core logic:**
- `platform/provisioner/` — Docker lifecycle management against a *configured* `lab_host`
  endpoint (correction D — never a hard-coded local socket), `environments`/`lab_hosts` CRUD,
  per-environment firewall chain management (correction B), idle reaper, admission control.
- `platform/egress-proxy/` — the shared allowlisting proxy every Lab container's egress must
  pass through (correction B).
- `platform/gateway/` — token→route resolution, HTTPS + WSS proxying (candidate: Traefik with
  a dynamic provider backed by the Platform API, or a small custom Node proxy — not decided,
  not needed for this design review).
- Platform DB migrations: `environments`, `environment_events`, `lab_hosts` tables — full
  schema in `docs/PHASE1_PLATFORM_FOUNDATION.md`.

**Modified, once implementation starts:**
- `backend/server.js` — split as described in §9 Phase 2. This is presently both "the lab
  image's backend" and "the only backend that exists"; under this design it forks into a
  narrower Lab-side executor and a separate, persistent Platform-side API. Exact file layout
  for that split is an implementation decision, not part of this design doc.
- `README.md` — the "Multi-Student Deployment" section (`README.md:128-146`) currently
  documents per-student **volumes** for persistence; that persistence model is superseded by
  §3 above and the README needs updating once this ships, so it doesn't contradict reality.
- `docs/ENGINEERING_SOP.md` — new section for the Platform/Provisioner/Gateway components and
  the reconciliation/reaper operational procedures from §8.

**Unchanged:**
- `Dockerfile`, `scripts/entrypoint.sh`, `scripts/nginx.conf` for the Lab image itself — it's
  still one container running k3s + backend + terminal, exactly as today, just multiplied and
  stripped of identity ownership.
- All scenario content (`scenarios/data/`, `scenarios/bundles/`, `scenarios/rootnode-path.json`)
  — unaffected either way; every scenario now runs with full admin of its own disposable
  cluster, so the earlier `shared_namespace`/`ephemeral_cluster` split has no bearing here.

**New Dockerfile needed (not yet written):** one for the Provisioner+Gateway services — these
are new components, not variants of the existing Lab image.

---

## Status

All four open questions this doc originally ended with are resolved — see the decisions block
at the top and §2–§5 above. Next: `docs/PHASE1_PLATFORM_FOUNDATION.md` — the concrete build
plan for the platform foundation (Postgres environment registry, provisioner skeleton,
networking, egress, resource policy, reaper) against these decisions.

## Phase 4: this boundary in production

Phase 4 (`docs/PHASE4_PRODUCTION_INFRASTRUCTURE.md`) codifies the Platform/Lab-Host split this
document designed as two real, separately-firewalled Proxmox VMs (`rn-platform-01`,
`rn-lab-host-01`) connected by two separate mTLS channels (Provisioner↔dockerd CONTROL plane,
Gateway↔Lab Host Router DATA plane) and a tightly scoped firewall allowance — see
`docs/PRODUCTION_NETWORK_SECURITY.md`. Phase 4 surfaced a gap in the Gateway's `env.internal_ip`
mechanism for reaching a Lab container (it assumes Gateway and Lab share a Docker daemon, which
breaks once they're genuinely on separate VMs); Phase 4.1
(`docs/PHASE4_1_PREPRODUCTION_HARDENING.md`) built and simulation-tested the fix, a Lab Host
Router (`platform/lab-router/`).