Files
rootnode-academy/docs/LAB_ARCHITECTURE.md
T
ced4568 40692926ee docs(phase3a): document production identity architecture
New docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md covers the schema, persistent session
design, instructor accounts, the full RBAC matrix, cohort scoping (including the documented
raw-environment-id simplification), student session hardening (explicitly still
classroom/pilot identification, not production auth), CSRF, rate limiting, audit logging,
the API inventory, a security hardening checklist, test results, known limitations, the
future OIDC path, and production HTTPS/cookie expectations — with an explicit BUILT NOW vs.
FUTURE split.

Updates ENGINEERING_SOP.md (new §9/§10, closing a stale '(§9)' forward-reference that dated
back to Phase 2B), LAB_ARCHITECTURE.md status line, README.md, platform/README.md, and
platform/api/README.md (bootstrap-owner usage, new config vars, no more ADMIN_PASSWORD).
2026-08-18 13:36:45 -05:00

487 lines
31 KiB
Markdown

# RootNode Academy — Ephemeral Lab Architecture
Status: **Phase 1 (platform foundation) + Phase 2A (Gateway + student flow) + Phase 2B
(Platform API split, Postgres academic data, server sessions, admin portal) + Phase 3A
(persistent sessions, per-instructor accounts, RBAC, audit log) IMPLEMENTED, 2026-08-18** — see
`docs/PHASE1_PLATFORM_FOUNDATION.md`, `docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md`,
`docs/PHASE2B_ADMIN_AND_PLATFORM_DATA.md`, and `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md`
for concrete schema/API/test results, and `platform/README.md` for how to run the stack. This
document remains the architecture record for the full ephemeral-lab direction; production
OIDC/SSO (an adapter interface exists, not a working integration — see the Phase 3A doc §16)
and production Proxmox deployment are still future work (Phase 3B+ — see the Phase 3A doc §18).
Supersedes the namespace-isolation
(`shared_namespace`/`ephemeral_cluster`) approach that was in flight before this redirect — see
`scenarios/SCHEMA.md`'s `isolation_type` note for the pointer back here.
Committed direction: **one fully isolated, disposable environment per student. No shared
Kubernetes control plane, ever.** This is the long-term architecture, not a stopgap.
**Decisions locked 2026-08-18** (resolves the four open questions this doc originally ended
with — see §2–§5 below for where each lands, and `docs/PHASE1_PLATFORM_FOUNDATION.md` for the
Phase 1 build plan against these decisions):
1. **Egress:** restricted, explicit allowlist. No unrestricted internet egress, ever. No route
to trusted RootNode management/CDAA/home/business networks, under any allowlist entry.
2. **Platform DB:** PostgreSQL. The Platform control plane is not built around SQLite. The
existing single-container local dev mode keeps using its own local SQLite as today —
unaffected, unrelated database.
3. **Suspend/resume:** experimental only, not load-bearing. The guaranteed lifecycle is
`provision → run → destroy → recreate`. Persistent academic state lives outside the Lab
specifically so destroying one never loses student progress — suspend/resume is a possible
future optimization, not something correctness depends on.
4. **Resource sizing:** not hard-coded. Quotas are policy/config-driven. Target host class is
~16 cores/32 threads, 128GB RAM, SSD/NVMe — actual per-lab limits and concurrency get set
by benchmarking against that class, not guessed up front.
**Corrections locked 2026-08-18** (round 2 — architectural fixes required before Phase 1
implementation begins; folded into §1, §4, §7, §9 below, not just listed here):
- **A. VM boundary.** Privileged Lab containers must never share an OS/VM boundary with the
Platform (Postgres, control-plane) in production — separate VMs under Proxmox, connected by
a Provisioner-initiated control channel only. The Lab Host is a *configured target*, not an
assumption that Docker is local. Single-machine is still fine for Phase 1 development — see §1/§9.
- **B. Egress is a firewall control, not an app convention.** `HTTP_PROXY`/`HTTPS_PROXY`
configure well-behaved apps for convenience; they enforce nothing. The firewall is the
control, and rule *ordering* matters — required infrastructure (proxy, DNS) must be
explicitly permitted before the blanket RFC1918 drops, or those drops shadow the allow
rules and nothing works. See §4 — this also corrects a real ordering bug in the original
design (explained there).
- **C. `student_id` is opaque in Phase 1.** No real foreign key to a `students` table —
students/progress stay on SQLite until a later, separate Platform DB migration. Documented,
not silently assumed. See §3/§9.
- **D. Lab host is an abstraction from day one**, even with exactly one configured host in
Phase 1 — the Provisioner is never allowed to hard-code "Docker is local." See §4.
---
## 0. What already exists (grounding, not invention)
The current repo (`~/Dev/Gitea/ced4568/rootnode-academy`) already documents almost this exact
intent in its own README, just never built out:
> "The upstream project assumes one learner, one container. RootNode Academy runs it across
> three delivery modes... One container **per student** — never share a cluster between
> learners... Put authentication in front (proxy-level)... Idle-timeout + auto-teardown."
And the existing image already **is** a complete single-student lab: nginx + React frontend +
Node/Express backend + `node-pty` WebSocket terminal + a single-node k3s cluster, all in one
`--privileged` container (confirmed required — k3s needs kernel namespaces/cgroups, per
`README.md:52`). That container is the unit this design multiplies, not something it replaces.
What changes: today that one container also owns identity/progress (SQLite at
`/data/progress.db`, one volume per student, mounted by hand). This design pulls that
ownership out into a persistent Platform, and makes the container itself fully disposable.
---
## 1. Proposed architecture
Two roles — and in production, on **two separate VM/OS boundaries**, not just two logical
components sharing a box. Correction A drives this: `--privileged` Docker containers are not
a security boundary from their own Docker host, so the boundary has to be a real VM boundary,
not a process boundary.
**Production topology:**
```
Proxmox VE (or equivalent hypervisor)
│
├── RootNode Platform VM (trusted — never runs student code)
│ │
│ │ Platform API roster · login · progress · curriculum · exam records ·
│ │ environment registry · instructor controls
│ │ Platform DB PostgreSQL — environments + audit log now;
│ │ students/progress join later (§3/§9, correction C)
│ │ Provisioner talks OUT to a *configured* Lab Host endpoint —
│ │ never assumes Docker is local (§4, correction D)
│ │ Gateway opaque-token routing to student lab containers;
│ │ only component with a leg in both worlds
│ │
│ ▼ (Provisioner-initiated control channel only — TCP+TLS to the Lab
│ Host's Docker API. Nothing on the Lab Host VM ever initiates a
│ connection back. See §4/§7 for why this direction matters.)
│
└── RootNode Lab Host VM (untrusted — this is where privileged student
│ containers actually run; holds nothing the
│ Platform needs protected)
│
├── Student A's LAB ── own private network, own nested k3s, no identity
├── Student B's LAB ── own private network, own nested k3s, no identity
└── Student C's LAB ── own private network, own nested k3s, no identity
```
Each Lab is **today's `Dockerfile` and `entrypoint.sh`, functionally unchanged** — still runs
k3s + the terminal + scenario setup/validate/teardown exactly as now. The only change inside
it: it stops being the system of record for who's using it or what they've completed (§3, §9).
A compromised student container's worst case is the Lab Host VM — never Postgres, never the
Platform API, because there is no network path from that VM back to the Platform VM at all
(§4, §7). That containment is the actual point of the two-VM split, not a deployment nicety.
**Phase 1 development exception:** running the Platform and a Lab container on one machine is
fine for local testing (§9) — the Provisioner just gets configured with a single `lab_host`
pointing at the local Docker socket instead of a remote TLS endpoint. The *code* never
special-cases "local" vs "remote" — same abstraction either way (§4/§9 of
`docs/PHASE1_PLATFORM_FOUNDATION.md`).
---
## 2. Environment lifecycle
State machine per environment, tracked in the Platform DB (`environments` table, §3):
**Guaranteed path** (what correctness depends on): `requested → provisioning → running →
resetting → running` (reset loop, same route) and `(provisioning|running|error) → destroying
→ destroyed`. `provisioning → error` and `running → error` cover failed/lost environments;
`error → destroying → destroyed` (or `→ resetting → running`, when recoverable) is the way out.
```
requested → provisioning → running ─┬─→ resetting → running (clean slate, same route)
└─→ destroying → destroyed
provisioning → error (failed to reach ready state)
running → error (health check fails mid-session)
error → destroying → destroyed (or → resetting → running, if recoverable)
[EXPERIMENTAL, not relied on for correctness — see decision #3]
running ──suspend──→ suspended ──resume──→ running (or → error, if resume fails)
```
- **provision** — Provisioner picks a host with capacity (§5), creates a private network for
this student (§4), starts a container from the pinned Lab image, generates a random
high-entropy routing token, waits for the existing readiness signal (k3s API ready + backend
`/api/health` OK — same check the current entrypoint already does), writes the `environments`
row, hands the route back to the Platform API.
- **start** — used only for `provisioning → running` (starting a freshly created container).
There is no supported path that starts a previously-stopped container in the guaranteed
lifecycle — see suspend/resume below.
- **suspend/resume — EXPERIMENTAL, decision #3.** `docker stop`/`docker start` on a k3s-in-container
Lab is *not assumed to work* and is not built as part of the guaranteed lifecycle. It's
worth testing (see the Phase 1 test plan) because if it turns out reliable it's a cheap
latency win later — but nothing in the platform depends on it succeeding. The only supported
way to bring an environment back is `destroy` + `provision` (or `reset`, which is that same
pair against one `environments` row/routing token). This is also *why* academic state living
outside the Lab (§3) isn't optional — it's what makes "just destroy and recreate" a fully
safe default instead of a data-loss risk.
- **reset** — destroy and recreate the container from the pinned image, **same routing token
and same `environments` row**, so the student's "return to my lab" link doesn't change.
This is the instructor-facing "reset student environment" control.
- **destroy** — stop + remove container, remove its private network, release the routing
token, mark the `environments` row `destroyed` with a timestamp. Progress already lives in
the Platform DB (§3), so destroying an environment never loses academic data — only
in-terminal, not-yet-validated work.
- **Idle reaper** (new, required — not optional) — background job in the Provisioner checks
`last_active_at` per running environment (heartbeat from the Gateway or from Lab `/api/health`
polling) and auto-suspends past an idle threshold, auto-destroys past a longer abandoned
threshold. Without this, "ephemeral by default" is just a description, not a guarantee —
a closed browser tab does not stop a running k3s cluster.
---
## 3. Data model changes
Everything below lives in the **Platform DB**, not inside any Lab container.
Already built (previous slice, unaffected by this redirect — still correct under this
architecture):
- `students(student_id, display_name, cohort, created_at, last_active_at)`
- `progress(student_id, scenario_id, status, attempts, ...)` — composite key, already
per-student.
- `sessions` / `exam_progress` — exam records, already student-attributed.
New:
- `environments(environment_id UUID PK, student_id, status, routing_token, container_id,
lab_host_id FK, internal_ip, cpu_limit, mem_limit_mb, storage_limit_mb, image_ref, created_at,
started_at, last_active_at, destroy_at, error_reason)` — **`student_id` is an opaque external
identifier here (correction C), not a foreign key.** Students/progress stay on the existing
SQLite in Phase 1 (see §9); Postgres has no `students` table yet, so there is nothing to key
against. The Provisioner trusts whatever `student_id` its caller passes — it does not
validate it against a roster. A real FK gets added when students/progress actually migrate
onto this Postgres instance, not before. `lab_host_id`, by contrast, *is* a real FK — both
tables live in the same Provisioner-owned Postgres DB (correction D, full schema in
`docs/PHASE1_PLATFORM_FOUNDATION.md`).
- `lab_hosts(lab_host_id PK, endpoint, enabled, ...)` — the abstraction correction D requires:
which Docker endpoint a given environment's container actually lives on. Phase 1 ships with
exactly one row (pointing at either a local socket in dev or the Lab Host VM's TLS endpoint
in production) — the table exists so environment creation is never coupled to "Docker is
local" as an assumption, even though there's no scheduling logic yet.
- `environment_events(id, environment_id, event_type, actor, detail, created_at)` — audit
trail for provision/start/stop/reset/destroy/error, needed for both incident response and
ordinary "why did my lab disappear" support.
**Removed from the Lab image's responsibility:** the `students` and roster-facing parts of the
schema I added to `backend/server.js` in the previous slice move to be Platform-only. The Lab
image's own SQLite either goes away entirely or is kept only as scratch/ephemeral state that's
allowed to die with the container — it must not be treated as authoritative for anything.
**Decision #2: PostgreSQL, not SQLite.** The Platform control plane — the environment registry
specifically, and the target for progress/roster once they migrate over — is Postgres. The
existing single-container local dev mode keeps its own local SQLite exactly as today; that's a
different, unrelated database serving a different purpose (single-tenant local testing), not
something this decision touches. See `docs/PHASE1_PLATFORM_FOUNDATION.md` for the exact schema
being stood up first.
---
## 4. Networking model
- **One private Docker network per student environment**, not one shared network for all
labs. Docker's default per-network isolation means Student A's container has no L2/L3 path
to Student B's container at all — this is stronger and simpler than trying to firewall
students apart on a shared network.
- **No published ports on Lab containers.** Today's image publishes `-p 7554:80`; under this
design that stops — the only thing that can reach a Lab container is the Gateway, over that
student's private network.
### Egress — correction B: the firewall is the control, not the proxy env vars
`HTTP_PROXY`/`HTTPS_PROXY` are set on each Lab container so well-behaved processes (containerd
pulling scenario images) route through the allowlisting proxy cleanly. **That is configuration,
not enforcement** — a process that ignores those env vars and dials out directly must still be
blocked, by something that doesn't depend on the container's cooperation. That something is a
host firewall rule, evaluated on every packet regardless of what any process inside the
container does.
**Ordering bug in the original draft, corrected here:** the previous version of this section
listed proxy/DNS `ACCEPT` rules before the RFC1918 `DROP` rules in prose, but specified
installing them with `iptables -I` (insert-at-head) in that same top-to-bottom order — which
actually reverses the result, since each `-I` without a rule number pushes the *new* rule to
position 1. Run in that order, the RFC1918 drops end up evaluated *before* the proxy/DNS
accepts, silently blocking the proxy itself if it happens to sit on an RFC1918 address (it
will — it's Platform-side infrastructure). Corrected mechanism (exact rules in
`docs/PHASE1_PLATFORM_FOUNDATION.md` §5): each student network gets its own dedicated iptables
chain, built with `-A` (append, so rules land in the order they're added — no head-insertion
trap), containing explicit `ACCEPT` rules for the egress proxy, DNS, and any specifically
required Platform endpoint *first*, then the RFC1918 `DROP`s, then a final default-`DROP`. That
chain is reached from `DOCKER-USER` (the chain Docker guarantees it won't silently rewrite) via
one jump rule matching that student's subnet.
**What no student lab may ever reach, regardless of allowlist state:**
- RootNode management networks
- the Platform database (Postgres)
- Docker/Proxmox management interfaces (including the Lab Host VM's own Docker API endpoint —
a Lab container talking back to the thing that provisioned it is exactly the escalation path
this design exists to prevent)
- CDAA infrastructure
- home/trusted networks
- other students' lab networks (already unreachable via Docker's own per-network isolation
above; the RFC1918 drop is a deliberate second layer, not redundant — it also catches any
case where two student networks' address ranges could otherwise see each other at the host
routing level)
**Decision #1, restated with the ordering fix:** restricted egress, explicit allowlist, no
unrestricted internet access ever, enforced at the firewall layer with required infrastructure
explicitly permitted *before* the blanket denies — not the reverse, and not left implicit.
### VM-to-VM boundary — correction A
The Lab Host VM has **no route back to the Platform VM at all**, in either direction, except
the one the Provisioner itself opens. The Provisioner (on the Platform VM) connects *out* to
the Lab Host's Docker API over TCP+TLS with a client certificate only the Provisioner holds;
nothing on the Lab Host VM — student containers or the VM's own OS — can open a connection back
toward the Platform VM. This is enforced at the hypervisor/network level (Proxmox VLAN/firewall
rules between the two VMs), the same category of control as the CDAA/home boundary below, not
something Docker networking alone provides. A compromised student container's blast radius
stops at the Lab Host VM.
**No route to trusted CDAA/home/business networks, full stop.** This is a host/hypervisor
boundary, not something Docker networking alone can guarantee — the box(es) these containers
run on need to sit on their own segment with no bridge/route to those networks, consistent with
the existing `RootNode_Academy_Business_Pack/06_Technology/01_Technical_Platform_Plan.md`
("dedicated student network/DMZ separated from trusted networks"). This design assumes that
host-level segmentation is a prerequisite, not something the app layer can create on its own.
### Lab host abstraction — correction D
The Provisioner never talks to "the local Docker socket" as a hard-coded assumption — it talks
to whichever `lab_hosts` row it's configured to use (§3), resolved to a Docker endpoint that
may be `unix:///var/run/docker.sock` (Phase 1 dev, single machine) or a remote
`tcp://<lab-host-vm>:2376` with TLS (production, per the topology in §1). Phase 1 ships exactly
one enabled `lab_host` and zero scheduling logic — the abstraction exists so adding a second
host later is a config row, not a rewrite.
- **Gateway** is the only component with a leg in both the public-facing network and every
student's private network. It resolves `routing_token → container internal IP:port` by
querying the Platform API (or a shared registry cache) per request/connection, then proxies
both HTTPS (scenario API, static frontend) and the WSS terminal connection.
---
## 5. Resource model
- Per-environment caps applied at container-create time: `--cpus`, `--memory` (with
`--memory-swap` pinned equal to `--memory` so nothing swaps), `--pids-limit` (fork-bomb
guard). Storage: needs a size-capped volume — exact mechanism (`--storage-opt size=`, a
loopback-backed bind mount, or a tmpfs cap) depends on the host's actual Docker storage
driver, which I haven't inspected — needs to be confirmed against the real deploy target
before picking one.
- **Decision #4: policy-driven, not hard-coded.** Limits come from config, not constants in
code — a resource-policy file the Provisioner reads at startup (and can reload), not numbers
baked into the create-container call. No specific numbers are chosen yet. Target host class
for planning purposes: ~16 cores/32 threads, 128GB RAM, SSD/NVMe. Actual per-lab limits and
how many environments run concurrently on that class get set by benchmarking (Phase 1's test
plan includes the first real measurements), not by guessing backward from the spec sheet.
- **Admission control:** Provisioner refuses to `provision` past the point where already-running
environments' summed limits would exceed host capacity. Simple counter/gate to start; can
become real bin-packing later if there's ever more than one host.
---
## 6. Routing/auth model
1. Student logs in at the Platform (existing roster/pick-your-name flow, unchanged).
2. Frontend asks the Platform for "my lab." If none running, Platform calls the Provisioner to
`provision` (or `start`, if one's suspended).
3. Platform returns an **opaque, high-entropy routing token** scoped to
`(student_id, environment_id)`, short TTL, revoked on `destroy`.
4. Frontend connects to the Gateway using that token — `https://labs.<domain>/e/<token>/...`
and `wss://labs.<domain>/e/<token>/shell-ws`. The Lab container itself never sees a
student identity; it's purely an execution surface addressed by an opaque route.
5. Gateway validates the token maps to a currently `running` environment before proxying —
rejects stale tokens from destroyed/reset environments outright, so there's no window where
an old link reaches a different (possibly now-different-student's) container.
6. **Instructor auth** extends the shared-passcode model already used for the roster-add
endpoint: instructor-only calls (`list environments`, `reset`, `terminate`, `resource usage`)
go through the Platform API under that same gate, which then calls the Provisioner
internally. Students and the public internet never talk to the Provisioner or the Docker
socket directly — only the Platform backend does.
---
## 7. Security boundaries (defense in depth)
1. **Browser ↔ Gateway** — public-facing, TLS, opaque routing tokens only. Nothing
cluster-admin-shaped is reachable from here.
2. **Gateway ↔ Lab container** — private per-student Docker network only, no published ports,
Gateway is the sole entry point.
3. **Inside the Lab container** — `--privileged` is required for k3s (confirmed, not
avoidable — this isn't a place to try to "hold the line" without breaking k3s). The
mitigation is containment, not avoidance: this privileged container has no reachable
network path to anything except the Gateway, runs as a throwaway that gets destroyed/reset
routinely, and cluster-admin *inside a student's own disposable cluster* is expected and
fine — that's the entire point of full isolation, a student doing CKS-tier cluster-admin
work only ever affects their own throwaway cluster.
4. **Lab container ↔ its own Docker host (correction A — the boundary that actually matters
here)** — `--privileged` is explicitly **not treated as a sufficient isolation boundary from
its own Docker host.** In production the Lab Host VM holding these containers is a separate
VM from the Platform VM, with no route back to it at all (§4). If a student ever escapes
their container, the ceiling is the Lab Host VM — never Postgres, never the Platform API,
never another student's environment.
5. **Provisioner ↔ Lab host's Docker API** — only the Provisioner can invoke it, over a
connection *it* initiates (TCP+TLS, client-cert auth, per §4's lab-host abstraction). The
Docker endpoint is never student-reachable, never internet-reachable, and nothing on the Lab
Host VM can initiate a connection back toward the Provisioner or the Platform VM.
6. **Platform VM ↔ other networks** — dedicated segment, no bridge to home/business/CDAA
trusted infrastructure, matching the existing Technical Platform Plan doc. Prerequisite, not
something this design builds. Same standard applies to the Lab Host VM independently — it
isn't exempted just because it's "the untrusted one."
7. **Routing tokens** — CSPRNG-generated, short TTL, invalidated on any lifecycle transition
out of `running`.
---
## 8. Failure/recovery behavior
- **Provisioning fails** (host at capacity, image pull fails, k3s doesn't reach ready state in
time) — environment marked `error`, student gets an explicit "your lab failed to start, try
again" message (never a silent hang), one automatic retry, then surfaced to the instructor
dashboard (future) as a flagged failure.
- **Mid-session crash** — Gateway's health check detects the upstream is unreachable, marks
the environment `error`, offers the student a "restart my lab" action (→ `reset`, since a
crashed in-container k3s is unlikely to resume cleanly). Because progress/identity/exam
records live in the Platform DB, this never loses the student's academic record — worst case
is losing in-progress, not-yet-validated terminal work.
- **Host failure / reboot** — every environment on that host becomes unreachable. On
Provisioner restart, it must **reconcile its registry against actual running containers**
rather than trust stale DB state — some containers may auto-restart via Docker's restart
policy, most won't; anything the registry claims is `running` that isn't actually running
gets marked `error`, not silently left wrong.
- **Idle/abandoned environments** — the reaper (§2) bounds resource leakage from crashed tabs
or students who just walked away, independent of any other failure path.
- **Core resilience property of this whole design:** because the Platform DB — not the
disposable environment — is the system of record, every failure mode above is recoverable
without academic data loss. That's the actual payoff of the split in §1/§3, not just an
architectural nicety.
---
## 9. Migration path from the current single shared container
- **Phase 0 (already true today, keep it):** running the existing image standalone —
`docker run --privileged -p 7554:80 ... rootnode-academy:latest` — is exactly the
single-user local dev/test mode you asked to keep available. No change needed to support
this; it already works and stays supported as-is.
- **Phase 1:** Build Provisioner + `environments`/`environment_events`/`lab_hosts` registry
around the **existing, unmodified** Lab image, on Postgres, with the egress firewall/proxy
from §4 in place from the start (not bolted on later). Development exception (correction A):
running Platform + Provisioner + one Lab container all on a single machine is fine here — the
Provisioner is configured with one `lab_host` pointing at the local Docker socket instead of
a remote TLS endpoint, same code path either way. Prove two lab environments can run side by
side, fully isolated from each other, before any production traffic or a second VM enters
the picture. `students`/`progress` stay on the existing SQLite through this phase
(correction C) — `environments.student_id` is opaque, not a foreign key, until a later,
separate migration actually moves roster/progress onto Postgres.
- **Phase 2 (the one real code change inside the Lab image):** narrow `backend/server.js`
inside the Lab image down to a "dumb executor" — scenario setup/validate/teardown/answer and
the WS terminal stay, but it stops owning `students`/roster and stops being the source of
truth for progress. Either it calls back to the Platform API to report validation results,
or — cleaner — the Platform API becomes the only thing the frontend talks to for
progress/curriculum/identity, and it calls the Lab container (via the route the Provisioner
already tracks) purely to execute against that student's cluster and get a pass/fail back.
- **Phase 3:** Wire instructor controls (list/reset/terminate/resource-usage) against the new
registry — this is also where the "no auth on roster-add" gap from the previous slice gets
closed for real, since instructor auth is now a load-bearing security boundary, not just
roster hygiene.
- **Phase 4 (cutover):** point the real domain at the Gateway instead of a single container.
Keep the Phase 0 single-container path available under local/dev tooling, per your explicit
requirement.
- **Rollback safety:** at every phase through Phase 3, the current single-shared-container path
keeps working completely unmodified — nothing about this migration risks the working
baseline until the Phase 4 cutover, which is also the only step that's meaningfully
reversible-with-effort rather than trivially reversible.
---
## 10. Exact files/components this touches (once approved — no code written yet)
**New, none of it touching the existing Lab image's core logic:**
- `platform/provisioner/` — Docker lifecycle management against a *configured* `lab_host`
endpoint (correction D — never a hard-coded local socket), `environments`/`lab_hosts` CRUD,
per-environment firewall chain management (correction B), idle reaper, admission control.
- `platform/egress-proxy/` — the shared allowlisting proxy every Lab container's egress must
pass through (correction B).
- `platform/gateway/` — token→route resolution, HTTPS + WSS proxying (candidate: Traefik with
a dynamic provider backed by the Platform API, or a small custom Node proxy — not decided,
not needed for this design review).
- Platform DB migrations: `environments`, `environment_events`, `lab_hosts` tables — full
schema in `docs/PHASE1_PLATFORM_FOUNDATION.md`.
**Modified, once implementation starts:**
- `backend/server.js` — split as described in §9 Phase 2. This is presently both "the lab
image's backend" and "the only backend that exists"; under this design it forks into a
narrower Lab-side executor and a separate, persistent Platform-side API. Exact file layout
for that split is an implementation decision, not part of this design doc.
- `README.md` — the "Multi-Student Deployment" section (`README.md:128-146`) currently
documents per-student **volumes** for persistence; that persistence model is superseded by
§3 above and the README needs updating once this ships, so it doesn't contradict reality.
- `docs/ENGINEERING_SOP.md` — new section for the Platform/Provisioner/Gateway components and
the reconciliation/reaper operational procedures from §8.
**Unchanged:**
- `Dockerfile`, `scripts/entrypoint.sh`, `scripts/nginx.conf` for the Lab image itself — it's
still one container running k3s + backend + terminal, exactly as today, just multiplied and
stripped of identity ownership.
- All scenario content (`scenarios/data/`, `scenarios/bundles/`, `scenarios/rootnode-path.json`)
— unaffected either way; every scenario now runs with full admin of its own disposable
cluster, so the earlier `shared_namespace`/`ephemeral_cluster` split has no bearing here.
**New Dockerfile needed (not yet written):** one for the Provisioner+Gateway services — these
are new components, not variants of the existing Lab image.
---
## Status
All four open questions this doc originally ended with are resolved — see the decisions block
at the top and §2–§5 above. Next: `docs/PHASE1_PLATFORM_FOUNDATION.md` — the concrete build
plan for the platform foundation (Postgres environment registry, provisioner skeleton,
networking, egress, resource policy, reaper) against these decisions.