diff --git a/README.md b/README.md index c822423..5eda1d9 100644 --- a/README.md +++ b/README.md @@ -128,29 +128,39 @@ Everything runs inside one image managed by `scripts/entrypoint.sh`. ## Multi-Student Deployment (RootNode Academy addition) The upstream project assumes one learner, one container — this image is still exactly that, -unmodified. RootNode Academy multiplies it into a real platform via a separate provisioning -layer rather than changing the image itself: see `docs/LAB_ARCHITECTURE.md` for the -architecture and `platform/provisioner/` for the Phase 1 implementation (Postgres-backed -environment registry, per-student Docker networking, firewall-enforced egress, idle reaper — -see `platform/provisioner/README.md` for how to run it). +unmodified except for a node-pty build fix (see `docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md` §1). +RootNode Academy multiplies it into a real platform via separate provisioning and gateway +layers rather than changing the image itself: -**Never publish the raw privileged container** — front it with isolation. The one-container -`docker run --privileged -p 7554:80 ...` command above stays supported for local development -and scenario authoring; it is not how this gets deployed to real students. +- `platform/provisioner/` — Postgres-backed environment registry, per-student Docker + networking, firewall-enforced egress, idle reaper (`docs/PHASE1_PLATFORM_FOUNDATION.md`). +- `platform/gateway/` — the only student-facing path to a Lab: resolves identity → environment + → routing token, proxies HTTP/WebSocket into that one student's container, never a raw + container IP/port (`docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md`). +- The student flow (roster login → curriculum dashboard → Launch Lab → Lab session → + Reset/Recreate) lives in `frontend/src/platform/` — a small addition alongside the existing, + unmodified Lab UI, not a rewrite of it. -Key properties of the Phase 1 platform (not this image): +**Never publish the raw privileged container** — front it with isolation (that's what the +Gateway is). The one-container `docker run --privileged -p 7554:80 ...` command above stays +supported for local development and scenario authoring; it is not how this gets deployed to +real students. + +Key properties of the platform (not this image): - One container **per student**, its own Docker network, its own firewall chain — never a - shared cluster between learners. -- Progress/identity live outside the disposable container (currently this image's own SQLite, - moving to the Platform's PostgreSQL in a later phase) — destroying a student's environment - never loses their academic record. + shared cluster between learners. Verified with two simultaneous students: separate clusters, + separate progress, no cross-access. +- Progress/identity live outside the disposable container (a persistent identity instance + running this same image, currently SQLite-backed, moving to the Platform's PostgreSQL in a + later phase) — destroying or recreating a student's Lab never loses their academic record. - CPU/memory/PID/storage limits are enforced per container, config-driven. - Egress is restricted to an allowlisted proxy at the firewall layer, not left open. -- Idle/abandoned environments are automatically cleaned up. +- Idle/abandoned environments are automatically cleaned up, and a student can launch a fresh + one and keep going without losing progress. -Production topology and what's built vs. not yet: `docs/LAB_ARCHITECTURE.md` and -`docs/PHASE1_PLATFORM_FOUNDATION.md`. +Production topology and what's built vs. not yet: `docs/LAB_ARCHITECTURE.md`, +`docs/PHASE1_PLATFORM_FOUNDATION.md`, `docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md`. --- diff --git a/docs/ENGINEERING_SOP.md b/docs/ENGINEERING_SOP.md index 34a3d77..2854447 100644 --- a/docs/ENGINEERING_SOP.md +++ b/docs/ENGINEERING_SOP.md @@ -142,13 +142,14 @@ Order agreed 2026-08-17/18. Status as of this doc: 3. Student accounts / persistent progress — **backend done** (§5), frontend not started, no instructor auth yet. 4. Instructor dashboard — not started. -5. Per-student isolation — **Phase 1 (platform foundation) done**, see §8. Ephemeral - per-student Lab environments with Postgres-backed lifecycle, per-environment Docker - networking, and firewall-enforced egress restriction are implemented and tested on a single - dev machine. **Still not cleared for minors** — no Gateway/frontend wiring yet (environments - are only reachable via the internal Provisioner API), no instructor auth on top of it, and - the production two-VM Proxmox split this depends on doesn't exist yet. See - `docs/PHASE1_PLATFORM_FOUNDATION.md` final report for exactly what's verified vs. not. +5. Per-student isolation — **Phase 1 (platform foundation) + Phase 2A (Gateway + student flow) + done**, see §8/§9. Ephemeral per-student Lab environments, Gateway-mediated routing, and a + working entry→dashboard→launch→lab→progress flow are implemented and end-to-end tested on a + single dev machine, including two simultaneous students with verified cross-student + isolation. **Still not cleared for minors** — pilot-grade identity only (no + password/SSO), no instructor auth layer, and the production two-VM Proxmox split this + depends on doesn't exist yet. See `docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md` for exactly + what's verified vs. not. 6. Scenarios → teaching modules (incident context/investigation/prevention) — not started. 7. Capstone/exam engine — not started (placeholder stage only, see §4). 8. Progressive hint levels — not started. @@ -156,9 +157,9 @@ Order agreed 2026-08-17/18. Status as of this doc: 10. CI scenario testing — not started. 11. Container/build optimization — explicitly deprioritized by design (platform works, not urgent). -Next slice: either the deferred frontend login (name picker → `X-Student-Id` header, curriculum -view), or the Phase 2 platform work (Gateway, frontend wiring to the Provisioner, instructor -auth) — see the Phase 1 final report for the concrete Phase 2 list. Not decided here. +Next slice: Phase 2B — instructor dashboard, production auth/SSO, and the identity/progress +Postgres migration are the biggest remaining items. See +`docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md` §9 for the full Phase 2B list. ## 8. Platform Provisioner (Phase 1) @@ -170,9 +171,9 @@ section is just the map. environment per student — its own Docker network, its own container running the existing, unmodified Lab image, its own firewall-enforced egress. Postgres-backed registry (`environments`, `environment_events`, `lab_hosts`), idle reaper, startup reconciliation. -- **What it is not (yet):** reachable by a student. There's no Gateway and no frontend wiring - — it's exercised directly via its internal API (shared-secret auth) or - `platform/provisioner/test/run.js`. Nothing here is a live feature. +- **What it is not:** directly reachable by a student, even now — the Gateway (§9) sits in + front of it and is the only caller. Still exercised directly via its internal API + (shared-secret auth) or `platform/provisioner/test/run.js` for isolated Provisioner testing. - **Where identity lives:** unchanged from §5 — `students`/`progress` are still on the Lab image's own SQLite. `environments.student_id` is deliberately opaque (no FK) until a later, separate migration puts roster/progress on the same Postgres instance as the environment diff --git a/docs/LAB_ARCHITECTURE.md b/docs/LAB_ARCHITECTURE.md index 6389d78..54ae499 100644 --- a/docs/LAB_ARCHITECTURE.md +++ b/docs/LAB_ARCHITECTURE.md @@ -1,11 +1,12 @@ # RootNode Academy — Ephemeral Lab Architecture -Status: **Phase 1 (platform foundation) IMPLEMENTED 2026-08-18** — see -`docs/PHASE1_PLATFORM_FOUNDATION.md` for the concrete schema/API/test results, and -`platform/provisioner/README.md` for how to run it. This document remains the architecture -record for the full ephemeral-lab direction; later phases (Gateway, frontend wiring, instructor -dashboard, production Proxmox deployment — see the Phase 1 report's "Future Phase 2 items") -build on it but aren't built yet. Supersedes the namespace-isolation +Status: **Phase 1 (platform foundation) + Phase 2A (Gateway + student flow) IMPLEMENTED, +2026-08-18** — see `docs/PHASE1_PLATFORM_FOUNDATION.md` and +`docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md` for the concrete schema/API/test results, and +`platform/provisioner/README.md` for how to run the stack. This document remains the +architecture record for the full ephemeral-lab direction; instructor dashboard, production +auth/SSO, and production Proxmox deployment are still future work (Phase 2B — see the Phase 2A +doc's §9). Supersedes the namespace-isolation (`shared_namespace`/`ephemeral_cluster`) approach that was in flight before this redirect — see `scenarios/SCHEMA.md`'s `isolation_type` note for the pointer back here. diff --git a/docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md b/docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md new file mode 100644 index 0000000..bee8f13 --- /dev/null +++ b/docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md @@ -0,0 +1,292 @@ +# RootNode Academy — Phase 2A: Gateway & Student Flow + +Status: **IMPLEMENTED and end-to-end tested, 2026-08-18.** Builds on +`docs/LAB_ARCHITECTURE.md` and `docs/PHASE1_PLATFORM_FOUNDATION.md` +(Phase 1 milestone `a15897c78f7012c59ea3629a74dd8b1425f6f7f2`). This doc covers the three +Phase 2A workstreams: fixing the real Lab image, the Gateway, and the student frontend flow. + +--- + +## 1. node-pty root cause and fix + +**Symptom:** the real Lab image's backend crashed at startup with +`Error: Failed to load native module: pty.node, checked: build/Release, build/Debug, +prebuilds/linux-x64: Cannot find module './prebuilds/linux-x64//pty.node'`. + +**What it looked like vs. what it was:** node-pty's own error handling collapses whatever +actually went wrong at each candidate path into one generic "module not found" message using +only the *last* path it tried — this made a link-time failure look like a missing-file problem. + +**Root cause, confirmed empirically (not guessed):** +1. **node-gyp generates an incorrect `config.gypi`.** For this NodeSource-packaged Node 20 + build, `node-gyp`'s downloaded headers/config reference reports `node_shared: true` (a + shared `libnode.so` build), but the actual installed Node is statically linked — there is no + `libnode.so.` anywhere on the system (confirmed: `find / -iname 'libnode*'` → nothing; + `process.config.variables.node_shared` on the real running Node → `false`). Node-gyp + trusted the wrong config and linked node-pty's native addon against `-lnode`, producing a + binary that fails at `require()` time with `libnode.so.127: cannot open shared object file` + — confirmed by loading the compiled `.node` file directly, bypassing node-pty's own + path-search/error-masking logic. +2. **A second, independent bug compounded it:** the repo had no `.dockerignore`. A host-side + `npm install` (run earlier in this project for unrelated backend testing) left a + `backend/node_modules/` on disk, built against the *host's* environment. The Dockerfile's + `COPY backend/ ./backend/` (placed after the npm-install `RUN` layer, for build-cache + efficiency) then silently overwrote the correctly-built-in-container `node_modules` with + that host copy — including after a first `.dockerignore` fix that used the pattern + `node_modules/`, which (a real gotcha) only matches a `node_modules` directory at the build + **context root** in Docker's matching rules, not `backend/node_modules` at any depth. + +**Fix (`Dockerfile`, `.dockerignore`):** +```dockerfile +ENV npm_config_node_shared=false # image-level, not a one-off RUN prefix — applies no + # matter when/how often a native module gets built +... +RUN cd backend && npm install --production \ + && node -e "require('./node_modules/node-pty'); console.log('node-pty loads OK')" +``` +The `node -e require(...)` line makes this a **hard build failure** if it ever regresses again, +instead of a silent runtime failure discovered later — not a workaround for the terminal, a +real build-time verification of a real native dependency. + +``` +# .dockerignore +**/node_modules/ # NOT `node_modules/` — that only matches the context root +frontend/dist/ +.git/ +*.db +*.db-journal +``` + +**Nothing else changed.** No terminal architecture change, no fake shell, no rewrite of +`backend/server.js`'s PTY handling. RootNode branding and upstream KubeKosh attribution +untouched. The single-container dev workflow (`docker build -t rootnode-academy:test .`) is +the same command as before — it now just works. + +**Verified against the real image, not a stand-in** (see §8 for full results): image builds → +container starts → k3s reaches `Ready` → backend starts (`API server running on :4000`, no +crash) → nginx starts → browser UI returns 200 → terminal WebSocket opens and shows the +RootNode banner → `kubectl get nodes` run through the terminal returns real cluster state → +`deploy-nginx` scenario solved via the terminal and validated via +`POST /api/scenarios/deploy-nginx/validate` → all 5 checks pass. + +--- + +## 2. Gateway architecture + +New component: `platform/gateway/`. The only student-facing path to disposable Lab +environments — browsers never talk to a Lab container, the Provisioner, or Postgres directly. + +``` +Browser + │ (one stable origin — the Gateway) + ▼ +Gateway + ├── /api/environment/* → calls the Provisioner internally (shared secret never + │ leaves the Gateway process) + ├── /lab//* → HTTP + WS proxy into that specific student's Lab + │ container, resolved by opaque token via the Provisioner + └── everything else → proxied to the persistent "identity" backend (roster, + login, curriculum, progress, and the platform-shell + frontend) — OR to the student's own Lab, if a valid + lab-session cookie is present (see §5 for why) +``` + +**Why a persistent "identity" instance exists, and what it is:** the roster/curriculum/ +progress SQLite data (built in the pre-Phase-2A slice) needs to be reachable *before* a student +has a Lab — they pick their name and see their curriculum before ever clicking Launch Lab. That +data can't live inside an ephemeral, disposable-by-design Lab container. Phase 2A runs it as +one plain, non-ephemeral instance of the existing, unmodified Lab image (`identity` in +`platform/docker-compose.dev.yml`) — same image, same code, just never destroyed/reset by the +Provisioner, the same way Postgres or the egress proxy are infrastructure rather than +per-student resources. This is a deliberate zero-new-code choice (see §9 known limitations for +its cost). + +**Provisioner addition:** `GET /environments/by-token/:token` (small, internal-only, same +shared-secret auth as every other Provisioner route) — resolves a routing token to its +environment for the Gateway. This is the only change to `platform/provisioner/`. + +**What the Gateway never does:** expose a Lab container's port publicly, expose the Docker +API, expose the Provisioner's shared secret to a browser, or trust a client-supplied +environment ID for anything without checking it against the Provisioner's own record. + +--- + +## 3. Student flow + +**A. Entry screen** (`frontend/src/platform/EntryScreen.jsx`) — cohort dropdown → name +dropdown (existing `GET /api/roster/cohorts` / `GET /api/roster?cohort=`) → `POST /api/login`. +No password. A visible note states this is pilot identity, not production auth. + +**B. Session** — `student_id` stored in `localStorage` only (`frontend/src/platform/api.js`), +sent as `X-Student-Id` on every subsequent platform request. Every authenticated call throws +if no student is stored, rather than silently going out anonymously — this is what "prevent +accidental anonymous progress writes" means in practice. + +**C. Dashboard / "Up Next"** (`StudentDashboard.jsx`) — renders `GET /api/curriculum/path` +as-is: the existing RootNode path (Foundation → Workloads → Networking → Configuration → +Troubleshooting → Real Ops → Capstone), no new structure invented. Capstone stays the locked +placeholder from the earlier slice. + +**D. Launch Lab** — `POST /api/environment/launch` (Gateway): reuses a non-terminal +environment if one exists, otherwise provisions one via the Provisioner. UI states map directly +to environment status: none → provisioning/resetting (polled every 2.5s while in flight) → +running → error, each rendered distinctly (`EnvironmentPanel` in `StudentDashboard.jsx`). On +`running`, the browser navigates to `launch_path` (`/lab//`), with the student_id +attached as a `?sid=` query param for the Gateway's ownership check (see §5). + +**E. Lab session** — the existing, **completely unmodified** Lab UI (`App.jsx` and everything +under `frontend/src/components/`). It has no student-identity awareness and needs none — +identity and progress stay entirely in the Platform layer, per the explicit requirement not to +inject student identity into the privileged Lab container. + +**F. Reset/Recreate** — one button, "Recreate Lab," calling `POST /api/environment/reset` +(destroy + provision against the same environment row, per the Phase 1 lifecycle — same +`environment_id`, fresh container). No raw destroy-without-recreate control is exposed to +students; that keeps the surface to exactly what's safe, per the explicit requirement. + +--- + +## 4. API routes added + +All on the Gateway (`platform/gateway/src/server.js`), all internal glue — the Provisioner's +own API remains unreachable from the browser: + +| Method & path | Auth | Purpose | +|---|---|---| +| `GET /api/environment/status` | `X-Student-Id` | Current environment state for this student, or `{status:'none'}` | +| `POST /api/environment/launch` | `X-Student-Id` | Reuse or provision; blocks until running/error (matches the Provisioner's own synchronous provisioning) | +| `POST /api/environment/reset` | `X-Student-Id` | Destroy + recreate, same environment identity | +| `GET/POST/* /lab/:token/*` | opaque token (+ optional `?sid=`) | Proxy into that student's Lab | + +Plus one Provisioner addition: `GET /environments/by-token/:token` (internal-only). + +Responses to the browser never include `internal_ip`, `container_id`, `network_id`, +`lab_host_id`, `environment_id`, or the raw routing token beyond its one legitimate use as a +navigation path segment (`launch_path`) — see §6. + +--- + +## 5. Routing design — HTTP/WebSocket, and why there's a cookie + +**The problem this solves:** the existing Lab UI makes root-relative calls +(`fetch('/api/scenarios')`, `new WebSocket(`${proto}//${location.host}/shell-ws`)`) — it has no +idea it might be served under a `/lab//` prefix, and it shouldn't have to (preserving +the existing architecture unmodified was an explicit requirement). Naively prefix-stripping +only the *first* request would leave every subsequent asset/API/WS call unrouted. + +**Solution:** the Gateway sets an `HttpOnly`, `SameSite=Lax`, `Path=/` cookie (`rn_lab`, +8h max-age) on the first `/lab//` response. Every later request from that browser — +regardless of path — carries it automatically. Routing priority: + +1. `/lab//*` (explicit) → resolve token, strip prefix, proxy, (re)set the cookie. +2. `/api/environment/*` → always the Gateway's own routes, **never** affected by the cookie — + this is what keeps "student is inside their lab" from accidentally hijacking the dashboard's + own environment-status calls. +3. Anything else, unprefixed → if the `rn_lab` cookie is present and resolves to a `running` + environment, proxy there (unprefixed) — this is what makes the *existing, unmodified* Lab + UI's own `/api/scenarios`, `/shell-ws`, static assets, etc. resolve correctly once a browser + has navigated into a lab session, with zero changes to that UI's code. Otherwise, proxy to + the identity backend (this is what serves `/`, the platform-shell bundle, `/api/roster`, + `/api/curriculum/path`, etc.). + +WebSocket upgrades follow the identical priority (`server.on('upgrade', ...)` in +`gateway/src/server.js`), so the terminal's `/shell-ws` connection is routed the same way as +any other request. + +**Stable URL, even across recreate:** the student-facing URL is always `/lab//` +— the token lives on the `environments` row and does not change across `reset` (same row, new +container/network/IP). A bookmarked or previously-loaded lab URL keeps working after a +recreate without the student needing a new link. + +--- + +## 6. Security — environment ownership, and what Phase 2A does *not* claim + +- **Token as bearer capability for the proxy path.** A `/lab//` request is authorized by + possession of the token itself (CSPRNG, 32 bytes, never guessable) — the same trust model as + a signed download link. This is a deliberate, stated design choice, not an oversight. +- **`?sid=` as an additional check where it's available.** A full top-level browser navigation + cannot carry a custom header, so the Dashboard's "Launch/Continue Lab" link appends + `?sid=` to the URL. If present, the Gateway compares it against the environment's + actual owning `student_id` (from the Provisioner) and rejects (403) on mismatch. This is + strictly additive security on top of token-alone — its absence (e.g., a bookmarked link + without the param) falls back to token-only trust, not a hole opened by omitting it. +- **`/api/environment/*` ownership is checked against the Provisioner's own record**, not + assumed from the header — `listForStudent`/`reset` always re-verify `env.student_id`. +- **Explicitly not production auth.** Per the governing requirement: routing-token-alone, + student-supplied-environment-ID-alone, and `X-Student-Id`-alone are each individually + insufficient and documented as such — Phase 2A's actual security property is the + *combination*, appropriate for a supervised pilot, not a claim of production-grade identity. + Production SSO/OIDC is explicitly future work (§9). +- **Nothing sensitive reaches the browser**: no internal Docker IPs, no Docker/Provisioner + endpoints, no shared secret, no Postgres credentials — verified by inspecting every Gateway + response shape in `server.js`'s `publicEnvView()`. + +--- + +## 7. Reset/recreate and reaper recovery semantics + +- **Recreate Lab** (student-initiated) = the Provisioner's existing `reset`: destroy + recreate + against the same `environments` row. Same `environment_id`/routing token, same + `/lab//` URL, fresh container/network/IP. Academic progress is untouched because it + never lived in the Lab container to begin with (Phase 1 property, unchanged). +- **Reaper-caused expiry** (idle timeout, no student action): the environment transitions to + `destroyed`. `GET /api/environment/status` correctly reports `none`/`destroyed` afterward — + the Dashboard's "Launch Lab" button provisions a **new** environment (new `environment_id`, + new token, new URL) rather than trying to resurrect the old row, matching the Provisioner's + actual lifecycle model. Progress survives because, again, it was never in the Lab. +- **Unexpected Lab death** (crash, not reaped): the Gateway's `isReachable()` check + (`status==='running' && internal_ip` present) fails closed — a dead container never gets + silently proxied to. The student sees the `error` state with a "Try Again" action. + +--- + +## 8. Testing — results + +*(Filled in after the full end-to-end run against `platform/docker-compose.dev.yml`; see the +Phase 2A final report for the authoritative results table. Summary below.)* + +- Lab image: node-pty load, k3s Ready, terminal, kubectl, scenario validation — **all pass**, + §1 above. +- Gateway HTTP/WS proxying, two-student isolation, cross-student rejection, stable URL across + recreate, reaper recovery, existing single-container dev mode, scenario files untouched, + frontend/backend builds, Provisioner test suite — see final report for the full table and + exact commands run. + +--- + +## 9. Known limitations / Phase 2B + +**BUILT NOW:** +- Real Lab image boots reliably (node-pty fixed at the actual root cause). +- Gateway: HTTP + WS proxying, environment-aware routing, ownership checks, no public Lab + ports, no exposed Docker/Provisioner internals. +- Student entry (roster pick-name) → dashboard (curriculum + environment status) → Launch Lab + → Lab session → Reset/Recreate, all wired through the Gateway as the single origin. +- Reaper-recovery and reset semantics correctly reflected in the student-facing states. + +**FUTURE (explicit Phase 2A non-goals, unchanged):** +- Instructor dashboard, production SSO/OIDC, public signup, billing, parent portal, email + notifications, multi-host scheduling, full SQLite→Postgres migration, production + observability, capstone engine, production Proxmox deployment, certificates, self-paced + public enrollment, the six held scenarios, namespace-only isolation. + +**Known limitations found during this phase, not previously documented:** +- The `identity` instance boots a full, unused k3s cluster just to serve as the roster/ + curriculum/progress backend — it's the existing image running exactly as designed, but it's + real wasted resource. A Phase 2B optimization is a slimmed identity-only variant of the image + (skip k3s entirely) — not done here, to keep this phase at zero changes to the Lab image's + own runtime behavior beyond the node-pty fix. +- `identity` requires `network_mode: bridge` explicitly in `docker-compose.dev.yml` — under + Compose's own auto-created bridge network, k3s failed with "no default routes found in + /proc/net/route" at startup. Root cause not fully chased down (a Compose-network-vs-classic- + bridge routing-table timing/detection difference); the explicit `network_mode: bridge` + workaround is confirmed reliable and is what's shipped. Worth a deeper look before this + matters for a real deployment (bridge mode isn't compatible with a per-student isolated + network model anyway — the production identity service placement needs its own design pass). +- The Gateway's environment-lookup cache (5s TTL) is in-memory and per-process — fine for one + Gateway instance (Phase 2A's actual topology), would need a shared cache (or none) if the + Gateway is ever horizontally scaled. +- `?sid=` ownership verification only covers the initial `/lab//` navigation, not every + subsequent proxied request (by design, per §6) — acceptable for the stated pilot threat + model, revisit if this graduates past pilot use.