5e4f5518fd59a616cdb5ddfd35606ca1fe03984f
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  ced4568andClaude Sonnet 5
|
c89e39c5ee |
feat(gateway): add environment-aware HTTP and websocket routing
New internal component (platform/gateway/) — the only student-facing path to
disposable Lab environments. Browsers never reach a Lab container, the
Provisioner, or Postgres directly.
Routes:
- /api/environment/{status,launch,reset} — student-facing glue (X-Student-Id),
calls the Provisioner internally (shared secret never reaches the browser).
Reuses an existing non-terminal environment on launch rather than always
provisioning a new one; reset destroys+recreates the same environment row.
- /lab/<token>/* — HTTP+WS proxy into that one student's Lab container,
resolved by opaque routing token via a new Provisioner endpoint
(GET /environments/by-token/:token). No raw container IP/port ever reaches
the browser. Rejects invalid tokens (404), unready environments (202/503),
and cross-student access via an optional ?sid= ownership check (403) — see
docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md "Security" for why token-alone,
student-id-alone, and env-id-alone are each individually insufficient and
this is explicitly pilot-grade, not production auth.
- everything else — proxied to the persistent identity/curriculum backend
(roster, login, curriculum, progress, platform-shell frontend), or to the
student's own Lab if a valid lab-session cookie is present.
Routing design note: the existing Lab UI makes root-relative calls
(fetch('/api/scenarios'), the /shell-ws WebSocket) with no idea it might be
served under a /lab/<token>/ prefix — preserving that UI unmodified was a
hard requirement. Solved with an HttpOnly, SameSite=Lax cookie set on the
first /lab/<token>/ response: every later request from that browser carries
it, so the unmodified Lab UI's own unprefixed calls resolve correctly once a
student has navigated into a lab session, with zero changes to its code.
/api/environment/* is explicitly exempted from cookie-based routing so it
never gets hijacked by an active lab session.
Also added platform/docker-compose.dev.yml (the full local stack: Postgres +
egress-proxy + Provisioner + a persistent "identity" instance + Gateway) and
platform/README.md tying the components together. The "identity" instance
required an explicit `network_mode: bridge` — under Compose's own
auto-created bridge network, k3s failed at startup with "no default routes
found in /proc/net/route"; documented as a known limitation, not fully
root-caused.
Found and fixed during testing: express.json() applied globally broke
http-proxy's re-streaming of proxied POST bodies (e.g. POST /api/roster hung
forever, since the body had already been consumed by the parser before
proxy.web() tried to forward it). Removed — none of the Gateway's own routes
actually need a parsed body.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
|
||
|
ced4568andClaude Sonnet 5
|
9359abc9cb |
feat(reaper): add idle environment cleanup
Background loop (src/reaper.js), config-driven idle/abandoned thresholds and poll interval (no hard-coded numbers) — without this, "ephemeral by default" is just a description: a closed browser tab does not stop a running k3s cluster on its own. Past the idle threshold: logs a warning event only. Past the abandoned threshold: destroys via the same path DELETE /environments/:id uses, so cleanup is identical either way, and logs event_type='reaped_idle'. Failure-safe — one environment's teardown throwing never stops the reaper from checking the rest, and a Postgres/Docker hiccup on one tick just means "try again next tick," never a crashed process. Uses COALESCE(last_active_at, created_at) rather than bare last_active_at. Found during testing: an environment that fails during provisioning (before ever reaching 'running') has a NULL last_active_at, and NULL < anything is never true in SQL — without the fallback, a failed-and-never-cleaned-up environment would never have aged out here regardless of how long it sat there. This is the second, independent safety net on top of provisionEnvironment's own best-effort cleanup on failure (see the platform-provisioner commit). Verified end-to-end: backdated an environment's last_active_at past the abandoned threshold, waited for a real reaper tick, confirmed full deterministic cleanup (container, network, firewall chain all gone) with the correct actor='reaper' attribution in environment_events. Co-Authored-By: Claude Sonnet 5 <[email protected]> |
||
|
ced4568andClaude Sonnet 5
|
0116ee1522 |
feat(networking): enforce isolated lab egress policy
Egress enforcement is a firewall-layer control, not an application-layer convention (correction B). HTTP_PROXY/HTTPS_PROXY configure containerd for a clean pull path; they enforce nothing on their own. - src/egress.js — per-environment dedicated iptables chain (ROOTNODE-ENV-<short-id>), built with -A (append, order-preserving) rather than the -I (insert-at-head) pattern an earlier design draft used, which would have installed rules in reverse order and let the default-deny shadow the proxy/DNS accepts. Required infrastructure (egress proxy, DNS) is explicitly permitted first, then RFC1918 ranges are dropped unconditionally, then a final default-deny catches the public internet. One jump rule per environment hooks its subnet into its chain from DOCKER-USER (the chain Docker guarantees it won't silently rewrite). Destroy is deterministic: unhook, flush, delete — no comment-grep matching. The ordering fix is empirically verified, not just reasoned about: the egress proxy's own address is itself an RFC1918 address (it's Platform-side infrastructure), and testing confirms it's reachable while the broader RFC1918 range is still blocked — exactly the scenario the original ordering bug would have broken. - platform/egress-proxy/ — shared Squid instance (one per deployment, not per-student), destination-allowlisted (dstdomain ACL against a plain-text allowlist file), starting set covers docker.io/ghcr.io/gcr.io/quay.io/k8s.io. Proxy performs domain allowlisting; the firewall chain above is what makes it unbypassable, not the other way around. The egress proxy's address is resolved per-student-network (each network's Docker gateway IP, since the proxy runs with host networking and is present on every bridge gateway) rather than a fixed hostname — a static address wouldn't be reachable from more than one isolated student network at a time. Found while testing with two simultaneous environments. Verified: student-to-student isolation (A cannot reach B's internal IP), RFC1918 blocking (A cannot reach a simulated trusted-infra address), and the proxy allowlist itself (CONNECT tunnel to an allowlisted host succeeds, non-allowlisted gets a clean 403 from Squid). Co-Authored-By: Claude Sonnet 5 <[email protected]> |
||
|
ced4568andClaude Sonnet 5
|
c8dd33522f |
feat(provisioner): add disposable lab lifecycle
Internal-only service (platform/provisioner/) implementing the environment lifecycle from docs/PHASE1_PLATFORM_FOUNDATION.md: provision -> running -> (reset -> running | destroy -> destroyed), plus experimental suspend/resume (decision #3 — never on the guaranteed path, never relied on for correctness). - src/docker.js — the lab-host abstraction (correction D). Resolves a lab_hosts row into a connected dockerode client, unix:// socket or tcp:// with TLS client-cert auth. Refuses to connect to a tcp:// endpoint with no TLS material configured rather than connecting unauthenticated. Validated against both a local socket and a real remote Docker-in-Docker TLS endpoint during testing — same code path either way, only the lab_hosts config differs, which is the entire point. - src/lifecycle.js — provision/destroy/reset/suspend/resume orchestration, wired to docker.js + egress.js + the Postgres registry. Every Lab container is privileged (required for k3s, not avoidable), gets its own Docker network, no published host ports, deterministic naming (rootnode-lab-<short-id> / rootnode-lab-net-<short-id>), and a CSPRNG routing token. Includes reconcileOnStartup(), implementing the "don't trust stale state after a crash/restart" requirement from docs/LAB_ARCHITECTURE.md §8 — verified by killing a container out-of-band and confirming a Provisioner restart flags the mismatch instead of trusting the stale 'running' row. Found and fixed during testing: a provisioning failure (e.g. readiness timeout) left an orphaned container/network/firewall chain behind with only the DB row marked 'error'. provisionEnvironment's catch block now does best-effort teardown of whatever was already created before marking the row error — confirmed by deliberately failing a provision and checking no container/network/chain survived it. - src/server.js — the internal HTTP API (X-Provisioner-Key shared-secret auth on every route, matching the instructor-passcode pattern already used for the roster endpoint). Input validation on student_id/environment_id, structured JSON errors with a request id, routing tokens redacted from every response. - docker-compose.dev.yml — Postgres + egress-proxy + provisioner on one machine (the documented Phase 1 dev exception). provisioner and egress-proxy run with network_mode: host + NET_ADMIN/NET_RAW, which is what lets src/egress.js manipulate the host's DOCKER-USER chain without needing interactive host sudo. Co-Authored-By: Claude Sonnet 5 <[email protected]> |
||
|
ced4568andClaude Sonnet 5
|
62241d26d2 |
feat(platform-db): add lab host and environment registry
New Phase 1 Platform database — PostgreSQL, separate from the Lab image's own SQLite (which keeps its current students/progress data untouched, decision #2 / correction C in docs/LAB_ARCHITECTURE.md). Schema (platform/provisioner/migrations/): - lab_hosts: the abstraction that keeps Docker connectivity out of code — endpoint (unix:// or tcp://+TLS), enabled flag. Phase 1 ships one row. - environments: environment_id (UUID), student_id (opaque external identifier, no FK — students table isn't in this database yet), lab_host_id (real FK), status, routing_token (CSPRNG via crypto.randomBytes, opaque), container/ network identity, resource limits, lifecycle timestamps. - environment_events: append-only audit trail, one row per lifecycle transition, never logs the routing token in plaintext (see lifecycle.js's redact()). File-based migration runner (db.js) matching the check-before-alter idempotency pattern already used in the Lab image's backend/server.js, applied on every Provisioner boot. Resource limits and the lab_hosts seed are config files (config/resource-policy.yaml, config/lab-hosts.seed.yaml), not database tables — there's no admin UI to manage them via a database yet, so a DB-backed policy table would just be a write path nothing uses. Co-Authored-By: Claude Sonnet 5 <[email protected]> |