diff --git a/README.md b/README.md index 84089bb..2bd1ab3 100644 --- a/README.md +++ b/README.md @@ -127,27 +127,35 @@ Everything runs inside one image managed by `scripts/entrypoint.sh`. ## Multi-Student Deployment (RootNode Academy addition) -The upstream project assumes one learner, one container — this image is still exactly that, -unmodified except for a node-pty build fix (see `docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md` §1). -RootNode Academy multiplies it into a real platform via separate provisioning and gateway -layers rather than changing the image itself: +The upstream project assumes one learner, one container — this image is still exactly that. +Its backend/entrypoint/Dockerfile are unmodified except for a node-pty build fix (see +`docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md` §1); its frontend bundle gained progressive-hint +controls and an assessment-mode indicator in Phase 3B (`docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` +§10–11) — additive UI, not a rewrite of the Lab experience. RootNode Academy multiplies the +image into a real platform via separate provisioning and gateway layers rather than changing +its core: - `platform/provisioner/` — Postgres-backed environment registry, per-student Docker networking, firewall-enforced egress, idle reaper (`docs/PHASE1_PLATFORM_FOUNDATION.md`). -- `platform/api/` — lightweight Platform API: students, cohorts, curriculum, progress, - scenario attempts, persistent student/instructor sessions, per-instructor accounts (RBAC: - owner/admin/instructor), audit log, instructor/admin control plane. No k3s, no `--privileged` - (`docs/PHASE2B_ADMIN_AND_PLATFORM_DATA.md`, `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md`). +- `platform/api/` — lightweight Platform API: students, cohorts, curriculum, progress, detailed + scenario attempt telemetry, progressive hints, assessment/capstone engine, achievements, + persistent student/instructor sessions, per-instructor accounts (RBAC: owner/admin/ + instructor), audit log, instructor/admin control plane. No k3s, no `--privileged` + (`docs/PHASE2B_ADMIN_AND_PLATFORM_DATA.md`, `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md`, + `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md`). - `platform/gateway/` — the only student-facing path to a Lab: resolves identity from a server-controlled, persistent session cookie (not a client-suppliable header/param), proxies - HTTP/WebSocket into that one student's container, never a raw container IP/port - (`docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md`, session model in - `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` §3). -- The student flow (roster login → curriculum dashboard → Launch Lab → Lab session → - Reset/Recreate) lives in `frontend/src/platform/`; the instructor/admin portal (per-instructor - login, cohorts, students, instructor accounts, audit log, classroom "Prepare Labs," - environment controls) lives in `frontend/src/admin/` at the `/admin` route — both small - additions alongside the existing, unmodified Lab UI, not a rewrite of it. + HTTP/WebSocket into that one student's container, never a raw container IP/port, and + server-gates progressive hint reveal by stripping the Lab's own hint payload before it reaches + the browser (`docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md`, session model in + `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` §3, hint gating in + `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` §3). +- The student flow (roster login → curriculum dashboard → Launch Lab → practice/assessments/ + capstone → achievements) lives in `frontend/src/platform/`; the instructor/admin portal + (per-instructor login, cohorts, students, instructor accounts, audit log, classroom + "Prepare Labs," environment controls, learning analytics, assessment/capstone management) + lives in `frontend/src/admin/` at the `/admin` route — both additions alongside the existing + Lab UI, not a rewrite of it. **Never publish the raw privileged container** — front it with isolation (that's what the Gateway is). The one-container `docker run --privileged -p 7554:80 ...` command above stays @@ -171,6 +179,12 @@ Key properties of the platform (not this image): recorded in an audit log. - Sessions (student and instructor) are Postgres-backed and survive a Platform API restart — verified by actually restarting it mid-session and confirming access still works. +- Every validation attempt, hint reveal, curriculum stage completion, exam, and Capstone + attempt is recorded in Postgres, not the disposable Lab — a Lab reset or recreate never loses + attempt telemetry, hint progress, or an in-progress assessment's timer. Verified live: + reset a Lab mid-assessment, timer and drawn scenario set were unchanged afterward. +- Exams and the RootNode Capstone use a server-authoritative timer and a server-drawn, + server-stored scenario set — nothing the browser sends can change either. - CPU/memory/PID/storage limits are enforced per container, config-driven. - Egress is restricted to an allowlisted proxy at the firewall layer, not left open. - Idle/abandoned environments are automatically cleaned up, and a student can launch a fresh @@ -178,7 +192,7 @@ Key properties of the platform (not this image): Production topology and what's built vs. not yet: `docs/LAB_ARCHITECTURE.md`, `docs/PHASE1_PLATFORM_FOUNDATION.md`, `docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md`, -`docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md`. +`docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md`, `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md`. --- diff --git a/docs/ENGINEERING_SOP.md b/docs/ENGINEERING_SOP.md index 0a67c53..c8f4b3c 100644 --- a/docs/ENGINEERING_SOP.md +++ b/docs/ENGINEERING_SOP.md @@ -71,14 +71,18 @@ Two separate structures, don't confuse them: - **RootNode path** (`scenarios/rootnode-path.json`) — the guided onboarding sequence, separate from bundles on purpose (it's *staged*, bundles are *flat*, and the bundle loader would choke on a staged file if it were dropped in `scenarios/bundles/`). Loaded by - `loadCurriculumPath()`/`GET /api/curriculum/path`. Stages are Foundation → Workloads → - Networking → Configuration → Troubleshooting → Real Ops → Capstone, currently built by - reusing existing `1-k8s-basics` and `rootnode-real-ops` scenarios grouped by their existing - `category` field — no new scenario content was written for this. **Capstone stage is a - locked placeholder** (`"locked": true`, empty `scenario_ids`) — the capstone engine itself - is a separate, not-yet-built roadmap item (see §7, item 6). Do not put real scenarios in it - until that engine exists, or you'll reproduce the "duplicate beginner scenarios as capstone" - anti-pattern this path was built to avoid. + `loadCurriculumPath()`/`GET /api/curriculum/path` (raw content) and, as of Phase 3B, + `computeCurriculumState()`/`GET /api/curriculum/state` (rule-driven lock/complete/next-scenario + state — see `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` §4). Stages are Foundation → Workloads → + Networking → Configuration → Troubleshooting → Real Ops → Capstone, built by reusing existing + `1-k8s-basics` and `rootnode-real-ops` scenarios grouped by their existing `category` field — + no new scenario content was written for this. Each stage carries a `completion_rule` + (`all_required`/`threshold`/`prerequisites`). **Capstone is no longer a hardcoded placeholder** + — it's `"type": "capstone"` with an `"unlock_rule"` and a `"capstone_assessment_id"` pointing + at the `rootnode-capstone` assessment definition (Postgres, `assessment_definitions` table), + which is where its actual scenario draw lives — see the Phase 3B doc §7. Don't put real + scenarios directly into the JSON stage's `scenario_ids` for capstone; that field stays empty + by design, the draw is defined in the assessment definition instead. To add/reorder a stage: edit `rootnode-path.json` directly, referencing existing `scenarios/data/*.json` ids. No backend code change needed — it's read fresh via @@ -138,7 +142,9 @@ Order agreed 2026-08-17/18. Status as of this doc: 1. RootNode identity/branding pass — **partial.** Shell banner fixed (`⎈ RootNode Academy Lab — node: k8s-lab`). Upstream KubeKosh attribution left untouched in footer/README/LICENSE. Broader "make Real Ops first-class" UI work not started. -2. RootNode curriculum mode — **backend done** (§4), frontend not started. +2. RootNode curriculum mode — **done**. Rule-driven stage completion/unlock as of Phase 3B + (§12), student-facing tabs (My Learning/Practice/Assessments/Capstone/Achievements) at + `frontend/src/platform/`. 3. Student accounts / persistent progress — **done**, Postgres-authoritative as of Phase 2B (§9), sessions persistent as of Phase 3A (§10). Roster name-picking is still classroom/pilot identification, not strong student auth — documented, not silently glossed over @@ -156,15 +162,18 @@ Order agreed 2026-08-17/18. Status as of this doc: documented adapter, not a working integration (docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md §16). 6. Scenarios → teaching modules (incident context/investigation/prevention) — not started. -7. Capstone/exam engine — not started (placeholder stage only, see §4). -8. Progressive hint levels — not started. +7. Capstone/exam engine — **done** as of Phase 3B (§12). RootNode Capstone (Real Ops-weighted, + unlockable by curriculum-completion rule, not a permanent placeholder) plus a general exam + engine (definitions, cohort assignment, random scenario draw, server-authoritative timer). +8. Progressive hint levels — **done** as of Phase 3B (§12). Server-enforced sequential reveal, + tracked per attempt, policy-aware under assessment mode (free/disabled/flagged). 9. Production observability — not started. 10. CI scenario testing — not started. 11. Container/build optimization — explicitly deprioritized by design (platform works, not urgent). Next slice: production OIDC/SSO, the production two-VM Proxmox deployment, and teaching-module scenario content are the biggest remaining items. See -`docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` §18 for the full BUILT NOW vs. FUTURE split. +`docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` §21 for the full Phase 4 boundary. ## 8. Platform Provisioner (Phase 1) @@ -246,3 +255,28 @@ Going forward: after any `npm install` that changes a service's dependencies, th `package-lock.json` is part of the same commit as the `package.json` change — never left untracked, never regenerated-and-ignored. Validate a lockfile actually matches its `package.json` with `npm ci` (fails loudly on drift) before committing either. + +## 12. Learning & Assessment Engine (Phase 3B) + +Full design: `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md`. + +- **What it is:** the teaching/assessment layer on top of everything Phases 1–3A built — + detailed per-attempt telemetry (`scenario_attempts`, evolved from Phase 2B's one-row-per-click + shape into a real attempt lifecycle), server-enforced progressive hints, rule-driven + curriculum stage/capstone unlock logic, a full exam + capstone assessment engine + (`assessment_definitions`/`assessment_sessions`/`assessment_session_scenarios`), and + persistent `achievements` records. +- **Where the boundary moved:** the Gateway now rewrites one Lab response + (`GET /lab/:token/api/scenarios/:scenarioId`, stripping `hints`) so progressive hints can be + genuinely server-gated rather than just UI-collapsed — the Lab image's own code/bytes are + still untouched; this is a response rewrite at the Gateway boundary, same pattern as every + other Lab-mediation decision made in earlier phases. The Lab's own frontend bundle *did* + change this phase (`ScenarioPanel.jsx`, new `ProgressiveHints`/`AssessmentBanner` components) + — Phase 3B explicitly authorized enhancing Lab UX, unlike prior phases. +- **Test procedure:** `platform/test/e2e-phase3b.js` — covers attempt telemetry, hint + sequencing/persistence-through-Lab-reset, curriculum/capstone lock-unlock, the exam engine + (random draw, timer, expiry, restart persistence), two-student isolation, and instructor + analytics/RBAC. +- **Known limitations:** see the Phase 3B doc §20 — no automatic abandoned-attempt detection, no + per-cohort needs-attention threshold configuration, raw-JSON pool_spec authoring in the admin + UI (correct, not yet a guided form). diff --git a/docs/LAB_ARCHITECTURE.md b/docs/LAB_ARCHITECTURE.md index 43962be..dc0a924 100644 --- a/docs/LAB_ARCHITECTURE.md +++ b/docs/LAB_ARCHITECTURE.md @@ -2,14 +2,16 @@ Status: **Phase 1 (platform foundation) + Phase 2A (Gateway + student flow) + Phase 2B (Platform API split, Postgres academic data, server sessions, admin portal) + Phase 3A -(persistent sessions, per-instructor accounts, RBAC, audit log) IMPLEMENTED, 2026-08-18** — see -`docs/PHASE1_PLATFORM_FOUNDATION.md`, `docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md`, -`docs/PHASE2B_ADMIN_AND_PLATFORM_DATA.md`, and `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` -for concrete schema/API/test results, and `platform/README.md` for how to run the stack. This -document remains the architecture record for the full ephemeral-lab direction; production -OIDC/SSO (an adapter interface exists, not a working integration — see the Phase 3A doc §16) -and production Proxmox deployment are still future work (Phase 3B+ — see the Phase 3A doc §18). -Supersedes the namespace-isolation +(persistent sessions, per-instructor accounts, RBAC, audit log) + Phase 3B (learning/assessment +engine: attempt telemetry, progressive hints, curriculum rules, exam + capstone engine, +achievements) IMPLEMENTED, 2026-08-18** — see `docs/PHASE1_PLATFORM_FOUNDATION.md`, +`docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md`, `docs/PHASE2B_ADMIN_AND_PLATFORM_DATA.md`, +`docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md`, and +`docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` for concrete schema/API/test results, and +`platform/README.md` for how to run the stack. This document remains the architecture record +for the full ephemeral-lab direction; production OIDC/SSO (an adapter interface exists, not a +working integration) and production Proxmox deployment are still future work (Phase 4 — see the +Phase 3B doc §21). Supersedes the namespace-isolation (`shared_namespace`/`ephemeral_cluster`) approach that was in flight before this redirect — see `scenarios/SCHEMA.md`'s `isolation_type` note for the pointer back here. diff --git a/docs/PHASE3B_LEARNING_AND_ASSESSMENT.md b/docs/PHASE3B_LEARNING_AND_ASSESSMENT.md new file mode 100644 index 0000000..8925916 --- /dev/null +++ b/docs/PHASE3B_LEARNING_AND_ASSESSMENT.md @@ -0,0 +1,370 @@ +# Phase 3B — Learning & Assessment Engine + +Turns the platform infrastructure built in Phases 1–3A (identity, sessions, RBAC, disposable +Labs, Gateway routing, admin portal) into a real teaching and assessment system: detailed +attempt telemetry, progressive hints, rule-driven curriculum progression, a proper exam engine, +a RootNode Capstone, and persistent achievement records. Builds on +`docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` — read that first if you haven't. + +## 1. What changed, in one paragraph + +Phase 2B's `scenario_attempts` table was one row per validation click, written atomically at +completion, with `hint_count` deliberately left unmeasured. Phase 3B evolves it into a real +attempt lifecycle — one row per continuous working session on a scenario, with live counters +(validation attempts/passes/fails, hint reveals, duration) updated in place as the student +works. On top of that: a rule-driven curriculum-progression engine that actually locks/unlocks +stages instead of a permanent capstone placeholder; a progressive hint system that's genuinely +server-enforced (not just collapsed-by-default in the UI); a full exam/capstone assessment +engine with server-authoritative timers and random scenario draws; and persistent achievement +records for stage/curriculum/assessment/capstone completion. + +## 2. Learning record model + +`scenario_attempts` (migration `012`) gained: `cohort_id`, `curriculum_stage` (both snapshotted +at attempt-start, so a later cohort/curriculum change never rewrites history), `status` +(`in_progress`/`completed`/`abandoned`), `validation_attempt_count`, `validation_pass_count`, +`validation_fail_count`, `highest_hint_level`, `duration_seconds`, `assessment_session_id`. +`result` (Phase 2B) is now nullable — meaningful only once `status = 'completed'`. +`hint_count` (Phase 2B, always `NULL`) finally gets a `DEFAULT 0` and real increments. + +This is an **evolution, not a new table** — Phase 2B's historical rows are backfilled (one +validation performed, matching their original one-row-per-click semantics), not dropped or +reshaped. `platform/api/src/attempts.js` owns the lifecycle: + +- `getOrCreateOpenAttempt` — finds the open (`in_progress`) attempt for + `(student, scenario, environment)`, or starts one. Environment-scoped so a destroyed/recreated + Lab never silently resumes an attempt tied to compute that no longer exists. +- `recordValidation` — increments the attempt/pass/fail counters; closes the attempt + (`status='completed'`, `duration_seconds` computed) only on a pass. +- `recordHintReveal` — sequential only: level *N* can't be revealed before *N-1* has been. + Returns a 409 once every hint is exhausted. + +No telemetry is invented that the Lab doesn't actually emit — `duration_seconds` is measured +server-side from `started_at` to completion (not client-reported), and hint reveals only exist +because Phase 3B builds a real reveal endpoint (§3), not because the old static hint array was +reinterpreted as "measured." + +## 3. Validation telemetry & progressive hints + +`POST /api/scenarios/:id/start` — fire-and-forget from the Lab UI when a scenario becomes +active (both task and MCQ types); creates/reopens the attempt with a real `started_at`. + +`POST /api/progress/record` (Gateway-relayed from the Lab's existing validate/answer calls, +unchanged mechanism from Phase 2A/2B) now routes through `attempts.recordValidation` instead of +a flat insert, and — on a pass — checks curriculum/capstone/assessment consequences (§4, §6). + +`POST /api/scenarios/:id/hint` reveals hints **one at a time, server-enforced**. This required a +real architectural change: Phase 2B's Lab backend (unmodified, per the "Lab image stays +unmodified" convention through Phase 3A) returns the scenario's full `hints` array in +`GET /api/scenarios/:id` — always did, since the Lab has no student/attempt concept at all. A +client could already see hint 3 in the raw response even with the UI collapsed. Making the gate +real meant the Gateway now **strips `hints` from that response** (replacing it with a bare +`hint_count`) for exactly the multi-student, Platform-mediated deployment +(`platform/gateway/src/server.js`, `GET /lab/:token/api/scenarios/:scenarioId`) — the Lab +image's own bytes are still untouched; this is a response rewrite at the boundary, the same +pattern already used for the `hints`-stripping shape as everything else the Gateway mediates. +Standalone/single-container dev mode never goes through the Gateway, so it keeps the original +fully-preloaded array unchanged — the frontend detects which mode it's in +(`scenario.hints === undefined && typeof scenario.hint_count === 'number'`) and switches to +`ProgressiveHints` (`frontend/src/components/ProgressiveHints.jsx`) only in the gated case. + +Instructor-visible attempt summary matches the shape in the Phase 3B spec directly: + +``` +Student: Alice Scenario: Probe Port Mismatch Attempts: 2 +Validation: Failed 2, Passed 1 Hints used: 1/3 Time: 14m 22s Final: Completed +``` + +— all straight off `scenario_attempts` columns, no derived/estimated numbers, and explicitly +**not** turned into a score. See §12 for where instructors see this. + +## 4. Curriculum completion rules + +`scenarios/rootnode-path.json` gained a `version` field and, per stage, a `completion_rule`: + +```json +{ "type": "all_required" } // every scenario_id in the stage must be completed (default) +{ "type": "threshold", "percent": 80 } // at least N% completed +{ "type": "prerequisites", "scenario_ids": [...] } // only these specific ids must be completed +``` + +The capstone stage is no longer a hardcoded `"locked": true` placeholder — it carries +`"type": "capstone"`, an `"unlock_rule"` (`{"type":"stages_complete","stage_ids":[...]}`), and a +`"capstone_assessment_id"` pointing at the assessment definition that drives it (§6/§7). + +`platform/api/src/curriculum-rules.js`'s `computeCurriculumState(path, progress, capstoneStatus)` +is the rule engine — pure, synchronous, no DB access — answering: per-stage locked/complete/ +percent/next-scenario, the student's current stage, overall percent complete, and the capstone's +unlock status (server route supplies `capstoneStatus` after checking for a +`capstone_completed` achievement, since that's Postgres state this module doesn't touch). +Exposed at `GET /api/curriculum/state` (new; `GET /api/curriculum/path` from Phase 2A/2B is +unchanged and still used by the pre-existing dashboard summary). + +Stage/curriculum completion also triggers achievement recording (§8) automatically, from inside +`POST /api/progress/record` — no separate "check for completion" step the student or frontend +has to remember to call. + +## 5 & 6. Assessment model and exam engine + +`PRACTICE` is just... practice: the existing curriculum flow, hints freely available (subject to +the scenario actually having any), unrestricted retries, no timer, no session concept. + +`ASSESSMENT` (exam or capstone — same underlying engine, §7 covers what makes a capstone +different) is a **definition** (the reusable template: pool, duration, hint policy, completion +rule) producing **sessions** (one per student attempt, immutable snapshot of what was drawn and +under what rules). New tables (migration `013`): + +``` +assessment_definitions slug PK, type ('exam'|'capstone'), title, duration_minutes, + hint_policy ('free'|'disabled'|'flagged'), pool_spec JSONB, + completion_rule JSONB, unlock_rule JSONB (capstone only), + version INTEGER, status ('active'|'inactive') +assessment_cohort_assignments which cohorts a definition is available to +assessment_sessions one per student attempt — snapshots type/title/duration_minutes/ + hint_policy from the definition at draw time, plus + definition_version, started_at, server-computed expires_at, + submitted_at, random_seed, result JSONB +assessment_session_scenarios the drawn set, ordered, with per-scenario status + a real FK + back to the scenario_attempts row once worked +``` + +`exam_sessions` (Phase 1/2B, historical-import-only from the old Lab-local SQLite exam +bookkeeping) is **left untouched** — it was already scoped as dead historical data, never the +live mechanism, so there was nothing to migrate into the tables above without inventing data +that was never real. This is a deliberate non-duplication: the assessment tables above *are* +the real thing Phase 2B's note said didn't exist yet. + +`pool_spec` shape: + +```json +{ "required": ["scenario-id", ...], "random": [{ "count": 1, "from": ["a", "b", "c"] }, ...] } +``` + +`platform/api/src/assessments.js`'s `drawScenarios` includes every `required` id, plus `count` +random picks from each `random` group, shuffled together into presentation order. The draw uses +a tiny deterministic PRNG (mulberry32, seeded from `crypto.randomBytes` — no new dependency) +purely so the seed can be recorded for audit; a student never sees the seed or the undrawn pool +before starting. + +**One assessment at a time per student** — `startSession` refuses if the student already has a +`not_started`/`in_progress` session for *any* definition. This keeps "which scenario is this +validate call happening inside" unambiguous for hint-policy enforcement (§19) without needing +the client to declare a mode. + +**Timer**: `expires_at` is computed once, server-side, at `started_at + duration_minutes`, and +never recomputed from anything the client sends. Expiry is checked **lazily** — whenever a +session is read or written, if `now() > expires_at` and it's still `in_progress`, it flips to +`expired` right then. No background cron (explicitly out of scope this phase — see the +non-goals list). Verified live: a 1-minute test definition correctly transitioned to `expired` +on the first request after the minute passed, and unblocked starting a new assessment. + +## 7. RootNode Capstone + +A capstone is an `assessment_definitions` row with `type='capstone'` — same engine as an exam, +different framing and unlock gate. The shipped default (seeded in migration `013`, idempotent +`INSERT ... ON CONFLICT DO NOTHING`) is `rootnode-capstone`: + +- **Required** (always drawn): all four RootNode Real Ops scenarios + (`rootnode-runaway-monitor`, `broken-deployment`, `rootnode-probe-port-mismatch`, + `rootnode-enforce-limits`) — "feature prominently," per the spec, satisfied by making them + non-optional rather than just likely. +- **Random**: one Workloads task, one Configuration task, one Troubleshooting task — task-type + scenarios preferred over MCQ, since the point is operational ability, not recall. +- 90 minutes, `hint_policy: 'flagged'` (available, but every use is recorded — appropriate for + a capstone that should mostly reflect independent work without being punitive about needing a + nudge). +- `unlock_rule`: every non-capstone curriculum stage complete (Foundation through Real Ops). + +`GET /api/capstone` is a student-friendly view combining `computeCurriculumState`'s capstone +block with the definition and the student's own session history — +`locked` → `unlocked` → `completed`, with a prerequisite checklist while locked. Verified live, +end to end: locked with all six blocking stage ids, unlocked to zero blocking ids the moment the +last stage completed, a real draw (7 scenarios, all four Real Ops present), completion of all +seven finalizing the session and recording a `capstone_completed` achievement. + +## 8. Achievement / completion records + +`achievements` (migration `014`): `student_id`, `achievement_type` +(`stage_completed`/`curriculum_completed`/`assessment_completed`/`capstone_completed`), +`curriculum_path_id`/`curriculum_version` (for stage/curriculum types), +`assessment_session_id` (for assessment/capstone types), `achieved_at`, `metadata`. + +`recordAchievement` is idempotent via a `WHERE NOT EXISTS` guard (not a unique-index/ +`ON CONFLICT` dance — simple and correct at this write volume) — calling it again for something +already recorded is a safe no-op. Triggered automatically: `checkStageAndCurriculumAchievements` +runs inside `POST /api/progress/record` after every pass; `recordAssessmentAchievement` runs +after `assessments.recordScenarioOutcome`/`submitSession` finalizes a session, and only on an +actual pass (defensive — today's only `completion_rule` is `all_scenarios_passed`, which already +guarantees a `completed` session passed, but a future threshold-based rule shouldn't silently +grant achievements for a partial result). + +**Not a certificate.** No public/shareable artifact is generated — this is explicitly a Phase +4/5 concern (see the non-goals list); a certificate can be rendered from these records later +without needing new data. + +## 9. Assessment/exam Lab behavior + +An assessment doesn't provision a separate Lab — the student works through their drawn +scenarios inside the **same** disposable environment their curriculum work already uses. This +is what makes "Lab reset doesn't erase assessment state" nearly free: assessment state +(`assessment_sessions`, `assessment_session_scenarios`, `scenario_attempts`) lives entirely in +Platform Postgres; the Lab environment is purely compute. Resetting/recreating it changes the +container, never the rows. Verified live: reset an environment mid-assessment, +`expires_at`/`status` on the session were unchanged afterward. + +**Assessment-mode detection is entirely server-derived** — `findActiveAssessmentContextForScenario` +looks up whether the scenario a `validate`/`hint` call is happening against is currently in the +student's own active session's drawn set. Nothing the client sends declares "I'm in assessment +mode"; the server's own `assessment_session_scenarios` table is the only source of truth (see +§19). + +If Lab infrastructure fails mid-assessment, the assessment session itself is unaffected — an +instructor can reset (or, if truly needed, destroy/reprovision) the student's Lab via the +existing admin environment controls (Phase 2B) without touching assessment state at all; there +is no code path that resets an assessment as a side effect of a Lab operation. + +## 10 & 11. Student UI and Lab UX + +`frontend/src/platform/StudentDashboard.jsx` is now tabbed: **My Learning** (the original +Phase 2A/2B "Up Next" + Launch/Reset Lab flow, extracted unchanged into `MyLearningTab.jsx` and +extended with an overall percent-complete bar from the new rule-driven curriculum state), +**Practice** (scenario/stage status + recent attempt history), **Assessments** (available/ +active-with-live-countdown/history), **Capstone** (locked checklist → unlocked → start/resume → +completed, with attempt history), **Achievements** (a flat, dated list — not a leaderboard). + +Inside the Lab UI itself (`frontend/src/components/ScenarioPanel.jsx`, `App.jsx`): +`ProgressiveHints` replaces the static hint array render when the Gateway has stripped hints +(§3); an `AssessmentBanner` (new, top-level, polls `/api/assessments/active`) shows a live +countdown and per-scenario completion count whenever an assessment is running — distinct from, +and rendered alongside, the pre-existing Lab-local exam-mode UI (`ExamTimer`/`ExamSession` etc., +upstream KubeKosh feature, untouched). Both render nothing in standalone/single-container dev +mode (no Platform API to poll there), so neither regresses that deployment path. + +## 12. Instructor learning analytics + +`/admin` → **Learning**: cohort summary (student count, 7-day active-learner count, average +completion, per-stage completion counts), a **Needs Attention** list, and a per-student +drill-down (curriculum state, full attempt history with validation/hint counts, assessment +history). `/admin` → **Assessments**: create/edit definitions (admin/owner), assign to a cohort, +view/cancel sessions — visible to every instructor role, cohort-scoped server-side for the plain +`instructor` role the same way Phase 3A's other cohort-scoped routes are. + +`GET /api/admin/scenario-analytics` (optional `?cohort_id=`): completions, distinct students, +average validation attempts, total failures, total hints used, average completion time — one +`GROUP BY scenario_id` query over `scenario_attempts`, nothing pre-aggregated or cached. + +## 13. "Needs Attention" + +`GET /api/admin/cohorts/:id/needs-attention` computes four signals per student, on read (no +stored table, no background job): **Repeated Attempts** (an open attempt with ≥3 failed +validations), **May Need Help** (an open attempt where a hint has been used but the scenario +isn't done), **Needs Attention — inactive** (no activity in 7 days), and **Needs Attention — +incomplete assessment** (their most recent assessment session expired without completing). All +four thresholds are named constants in `admin-learning-routes.js` +(`NEEDS_ATTENTION_FAIL_THRESHOLD`, `NEEDS_ATTENTION_STALE_DAYS`), not hardcoded inline. Wording +is deliberately instructional (`Needs Attention`, `May Need Help`, `Repeated Attempts`) — an +explicit test asserts none of the returned labels match punitive language. + +## 14 & 15. Assessment/capstone management (admin) + +A structured form (`frontend/src/admin/AssessmentsView.jsx`) — id, title, type, duration, hint +policy, and a raw JSON `pool_spec` textarea. Not a drag-and-drop authoring tool, per the explicit +scope line; the JSON textarea is honest about what's actually being edited rather than hiding it +behind partial UI that can't express every pool shape. Editing a definition's +duration/hint_policy/pool_spec/completion_rule bumps its `version` automatically +(`admin-learning-routes.js` diffs the incoming fields against the current row) — cosmetic edits +(title, description) don't. Create/edit is admin/owner only server-side; assign-to-cohort and +session view/cancel are available to a scoped `instructor` too, matching the RBAC matrix (§18). + +## 16. Versioning + +Deliberately minimal, per the explicit "don't overbuild full CMS versioning" instruction: + +- **Curriculum path**: a plain `"version": 1` integer in `rootnode-path.json` itself, recorded + on `stage_completed`/`curriculum_completed` achievements as a snapshot. +- **Assessment definitions**: a plain incrementing `version` integer, bumped only on a semantic + change (duration/hint_policy/pool_spec/completion_rule — anything that would change what a + session is graded against). `assessment_sessions.definition_version` snapshots which version + was live at draw time, alongside a full redundant snapshot of the fields themselves + (title/duration/hint_policy) — so a later definition edit, or even deletion, can never change + what an already-started session is measured against. + +No full version-history table exists for either — if that's ever needed, `version` is already a +column to build on, not a retrofit. + +## 17. RBAC matrix (Phase 3B additions) + +| Action | owner | admin | instructor | student | +|---|---|---|---|---| +| Create/edit assessment definitions | ✅ | ✅ | ❌ | ❌ | +| Assign an assessment to a cohort | ✅ | ✅ | ✅ (own cohorts) | ❌ | +| View/cancel assessment sessions | ✅ (any) | ✅ (any) | ✅ (own cohorts) | ❌ | +| View cohort learning summary / needs-attention / scenario analytics | ✅ (global) | ✅ (global) | ✅ (own cohorts) | ❌ | +| View a student's learning detail | ✅ | ✅ | ✅ (if enrolled in an owned cohort) | own only | +| Start/work an assessment, view own attempts/hints/achievements | ❌ | ❌ | ❌ | ✅ (own only) | + +Enforced entirely server-side, reusing Phase 3A's `rbac.js` (`requireAdminOrOwner`, +`requireAnyInstructor`, `requireCohortAccess`, `requireStudentCohortAccess`) — no new +authorization primitives were needed, only new routes wired through the existing ones. + +## 18. Security / integrity + +- **Timer is server-authoritative** (§6) — `expires_at` fixed at draw time, never accepts a + client value, checked lazily on every touch. +- **Scenario draw is server-stored** — `assessment_session_scenarios` is the only source of + truth for what a student is supposed to be working on; nothing the client sends can add, + remove, or reorder it. +- **A student cannot submit/read another student's session** — every assessment route filters + by `student_id = req.studentId` server-side; a foreign session id returns 404, not 403 (same + "don't confirm existence" posture as Phase 3A's RBAC denials) — verified by an explicit test. +- **Gateway ownership** (Phase 2A/2B/3A) is unchanged and still enforced on every proxied Lab + request, HTTP and WebSocket, including the new `/start` and `/hint` relays. +- **Lab recreation never resets attempt/exam state** (§9) — verified live. +- **Hidden solution/pool data**: undrawn pool alternatives are never returned to the student + (only the actual draw); hints beyond the currently-unlocked level are never sent (§3, the + actual mechanism, not just a UI convention). +- **No invasive anti-cheat** — no browser surveillance, no lockdown mode. Explicitly out of + scope, per the non-goals list. + +## 19. Dev/test commands + +```bash +cd platform && docker compose -f docker-compose.dev.yml up -d +(cd api && PLATFORM_DB_URL=postgres://postgres:postgres@localhost:5433/rootnode_platform \ + node bootstrap-owner.js you@example.com "Your Name" "a-real-password-12-chars-plus") + +OWNER_EMAIL=you@example.com OWNER_PASSWORD=... node test/e2e-phase2a.js +OWNER_EMAIL=you@example.com OWNER_PASSWORD=... node test/e2e-phase2b.js +OWNER_EMAIL=you@example.com OWNER_PASSWORD=... node test/e2e-phase3a.js +OWNER_EMAIL=you@example.com OWNER_PASSWORD=... node test/e2e-phase3b.js +``` + +`e2e-phase3b.js` additionally shells out to `docker compose restart api` for its restart- +persistence case (set `SKIP_RESTART_TEST=1` to skip that one case if running somewhere that +isn't safe to restart). + +## 20. Known limitations + +- **No automatic "abandoned" detection** — an attempt a student never returns to stays + `in_progress` forever; there's no reaper tying Lab destruction back to attempt state. A + documented, deliberate scope line (would need wiring to Provisioner `environment_events`, + more machinery than the instructional-visibility goal needs). +- **`cohort_instructors.role_in_cohort`** (Phase 3A) still isn't consulted for finer-grained + assessment permissions — any cohort assignment grants the full scoped-instructor action set. +- **Needs-attention thresholds are fixed constants**, not per-cohort configurable yet. +- **No pagination** on the audit log (Phase 3A limitation, unchanged) or on scenario analytics. +- **Assessment pool_spec authoring is raw JSON** in the admin UI — correct and complete, but not + a guided form; a validation-assist UI is a natural but unbuilt next step. +- **One assessment at a time per student** — intentional (§6), but means a student can't have a + capstone and an exam "in flight" simultaneously even if that were ever desired. + +## 21. Phase 4 boundary + +Everything in this document runs on a single dev machine against +`platform/docker-compose.dev.yml`. **Not built, and explicitly out of scope this phase:** +production Hetzner/Proxmox deployment, production OIDC provider integration, a parent portal, +payments, public signup, automated email, equipment inventory, a multi-host scheduler, +Grafana/Prometheus production dashboards, public certificates, a marketplace/self-paced commerce +layer, the six held KubeKosh scenarios, invasive proctoring, and AI-driven grading. All of that +stays exactly where the Phase 3A doc's non-goals list and this document's §21 leave it — Phase 4 +territory. diff --git a/platform/README.md b/platform/README.md index 4936f9b..a271882 100644 --- a/platform/README.md +++ b/platform/README.md @@ -1,21 +1,27 @@ # RootNode Academy — Platform -The full multi-student platform, layered on top of the unmodified Lab image at the repo root. -Start here, then go to the component you need: +The full multi-student platform, layered on top of the Lab image at the repo root (backend/ +entrypoint/Dockerfile unmodified; frontend bundle gained Phase 3B learning UX, see +`docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` §10–11). Start here, then go to the component you need: - `provisioner/` — Postgres-backed environment registry and lifecycle. Design: `docs/LAB_ARCHITECTURE.md`, `docs/PHASE1_PLATFORM_FOUNDATION.md`. Operational reference: `provisioner/README.md`. -- `api/` — the lightweight Platform API: students, cohorts, enrollments, curriculum, progress, - scenario attempts, per-instructor accounts, persistent student/instructor sessions, RBAC, - audit logging, and the instructor/admin control plane. No k3s, no `--privileged` — this - replaced Phase 2A's "identity" instance, which was a full Lab image booting an unused k3s - cluster just to serve these APIs. Design: `docs/PHASE2B_ADMIN_AND_PLATFORM_DATA.md` (split + - Postgres migration), `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` (sessions, accounts, - RBAC, audit). Operational reference: `api/README.md`. +- `api/` — the lightweight Platform API: students, cohorts, enrollments, curriculum, detailed + scenario attempt telemetry, progressive hints, curriculum completion rules, the exam + + capstone assessment engine, achievements, per-instructor accounts, persistent student/ + instructor sessions, RBAC, audit logging, and the instructor/admin control plane. No k3s, no + `--privileged` — this replaced Phase 2A's "identity" instance, which was a full Lab image + booting an unused k3s cluster just to serve these APIs. Design: + `docs/PHASE2B_ADMIN_AND_PLATFORM_DATA.md` (split + Postgres migration), + `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` (sessions, accounts, RBAC, audit), + `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` (learning/assessment engine). Operational + reference: `api/README.md`. - `gateway/` — the only student-facing path to a Lab (HTTP + WebSocket proxy, environment-aware - routing derived from a server-controlled session — not a client-suppliable header/param). - Design: `docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md` (routing), `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` (persistent sessions, CSRF). + routing derived from a server-controlled session — not a client-suppliable header/param — and, + as of Phase 3B, server-side gating of progressive hint reveal). Design: + `docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md` (routing), `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` + (persistent sessions, CSRF), `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` §3 (hint gating). - `egress-proxy/` — shared allowlisting proxy every Lab's outbound traffic passes through. - `docker-compose.dev.yml` — the full stack (Postgres, egress proxy, Provisioner, Platform API, Gateway) on one machine. This is the Phase 1/2A/2B **local dev exception** — production is @@ -54,16 +60,21 @@ further instructor accounts from the Instructors tab). - `test/e2e-phase3a.js` — persistent sessions (including a real Platform API restart), instructor auth (no enumeration, rate limiting), RBAC across owner/admin/instructor, cohort-scoped instructor access, CSRF enforcement, password change, and audit logging. +- `test/e2e-phase3b.js` — attempt telemetry, progressive hint sequencing/persistence through a + Lab reset, curriculum stage/Capstone lock-unlock, the exam engine (random draw, server + timer, expiry, restart persistence), the Capstone workflow end to end, two-student assessment + isolation, and instructor learning analytics/RBAC. -All three e2e suites need `OWNER_EMAIL` / `OWNER_PASSWORD` set to a real bootstrapped instructor +All four e2e suites need `OWNER_EMAIL` / `OWNER_PASSWORD` set to a real bootstrapped instructor account (see "Running the full stack locally" above) — the old `ADMIN_PASSWORD` mechanism is -gone. They're slow (real k3s boots) and need the full stack up. None of them substitute for the -manual real-Lab-image checks (node-pty, terminal WebSocket, `kubectl`, scenario validation) in -`docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md` §8 — those need a browser-equivalent client and -aren't automated here. +gone. They're slow (real k3s boots, `e2e-phase3b.js` also runs a real API restart) and need the +full stack up. None of them substitute for the manual real-Lab-image checks (node-pty, terminal +WebSocket, `kubectl`, scenario validation) in `docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md` §8 — +those need a browser-equivalent client and aren't automated here. ## Known limitations See `provisioner/README.md` (Phase 1), `docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md` §9 (Phase -2A), and `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` (Phase 3A, supersedes Phase 2B's -identity limitations) for the full, honest lists. +2A), `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` (Phase 3A identity limitations), and +`docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` §20 (Phase 3B learning/assessment limitations) for the +full, honest lists. diff --git a/platform/api/README.md b/platform/api/README.md index 8a210dd..b9912d9 100644 --- a/platform/api/README.md +++ b/platform/api/README.md @@ -1,9 +1,11 @@ -# RootNode Academy — Platform API (Phase 3A) +# RootNode Academy — Platform API (Phase 3B) Lightweight service: no k3s, no `--privileged`, no Lab terminal runtime. See -`docs/PHASE2B_ADMIN_AND_PLATFORM_DATA.md` for the original split/migration design and +`docs/PHASE2B_ADMIN_AND_PLATFORM_DATA.md` for the original split/migration design, `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` for persistent sessions, per-instructor -accounts, RBAC, and audit logging — this file is the operational reference. +accounts, RBAC, and audit logging, and `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` for attempt +telemetry, progressive hints, curriculum rules, and the exam/capstone engine — this file is the +operational reference. ## Configuration reference @@ -71,14 +73,18 @@ with instructor accounts, which are Phase 3A-native and start empty until you bo ## Tests `platform/test/e2e-phase2a.js` (student-flow regression), `platform/test/e2e-phase2b.js` (the -classroom acceptance workflow), and `platform/test/e2e-phase3a.js` (persistent sessions -including a real restart, instructor auth, RBAC, cohort scoping, CSRF, audit log). All three -need the full stack up and `OWNER_EMAIL` / `OWNER_PASSWORD` set to a bootstrapped account — see -`platform/README.md`. +classroom acceptance workflow), `platform/test/e2e-phase3a.js` (persistent sessions including a +real restart, instructor auth, RBAC, cohort scoping, CSRF, audit log), and +`platform/test/e2e-phase3b.js` (attempt telemetry, progressive hints, curriculum/Capstone +unlock rules, the exam engine including a real restart, two-student assessment isolation, +instructor analytics). All four need the full stack up and `OWNER_EMAIL` / `OWNER_PASSWORD` set +to a bootstrapped account — see `platform/README.md`. ## Known limitations -See `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` "Known limitations" — sessions are -Postgres-backed (survive a restart) but there's no distributed revocation channel to the -Gateway's short-lived whoami cache, exam mode's live bookkeeping stays Lab-local, and OIDC/SSO -is a documented adapter interface, not a working integration yet. +See `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` "Known limitations" (sessions are +Postgres-backed and survive a restart, but there's no distributed revocation channel to the +Gateway's short-lived whoami cache; OIDC/SSO is a documented adapter interface, not a working +integration yet) and `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` §20 (no automatic +abandoned-attempt detection, needs-attention thresholds are fixed constants, `pool_spec` +authoring is raw JSON in the admin UI).