# Phase 3B — Learning & Assessment Engine Turns the platform infrastructure built in Phases 1–3A (identity, sessions, RBAC, disposable Labs, Gateway routing, admin portal) into a real teaching and assessment system: detailed attempt telemetry, progressive hints, rule-driven curriculum progression, a proper exam engine, a RootNode Capstone, and persistent achievement records. Builds on `docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` — read that first if you haven't. ## 1. What changed, in one paragraph Phase 2B's `scenario_attempts` table was one row per validation click, written atomically at completion, with `hint_count` deliberately left unmeasured. Phase 3B evolves it into a real attempt lifecycle — one row per continuous working session on a scenario, with live counters (validation attempts/passes/fails, hint reveals, duration) updated in place as the student works. On top of that: a rule-driven curriculum-progression engine that actually locks/unlocks stages instead of a permanent capstone placeholder; a progressive hint system that's genuinely server-enforced (not just collapsed-by-default in the UI); a full exam/capstone assessment engine with server-authoritative timers and random scenario draws; and persistent achievement records for stage/curriculum/assessment/capstone completion. ## 2. Learning record model `scenario_attempts` (migration `012`) gained: `cohort_id`, `curriculum_stage` (both snapshotted at attempt-start, so a later cohort/curriculum change never rewrites history), `status` (`in_progress`/`completed`/`abandoned`), `validation_attempt_count`, `validation_pass_count`, `validation_fail_count`, `highest_hint_level`, `duration_seconds`, `assessment_session_id`. `result` (Phase 2B) is now nullable — meaningful only once `status = 'completed'`. `hint_count` (Phase 2B, always `NULL`) finally gets a `DEFAULT 0` and real increments. This is an **evolution, not a new table** — Phase 2B's historical rows are backfilled (one validation performed, matching their original one-row-per-click semantics), not dropped or reshaped. `platform/api/src/attempts.js` owns the lifecycle: - `getOrCreateOpenAttempt` — finds the open (`in_progress`) attempt for `(student, scenario, environment)`, or starts one. Environment-scoped so a destroyed/recreated Lab never silently resumes an attempt tied to compute that no longer exists. - `recordValidation` — increments the attempt/pass/fail counters; closes the attempt (`status='completed'`, `duration_seconds` computed) only on a pass. - `recordHintReveal` — sequential only: level *N* can't be revealed before *N-1* has been. Returns a 409 once every hint is exhausted. No telemetry is invented that the Lab doesn't actually emit — `duration_seconds` is measured server-side from `started_at` to completion (not client-reported), and hint reveals only exist because Phase 3B builds a real reveal endpoint (§3), not because the old static hint array was reinterpreted as "measured." ## 3. Validation telemetry & progressive hints `POST /api/scenarios/:id/start` — fire-and-forget from the Lab UI when a scenario becomes active (both task and MCQ types); creates/reopens the attempt with a real `started_at`. `POST /api/progress/record` (Gateway-relayed from the Lab's existing validate/answer calls, unchanged mechanism from Phase 2A/2B) now routes through `attempts.recordValidation` instead of a flat insert, and — on a pass — checks curriculum/capstone/assessment consequences (§4, §6). `POST /api/scenarios/:id/hint` reveals hints **one at a time, server-enforced**. This required a real architectural change: Phase 2B's Lab backend (unmodified, per the "Lab image stays unmodified" convention through Phase 3A) returns the scenario's full `hints` array in `GET /api/scenarios/:id` — always did, since the Lab has no student/attempt concept at all. A client could already see hint 3 in the raw response even with the UI collapsed. Making the gate real meant the Gateway now **strips `hints` from that response** (replacing it with a bare `hint_count`) for exactly the multi-student, Platform-mediated deployment (`platform/gateway/src/server.js`, `GET /lab/:token/api/scenarios/:scenarioId`) — the Lab image's own bytes are still untouched; this is a response rewrite at the boundary, the same pattern already used for the `hints`-stripping shape as everything else the Gateway mediates. Standalone/single-container dev mode never goes through the Gateway, so it keeps the original fully-preloaded array unchanged — the frontend detects which mode it's in (`scenario.hints === undefined && typeof scenario.hint_count === 'number'`) and switches to `ProgressiveHints` (`frontend/src/components/ProgressiveHints.jsx`) only in the gated case. Instructor-visible attempt summary matches the shape in the Phase 3B spec directly: ``` Student: Alice Scenario: Probe Port Mismatch Attempts: 2 Validation: Failed 2, Passed 1 Hints used: 1/3 Time: 14m 22s Final: Completed ``` — all straight off `scenario_attempts` columns, no derived/estimated numbers, and explicitly **not** turned into a score. See §12 for where instructors see this. ## 4. Curriculum completion rules `scenarios/rootnode-path.json` gained a `version` field and, per stage, a `completion_rule`: ```json { "type": "all_required" } // every scenario_id in the stage must be completed (default) { "type": "threshold", "percent": 80 } // at least N% completed { "type": "prerequisites", "scenario_ids": [...] } // only these specific ids must be completed ``` The capstone stage is no longer a hardcoded `"locked": true` placeholder — it carries `"type": "capstone"`, an `"unlock_rule"` (`{"type":"stages_complete","stage_ids":[...]}`), and a `"capstone_assessment_id"` pointing at the assessment definition that drives it (§6/§7). `platform/api/src/curriculum-rules.js`'s `computeCurriculumState(path, progress, capstoneStatus)` is the rule engine — pure, synchronous, no DB access — answering: per-stage locked/complete/ percent/next-scenario, the student's current stage, overall percent complete, and the capstone's unlock status (server route supplies `capstoneStatus` after checking for a `capstone_completed` achievement, since that's Postgres state this module doesn't touch). Exposed at `GET /api/curriculum/state` (new; `GET /api/curriculum/path` from Phase 2A/2B is unchanged and still used by the pre-existing dashboard summary). Stage/curriculum completion also triggers achievement recording (§8) automatically, from inside `POST /api/progress/record` — no separate "check for completion" step the student or frontend has to remember to call. ## 5 & 6. Assessment model and exam engine `PRACTICE` is just... practice: the existing curriculum flow, hints freely available (subject to the scenario actually having any), unrestricted retries, no timer, no session concept. `ASSESSMENT` (exam or capstone — same underlying engine, §7 covers what makes a capstone different) is a **definition** (the reusable template: pool, duration, hint policy, completion rule) producing **sessions** (one per student attempt, immutable snapshot of what was drawn and under what rules). New tables (migration `013`): ``` assessment_definitions slug PK, type ('exam'|'capstone'), title, duration_minutes, hint_policy ('free'|'disabled'|'flagged'), pool_spec JSONB, completion_rule JSONB, unlock_rule JSONB (capstone only), version INTEGER, status ('active'|'inactive') assessment_cohort_assignments which cohorts a definition is available to assessment_sessions one per student attempt — snapshots type/title/duration_minutes/ hint_policy from the definition at draw time, plus definition_version, started_at, server-computed expires_at, submitted_at, random_seed, result JSONB assessment_session_scenarios the drawn set, ordered, with per-scenario status + a real FK back to the scenario_attempts row once worked ``` `exam_sessions` (Phase 1/2B, historical-import-only from the old Lab-local SQLite exam bookkeeping) is **left untouched** — it was already scoped as dead historical data, never the live mechanism, so there was nothing to migrate into the tables above without inventing data that was never real. This is a deliberate non-duplication: the assessment tables above *are* the real thing Phase 2B's note said didn't exist yet. `pool_spec` shape: ```json { "required": ["scenario-id", ...], "random": [{ "count": 1, "from": ["a", "b", "c"] }, ...] } ``` `platform/api/src/assessments.js`'s `drawScenarios` includes every `required` id, plus `count` random picks from each `random` group, shuffled together into presentation order. The draw uses a tiny deterministic PRNG (mulberry32, seeded from `crypto.randomBytes` — no new dependency) purely so the seed can be recorded for audit; a student never sees the seed or the undrawn pool before starting. **One assessment at a time per student** — `startSession` refuses if the student already has a `not_started`/`in_progress` session for *any* definition. This keeps "which scenario is this validate call happening inside" unambiguous for hint-policy enforcement (§19) without needing the client to declare a mode. **Timer**: `expires_at` is computed once, server-side, at `started_at + duration_minutes`, and never recomputed from anything the client sends. Expiry is checked **lazily** — whenever a session is read or written, if `now() > expires_at` and it's still `in_progress`, it flips to `expired` right then. No background cron (explicitly out of scope this phase — see the non-goals list). Verified live: a 1-minute test definition correctly transitioned to `expired` on the first request after the minute passed, and unblocked starting a new assessment. ## 7. RootNode Capstone A capstone is an `assessment_definitions` row with `type='capstone'` — same engine as an exam, different framing and unlock gate. The shipped default (seeded in migration `013`, idempotent `INSERT ... ON CONFLICT DO NOTHING`) is `rootnode-capstone`: - **Required** (always drawn): all four RootNode Real Ops scenarios (`rootnode-runaway-monitor`, `broken-deployment`, `rootnode-probe-port-mismatch`, `rootnode-enforce-limits`) — "feature prominently," per the spec, satisfied by making them non-optional rather than just likely. - **Random**: one Workloads task, one Configuration task, one Troubleshooting task — task-type scenarios preferred over MCQ, since the point is operational ability, not recall. - 90 minutes, `hint_policy: 'flagged'` (available, but every use is recorded — appropriate for a capstone that should mostly reflect independent work without being punitive about needing a nudge). - `unlock_rule`: every non-capstone curriculum stage complete (Foundation through Real Ops). `GET /api/capstone` is a student-friendly view combining `computeCurriculumState`'s capstone block with the definition and the student's own session history — `locked` → `unlocked` → `completed`, with a prerequisite checklist while locked. Verified live, end to end: locked with all six blocking stage ids, unlocked to zero blocking ids the moment the last stage completed, a real draw (7 scenarios, all four Real Ops present), completion of all seven finalizing the session and recording a `capstone_completed` achievement. ## 8. Achievement / completion records `achievements` (migration `014`): `student_id`, `achievement_type` (`stage_completed`/`curriculum_completed`/`assessment_completed`/`capstone_completed`), `curriculum_path_id`/`curriculum_version` (for stage/curriculum types), `assessment_session_id` (for assessment/capstone types), `achieved_at`, `metadata`. `recordAchievement` is idempotent via a `WHERE NOT EXISTS` guard (not a unique-index/ `ON CONFLICT` dance — simple and correct at this write volume) — calling it again for something already recorded is a safe no-op. Triggered automatically: `checkStageAndCurriculumAchievements` runs inside `POST /api/progress/record` after every pass; `recordAssessmentAchievement` runs after `assessments.recordScenarioOutcome`/`submitSession` finalizes a session, and only on an actual pass (defensive — today's only `completion_rule` is `all_scenarios_passed`, which already guarantees a `completed` session passed, but a future threshold-based rule shouldn't silently grant achievements for a partial result). **Not a certificate.** No public/shareable artifact is generated — this is explicitly a Phase 4/5 concern (see the non-goals list); a certificate can be rendered from these records later without needing new data. ## 9. Assessment/exam Lab behavior An assessment doesn't provision a separate Lab — the student works through their drawn scenarios inside the **same** disposable environment their curriculum work already uses. This is what makes "Lab reset doesn't erase assessment state" nearly free: assessment state (`assessment_sessions`, `assessment_session_scenarios`, `scenario_attempts`) lives entirely in Platform Postgres; the Lab environment is purely compute. Resetting/recreating it changes the container, never the rows. Verified live: reset an environment mid-assessment, `expires_at`/`status` on the session were unchanged afterward. **Assessment-mode detection is entirely server-derived** — `findActiveAssessmentContextForScenario` looks up whether the scenario a `validate`/`hint` call is happening against is currently in the student's own active session's drawn set. Nothing the client sends declares "I'm in assessment mode"; the server's own `assessment_session_scenarios` table is the only source of truth (see §19). If Lab infrastructure fails mid-assessment, the assessment session itself is unaffected — an instructor can reset (or, if truly needed, destroy/reprovision) the student's Lab via the existing admin environment controls (Phase 2B) without touching assessment state at all; there is no code path that resets an assessment as a side effect of a Lab operation. ## 10 & 11. Student UI and Lab UX `frontend/src/platform/StudentDashboard.jsx` is now tabbed: **My Learning** (the original Phase 2A/2B "Up Next" + Launch/Reset Lab flow, extracted unchanged into `MyLearningTab.jsx` and extended with an overall percent-complete bar from the new rule-driven curriculum state), **Practice** (scenario/stage status + recent attempt history), **Assessments** (available/ active-with-live-countdown/history), **Capstone** (locked checklist → unlocked → start/resume → completed, with attempt history), **Achievements** (a flat, dated list — not a leaderboard). Inside the Lab UI itself (`frontend/src/components/ScenarioPanel.jsx`, `App.jsx`): `ProgressiveHints` replaces the static hint array render when the Gateway has stripped hints (§3); an `AssessmentBanner` (new, top-level, polls `/api/assessments/active`) shows a live countdown and per-scenario completion count whenever an assessment is running — distinct from, and rendered alongside, the pre-existing Lab-local exam-mode UI (`ExamTimer`/`ExamSession` etc., upstream KubeKosh feature, untouched). Both render nothing in standalone/single-container dev mode (no Platform API to poll there), so neither regresses that deployment path. ## 12. Instructor learning analytics `/admin` → **Learning**: cohort summary (student count, 7-day active-learner count, average completion, per-stage completion counts), a **Needs Attention** list, and a per-student drill-down (curriculum state, full attempt history with validation/hint counts, assessment history). `/admin` → **Assessments**: create/edit definitions (admin/owner), assign to a cohort, view/cancel sessions — visible to every instructor role, cohort-scoped server-side for the plain `instructor` role the same way Phase 3A's other cohort-scoped routes are. `GET /api/admin/scenario-analytics` (optional `?cohort_id=`): completions, distinct students, average validation attempts, total failures, total hints used, average completion time — one `GROUP BY scenario_id` query over `scenario_attempts`, nothing pre-aggregated or cached. ## 13. "Needs Attention" `GET /api/admin/cohorts/:id/needs-attention` computes four signals per student, on read (no stored table, no background job): **Repeated Attempts** (an open attempt with ≥3 failed validations), **May Need Help** (an open attempt where a hint has been used but the scenario isn't done), **Needs Attention — inactive** (no activity in 7 days), and **Needs Attention — incomplete assessment** (their most recent assessment session expired without completing). All four thresholds are named constants in `admin-learning-routes.js` (`NEEDS_ATTENTION_FAIL_THRESHOLD`, `NEEDS_ATTENTION_STALE_DAYS`), not hardcoded inline. Wording is deliberately instructional (`Needs Attention`, `May Need Help`, `Repeated Attempts`) — an explicit test asserts none of the returned labels match punitive language. ## 14 & 15. Assessment/capstone management (admin) A structured form (`frontend/src/admin/AssessmentsView.jsx`) — id, title, type, duration, hint policy, and a raw JSON `pool_spec` textarea. Not a drag-and-drop authoring tool, per the explicit scope line; the JSON textarea is honest about what's actually being edited rather than hiding it behind partial UI that can't express every pool shape. Editing a definition's duration/hint_policy/pool_spec/completion_rule bumps its `version` automatically (`admin-learning-routes.js` diffs the incoming fields against the current row) — cosmetic edits (title, description) don't. Create/edit is admin/owner only server-side; assign-to-cohort and session view/cancel are available to a scoped `instructor` too, matching the RBAC matrix (§18). ## 16. Versioning Deliberately minimal, per the explicit "don't overbuild full CMS versioning" instruction: - **Curriculum path**: a plain `"version": 1` integer in `rootnode-path.json` itself, recorded on `stage_completed`/`curriculum_completed` achievements as a snapshot. - **Assessment definitions**: a plain incrementing `version` integer, bumped only on a semantic change (duration/hint_policy/pool_spec/completion_rule — anything that would change what a session is graded against). `assessment_sessions.definition_version` snapshots which version was live at draw time, alongside a full redundant snapshot of the fields themselves (title/duration/hint_policy) — so a later definition edit, or even deletion, can never change what an already-started session is measured against. No full version-history table exists for either — if that's ever needed, `version` is already a column to build on, not a retrofit. ## 17. RBAC matrix (Phase 3B additions) | Action | owner | admin | instructor | student | |---|---|---|---|---| | Create/edit assessment definitions | ✅ | ✅ | ❌ | ❌ | | Assign an assessment to a cohort | ✅ | ✅ | ✅ (own cohorts) | ❌ | | View/cancel assessment sessions | ✅ (any) | ✅ (any) | ✅ (own cohorts) | ❌ | | View cohort learning summary / needs-attention / scenario analytics | ✅ (global) | ✅ (global) | ✅ (own cohorts) | ❌ | | View a student's learning detail | ✅ | ✅ | ✅ (if enrolled in an owned cohort) | own only | | Start/work an assessment, view own attempts/hints/achievements | ❌ | ❌ | ❌ | ✅ (own only) | Enforced entirely server-side, reusing Phase 3A's `rbac.js` (`requireAdminOrOwner`, `requireAnyInstructor`, `requireCohortAccess`, `requireStudentCohortAccess`) — no new authorization primitives were needed, only new routes wired through the existing ones. ## 18. Security / integrity - **Timer is server-authoritative** (§6) — `expires_at` fixed at draw time, never accepts a client value, checked lazily on every touch. - **Scenario draw is server-stored** — `assessment_session_scenarios` is the only source of truth for what a student is supposed to be working on; nothing the client sends can add, remove, or reorder it. - **A student cannot submit/read another student's session** — every assessment route filters by `student_id = req.studentId` server-side; a foreign session id returns 404, not 403 (same "don't confirm existence" posture as Phase 3A's RBAC denials) — verified by an explicit test. - **Gateway ownership** (Phase 2A/2B/3A) is unchanged and still enforced on every proxied Lab request, HTTP and WebSocket, including the new `/start` and `/hint` relays. - **Lab recreation never resets attempt/exam state** (§9) — verified live. - **Hidden solution/pool data**: undrawn pool alternatives are never returned to the student (only the actual draw); hints beyond the currently-unlocked level are never sent (§3, the actual mechanism, not just a UI convention). - **No invasive anti-cheat** — no browser surveillance, no lockdown mode. Explicitly out of scope, per the non-goals list. ## 19. Dev/test commands ```bash cd platform && docker compose -f docker-compose.dev.yml up -d (cd api && PLATFORM_DB_URL=postgres://postgres:postgres@localhost:5433/rootnode_platform \ node bootstrap-owner.js you@example.com "Your Name" "a-real-password-12-chars-plus") OWNER_EMAIL=you@example.com OWNER_PASSWORD=... node test/e2e-phase2a.js OWNER_EMAIL=you@example.com OWNER_PASSWORD=... node test/e2e-phase2b.js OWNER_EMAIL=you@example.com OWNER_PASSWORD=... node test/e2e-phase3a.js OWNER_EMAIL=you@example.com OWNER_PASSWORD=... node test/e2e-phase3b.js ``` `e2e-phase3b.js` additionally shells out to `docker compose restart api` for its restart- persistence case (set `SKIP_RESTART_TEST=1` to skip that one case if running somewhere that isn't safe to restart). ## 20. Known limitations - **No automatic "abandoned" detection** — an attempt a student never returns to stays `in_progress` forever; there's no reaper tying Lab destruction back to attempt state. A documented, deliberate scope line (would need wiring to Provisioner `environment_events`, more machinery than the instructional-visibility goal needs). - **`cohort_instructors.role_in_cohort`** (Phase 3A) still isn't consulted for finer-grained assessment permissions — any cohort assignment grants the full scoped-instructor action set. - **Needs-attention thresholds are fixed constants**, not per-cohort configurable yet. - **No pagination** on the audit log (Phase 3A limitation, unchanged) or on scenario analytics. - **Assessment pool_spec authoring is raw JSON** in the admin UI — correct and complete, but not a guided form; a validation-assist UI is a natural but unbuilt next step. - **One assessment at a time per student** — intentional (§6), but means a student can't have a capstone and an exam "in flight" simultaneously even if that were ever desired. ## 21. Phase 4 boundary Everything in this document runs on a single dev machine against `platform/docker-compose.dev.yml`. **Not built, and explicitly out of scope this phase:** production Hetzner/Proxmox deployment, production OIDC provider integration, a parent portal, payments, public signup, automated email, equipment inventory, a multi-host scheduler, Grafana/Prometheus production dashboards, public certificates, a marketplace/self-paced commerce layer, the six held KubeKosh scenarios, invasive proctoring, and AI-driven grading. All of that stays exactly where the Phase 3A doc's non-goals list and this document's §21 leave it — Phase 4 territory.