rootnode-academy/docs/PHASE3B_LEARNING_AND_ASSESSMENT.md

Phase 3B — Learning & Assessment Engine

Turns the platform infrastructure built in Phases 1–3A (identity, sessions, RBAC, disposable Labs, Gateway routing, admin portal) into a real teaching and assessment system: detailed attempt telemetry, progressive hints, rule-driven curriculum progression, a proper exam engine, a RootNode Capstone, and persistent achievement records. Builds on docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md — read that first if you haven't.

1. What changed, in one paragraph

Phase 2B's scenario_attempts table was one row per validation click, written atomically at completion, with hint_count deliberately left unmeasured. Phase 3B evolves it into a real attempt lifecycle — one row per continuous working session on a scenario, with live counters (validation attempts/passes/fails, hint reveals, duration) updated in place as the student works. On top of that: a rule-driven curriculum-progression engine that actually locks/unlocks stages instead of a permanent capstone placeholder; a progressive hint system that's genuinely server-enforced (not just collapsed-by-default in the UI); a full exam/capstone assessment engine with server-authoritative timers and random scenario draws; and persistent achievement records for stage/curriculum/assessment/capstone completion.

2. Learning record model

scenario_attempts (migration 012) gained: cohort_id, curriculum_stage (both snapshotted at attempt-start, so a later cohort/curriculum change never rewrites history), status (in_progress/completed/abandoned), validation_attempt_count, validation_pass_count, validation_fail_count, highest_hint_level, duration_seconds, assessment_session_id. result (Phase 2B) is now nullable — meaningful only once status = 'completed'. hint_count (Phase 2B, always NULL) finally gets a DEFAULT 0 and real increments.

This is an evolution, not a new table — Phase 2B's historical rows are backfilled (one validation performed, matching their original one-row-per-click semantics), not dropped or reshaped. platform/api/src/attempts.js owns the lifecycle:

• getOrCreateOpenAttempt — finds the open (in_progress) attempt for (student, scenario, environment), or starts one. Environment-scoped so a destroyed/recreated Lab never silently resumes an attempt tied to compute that no longer exists.
• recordValidation — increments the attempt/pass/fail counters; closes the attempt (status='completed', duration_seconds computed) only on a pass.
• recordHintReveal — sequential only: level N can't be revealed before N-1 has been. Returns a 409 once every hint is exhausted.

No telemetry is invented that the Lab doesn't actually emit — duration_seconds is measured server-side from started_at to completion (not client-reported), and hint reveals only exist because Phase 3B builds a real reveal endpoint (§3), not because the old static hint array was reinterpreted as "measured."

3. Validation telemetry & progressive hints

POST /api/scenarios/:id/start — fire-and-forget from the Lab UI when a scenario becomes active (both task and MCQ types); creates/reopens the attempt with a real started_at.

POST /api/progress/record (Gateway-relayed from the Lab's existing validate/answer calls, unchanged mechanism from Phase 2A/2B) now routes through attempts.recordValidation instead of a flat insert, and — on a pass — checks curriculum/capstone/assessment consequences (§4, §6).

POST /api/scenarios/:id/hint reveals hints one at a time, server-enforced. This required a real architectural change: Phase 2B's Lab backend (unmodified, per the "Lab image stays unmodified" convention through Phase 3A) returns the scenario's full hints array in GET /api/scenarios/:id — always did, since the Lab has no student/attempt concept at all. A client could already see hint 3 in the raw response even with the UI collapsed. Making the gate real meant the Gateway now strips hints from that response (replacing it with a bare hint_count) for exactly the multi-student, Platform-mediated deployment (platform/gateway/src/server.js, GET /lab/:token/api/scenarios/:scenarioId) — the Lab image's own bytes are still untouched; this is a response rewrite at the boundary, the same pattern already used for the hints-stripping shape as everything else the Gateway mediates. Standalone/single-container dev mode never goes through the Gateway, so it keeps the original fully-preloaded array unchanged — the frontend detects which mode it's in (scenario.hints === undefined && typeof scenario.hint_count === 'number') and switches to ProgressiveHints (frontend/src/components/ProgressiveHints.jsx) only in the gated case.

Instructor-visible attempt summary matches the shape in the Phase 3B spec directly:

Student: Alice   Scenario: Probe Port Mismatch   Attempts: 2
Validation: Failed 2, Passed 1   Hints used: 1/3   Time: 14m 22s   Final: Completed

— all straight off scenario_attempts columns, no derived/estimated numbers, and explicitly not turned into a score. See §12 for where instructors see this.

4. Curriculum completion rules

scenarios/rootnode-path.json gained a version field and, per stage, a completion_rule:

{ "type": "all_required" }              // every scenario_id in the stage must be completed (default)
{ "type": "threshold", "percent": 80 }  // at least N% completed
{ "type": "prerequisites", "scenario_ids": [...] }  // only these specific ids must be completed

The capstone stage is no longer a hardcoded "locked": true placeholder — it carries "type": "capstone", an "unlock_rule" ({"type":"stages_complete","stage_ids":[...]}), and a "capstone_assessment_id" pointing at the assessment definition that drives it (§6/§7).

platform/api/src/curriculum-rules.js's computeCurriculumState(path, progress, capstoneStatus) is the rule engine — pure, synchronous, no DB access — answering: per-stage locked/complete/ percent/next-scenario, the student's current stage, overall percent complete, and the capstone's unlock status (server route supplies capstoneStatus after checking for a capstone_completed achievement, since that's Postgres state this module doesn't touch). Exposed at GET /api/curriculum/state (new; GET /api/curriculum/path from Phase 2A/2B is unchanged and still used by the pre-existing dashboard summary).

Stage/curriculum completion also triggers achievement recording (§8) automatically, from inside POST /api/progress/record — no separate "check for completion" step the student or frontend has to remember to call.

5 & 6. Assessment model and exam engine

PRACTICE is just... practice: the existing curriculum flow, hints freely available (subject to the scenario actually having any), unrestricted retries, no timer, no session concept.

ASSESSMENT (exam or capstone — same underlying engine, §7 covers what makes a capstone different) is a definition (the reusable template: pool, duration, hint policy, completion rule) producing sessions (one per student attempt, immutable snapshot of what was drawn and under what rules). New tables (migration 013):

assessment_definitions          slug PK, type ('exam'|'capstone'), title, duration_minutes,
                                 hint_policy ('free'|'disabled'|'flagged'), pool_spec JSONB,
                                 completion_rule JSONB, unlock_rule JSONB (capstone only),
                                 version INTEGER, status ('active'|'inactive')
assessment_cohort_assignments   which cohorts a definition is available to
assessment_sessions             one per student attempt — snapshots type/title/duration_minutes/
                                 hint_policy from the definition at draw time, plus
                                 definition_version, started_at, server-computed expires_at,
                                 submitted_at, random_seed, result JSONB
assessment_session_scenarios    the drawn set, ordered, with per-scenario status + a real FK
                                 back to the scenario_attempts row once worked

exam_sessions (Phase 1/2B, historical-import-only from the old Lab-local SQLite exam bookkeeping) is left untouched — it was already scoped as dead historical data, never the live mechanism, so there was nothing to migrate into the tables above without inventing data that was never real. This is a deliberate non-duplication: the assessment tables above are the real thing Phase 2B's note said didn't exist yet.

pool_spec shape:

{ "required": ["scenario-id", ...], "random": [{ "count": 1, "from": ["a", "b", "c"] }, ...] }

platform/api/src/assessments.js's drawScenarios includes every required id, plus count random picks from each random group, shuffled together into presentation order. The draw uses a tiny deterministic PRNG (mulberry32, seeded from crypto.randomBytes — no new dependency) purely so the seed can be recorded for audit; a student never sees the seed or the undrawn pool before starting.

One assessment at a time per student — startSession refuses if the student already has a not_started/in_progress session for any definition. This keeps "which scenario is this validate call happening inside" unambiguous for hint-policy enforcement (§19) without needing the client to declare a mode.

Timer: expires_at is computed once, server-side, at started_at + duration_minutes, and never recomputed from anything the client sends. Expiry is checked lazily — whenever a session is read or written, if now() > expires_at and it's still in_progress, it flips to expired right then. No background cron (explicitly out of scope this phase — see the non-goals list). Verified live: a 1-minute test definition correctly transitioned to expired on the first request after the minute passed, and unblocked starting a new assessment.

7. RootNode Capstone

A capstone is an assessment_definitions row with type='capstone' — same engine as an exam, different framing and unlock gate. The shipped default (seeded in migration 013, idempotent INSERT ... ON CONFLICT DO NOTHING) is rootnode-capstone:

• Required (always drawn): all four RootNode Real Ops scenarios (rootnode-runaway-monitor, broken-deployment, rootnode-probe-port-mismatch, rootnode-enforce-limits) — "feature prominently," per the spec, satisfied by making them non-optional rather than just likely.
• Random: one Workloads task, one Configuration task, one Troubleshooting task — task-type scenarios preferred over MCQ, since the point is operational ability, not recall.
• 90 minutes, hint_policy: 'flagged' (available, but every use is recorded — appropriate for a capstone that should mostly reflect independent work without being punitive about needing a nudge).
• unlock_rule: every non-capstone curriculum stage complete (Foundation through Real Ops).

GET /api/capstone is a student-friendly view combining computeCurriculumState's capstone block with the definition and the student's own session history — locked → unlocked → completed, with a prerequisite checklist while locked. Verified live, end to end: locked with all six blocking stage ids, unlocked to zero blocking ids the moment the last stage completed, a real draw (7 scenarios, all four Real Ops present), completion of all seven finalizing the session and recording a capstone_completed achievement.

8. Achievement / completion records

achievements (migration 014): student_id, achievement_type (stage_completed/curriculum_completed/assessment_completed/capstone_completed), curriculum_path_id/curriculum_version (for stage/curriculum types), assessment_session_id (for assessment/capstone types), achieved_at, metadata.

recordAchievement is idempotent via a WHERE NOT EXISTS guard (not a unique-index/ ON CONFLICT dance — simple and correct at this write volume) — calling it again for something already recorded is a safe no-op. Triggered automatically: checkStageAndCurriculumAchievements runs inside POST /api/progress/record after every pass; recordAssessmentAchievement runs after assessments.recordScenarioOutcome/submitSession finalizes a session, and only on an actual pass (defensive — today's only completion_rule is all_scenarios_passed, which already guarantees a completed session passed, but a future threshold-based rule shouldn't silently grant achievements for a partial result).

Not a certificate. No public/shareable artifact is generated — this is explicitly a Phase 4/5 concern (see the non-goals list); a certificate can be rendered from these records later without needing new data.

9. Assessment/exam Lab behavior

An assessment doesn't provision a separate Lab — the student works through their drawn scenarios inside the same disposable environment their curriculum work already uses. This is what makes "Lab reset doesn't erase assessment state" nearly free: assessment state (assessment_sessions, assessment_session_scenarios, scenario_attempts) lives entirely in Platform Postgres; the Lab environment is purely compute. Resetting/recreating it changes the container, never the rows. Verified live: reset an environment mid-assessment, expires_at/status on the session were unchanged afterward.

Assessment-mode detection is entirely server-derived — findActiveAssessmentContextForScenario looks up whether the scenario a validate/hint call is happening against is currently in the student's own active session's drawn set. Nothing the client sends declares "I'm in assessment mode"; the server's own assessment_session_scenarios table is the only source of truth (see §19).

If Lab infrastructure fails mid-assessment, the assessment session itself is unaffected — an instructor can reset (or, if truly needed, destroy/reprovision) the student's Lab via the existing admin environment controls (Phase 2B) without touching assessment state at all; there is no code path that resets an assessment as a side effect of a Lab operation.

10 & 11. Student UI and Lab UX

frontend/src/platform/StudentDashboard.jsx is now tabbed: My Learning (the original Phase 2A/2B "Up Next" + Launch/Reset Lab flow, extracted unchanged into MyLearningTab.jsx and extended with an overall percent-complete bar from the new rule-driven curriculum state), Practice (scenario/stage status + recent attempt history), Assessments (available/ active-with-live-countdown/history), Capstone (locked checklist → unlocked → start/resume → completed, with attempt history), Achievements (a flat, dated list — not a leaderboard).

Inside the Lab UI itself (frontend/src/components/ScenarioPanel.jsx, App.jsx): ProgressiveHints replaces the static hint array render when the Gateway has stripped hints (§3); an AssessmentBanner (new, top-level, polls /api/assessments/active) shows a live countdown and per-scenario completion count whenever an assessment is running — distinct from, and rendered alongside, the pre-existing Lab-local exam-mode UI (ExamTimer/ExamSession etc., upstream KubeKosh feature, untouched). Both render nothing in standalone/single-container dev mode (no Platform API to poll there), so neither regresses that deployment path.

12. Instructor learning analytics

/admin → Learning: cohort summary (student count, 7-day active-learner count, average completion, per-stage completion counts), a Needs Attention list, and a per-student drill-down (curriculum state, full attempt history with validation/hint counts, assessment history). /admin → Assessments: create/edit definitions (admin/owner), assign to a cohort, view/cancel sessions — visible to every instructor role, cohort-scoped server-side for the plain instructor role the same way Phase 3A's other cohort-scoped routes are.

GET /api/admin/scenario-analytics (optional ?cohort_id=): completions, distinct students, average validation attempts, total failures, total hints used, average completion time — one GROUP BY scenario_id query over scenario_attempts, nothing pre-aggregated or cached.

13. "Needs Attention"

GET /api/admin/cohorts/:id/needs-attention computes four signals per student, on read (no stored table, no background job): Repeated Attempts (an open attempt with ≥3 failed validations), May Need Help (an open attempt where a hint has been used but the scenario isn't done), Needs Attention — inactive (no activity in 7 days), and Needs Attention — incomplete assessment (their most recent assessment session expired without completing). All four thresholds are named constants in admin-learning-routes.js (NEEDS_ATTENTION_FAIL_THRESHOLD, NEEDS_ATTENTION_STALE_DAYS), not hardcoded inline. Wording is deliberately instructional (Needs Attention, May Need Help, Repeated Attempts) — an explicit test asserts none of the returned labels match punitive language.

14 & 15. Assessment/capstone management (admin)

A structured form (frontend/src/admin/AssessmentsView.jsx) — id, title, type, duration, hint policy, and a raw JSON pool_spec textarea. Not a drag-and-drop authoring tool, per the explicit scope line; the JSON textarea is honest about what's actually being edited rather than hiding it behind partial UI that can't express every pool shape. Editing a definition's duration/hint_policy/pool_spec/completion_rule bumps its version automatically (admin-learning-routes.js diffs the incoming fields against the current row) — cosmetic edits (title, description) don't. Create/edit is admin/owner only server-side; assign-to-cohort and session view/cancel are available to a scoped instructor too, matching the RBAC matrix (§18).

16. Versioning

Deliberately minimal, per the explicit "don't overbuild full CMS versioning" instruction:

• Curriculum path: a plain "version": 1 integer in rootnode-path.json itself, recorded on stage_completed/curriculum_completed achievements as a snapshot.
• Assessment definitions: a plain incrementing version integer, bumped only on a semantic change (duration/hint_policy/pool_spec/completion_rule — anything that would change what a session is graded against). assessment_sessions.definition_version snapshots which version was live at draw time, alongside a full redundant snapshot of the fields themselves (title/duration/hint_policy) — so a later definition edit, or even deletion, can never change what an already-started session is measured against.

No full version-history table exists for either — if that's ever needed, version is already a column to build on, not a retrofit.

17. RBAC matrix (Phase 3B additions)

Action	owner	admin	instructor	student
Create/edit assessment definitions	✅	✅	❌	❌
Assign an assessment to a cohort	✅	✅	✅ (own cohorts)	❌
View/cancel assessment sessions	✅ (any)	✅ (any)	✅ (own cohorts)	❌
View cohort learning summary / needs-attention / scenario analytics	✅ (global)	✅ (global)	✅ (own cohorts)	❌
View a student's learning detail	✅	✅	✅ (if enrolled in an owned cohort)	own only
Start/work an assessment, view own attempts/hints/achievements	❌	❌	❌	✅ (own only)

Enforced entirely server-side, reusing Phase 3A's rbac.js (requireAdminOrOwner, requireAnyInstructor, requireCohortAccess, requireStudentCohortAccess) — no new authorization primitives were needed, only new routes wired through the existing ones.

18. Security / integrity

• Timer is server-authoritative (§6) — expires_at fixed at draw time, never accepts a client value, checked lazily on every touch.
• Scenario draw is server-stored — assessment_session_scenarios is the only source of truth for what a student is supposed to be working on; nothing the client sends can add, remove, or reorder it.
• A student cannot submit/read another student's session — every assessment route filters by student_id = req.studentId server-side; a foreign session id returns 404, not 403 (same "don't confirm existence" posture as Phase 3A's RBAC denials) — verified by an explicit test.
• Gateway ownership (Phase 2A/2B/3A) is unchanged and still enforced on every proxied Lab request, HTTP and WebSocket, including the new /start and /hint relays.
• Lab recreation never resets attempt/exam state (§9) — verified live.
• Hidden solution/pool data: undrawn pool alternatives are never returned to the student (only the actual draw); hints beyond the currently-unlocked level are never sent (§3, the actual mechanism, not just a UI convention).
• No invasive anti-cheat — no browser surveillance, no lockdown mode. Explicitly out of scope, per the non-goals list.

19. Dev/test commands

cd platform && docker compose -f docker-compose.dev.yml up -d
(cd api && PLATFORM_DB_URL=postgres://postgres:postgres@localhost:5433/rootnode_platform \
  node bootstrap-owner.js [email protected] "Your Name" "a-real-password-12-chars-plus")

OWNER_EMAIL=[email protected] OWNER_PASSWORD=... node test/e2e-phase2a.js
OWNER_EMAIL=[email protected] OWNER_PASSWORD=... node test/e2e-phase2b.js
OWNER_EMAIL=[email protected] OWNER_PASSWORD=... node test/e2e-phase3a.js
OWNER_EMAIL=[email protected] OWNER_PASSWORD=... node test/e2e-phase3b.js

e2e-phase3b.js additionally shells out to docker compose restart api for its restart- persistence case (set SKIP_RESTART_TEST=1 to skip that one case if running somewhere that isn't safe to restart).

20. Known limitations

• No automatic "abandoned" detection — an attempt a student never returns to stays in_progress forever; there's no reaper tying Lab destruction back to attempt state. A documented, deliberate scope line (would need wiring to Provisioner environment_events, more machinery than the instructional-visibility goal needs).
• cohort_instructors.role_in_cohort (Phase 3A) still isn't consulted for finer-grained assessment permissions — any cohort assignment grants the full scoped-instructor action set.
• Needs-attention thresholds are fixed constants, not per-cohort configurable yet.
• No pagination on the audit log (Phase 3A limitation, unchanged) or on scenario analytics.
• Assessment pool_spec authoring is raw JSON in the admin UI — correct and complete, but not a guided form; a validation-assist UI is a natural but unbuilt next step.
• One assessment at a time per student — intentional (§6), but means a student can't have a capstone and an exam "in flight" simultaneously even if that were ever desired.

21. Phase 4 boundary

Everything in this document runs on a single dev machine against platform/docker-compose.dev.yml. Not built, and explicitly out of scope this phase: production Hetzner/Proxmox deployment, production OIDC provider integration, a parent portal, payments, public signup, automated email, equipment inventory, a multi-host scheduler, Grafana/Prometheus production dashboards, public certificates, a marketplace/self-paced commerce layer, the six held KubeKosh scenarios, invasive proctoring, and AI-driven grading. All of that stays exactly where the Phase 3A doc's non-goals list and this document's §21 leave it — Phase 4 territory.