rootnode-academy/docs/PHASE3B_LEARNING_AND_ASSESSMENT.md

# Phase 3B — Learning & Assessment Engine

Turns the platform infrastructure built in Phases 1–3A (identity, sessions, RBAC, disposable
Labs, Gateway routing, admin portal) into a real teaching and assessment system: detailed
attempt telemetry, progressive hints, rule-driven curriculum progression, a proper exam engine,
a RootNode Capstone, and persistent achievement records. Builds on
`docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` — read that first if you haven't.

## 1. What changed, in one paragraph

Phase 2B's `scenario_attempts` table was one row per validation click, written atomically at
completion, with `hint_count` deliberately left unmeasured. Phase 3B evolves it into a real
attempt lifecycle — one row per continuous working session on a scenario, with live counters
(validation attempts/passes/fails, hint reveals, duration) updated in place as the student
works. On top of that: a rule-driven curriculum-progression engine that actually locks/unlocks
stages instead of a permanent capstone placeholder; a progressive hint system that's genuinely
server-enforced (not just collapsed-by-default in the UI); a full exam/capstone assessment
engine with server-authoritative timers and random scenario draws; and persistent achievement
records for stage/curriculum/assessment/capstone completion.

## 2. Learning record model

`scenario_attempts` (migration `012`) gained: `cohort_id`, `curriculum_stage` (both snapshotted
at attempt-start, so a later cohort/curriculum change never rewrites history), `status`
(`in_progress`/`completed`/`abandoned`), `validation_attempt_count`, `validation_pass_count`,
`validation_fail_count`, `highest_hint_level`, `duration_seconds`, `assessment_session_id`.
`result` (Phase 2B) is now nullable — meaningful only once `status = 'completed'`.
`hint_count` (Phase 2B, always `NULL`) finally gets a `DEFAULT 0` and real increments.

This is an **evolution, not a new table** — Phase 2B's historical rows are backfilled (one
validation performed, matching their original one-row-per-click semantics), not dropped or
reshaped. `platform/api/src/attempts.js` owns the lifecycle:

- `getOrCreateOpenAttempt` — finds the open (`in_progress`) attempt for
  `(student, scenario, environment)`, or starts one. Environment-scoped so a destroyed/recreated
  Lab never silently resumes an attempt tied to compute that no longer exists.
- `recordValidation` — increments the attempt/pass/fail counters; closes the attempt
  (`status='completed'`, `duration_seconds` computed) only on a pass.
- `recordHintReveal` — sequential only: level *N* can't be revealed before *N-1* has been.
  Returns a 409 once every hint is exhausted.

No telemetry is invented that the Lab doesn't actually emit — `duration_seconds` is measured
server-side from `started_at` to completion (not client-reported), and hint reveals only exist
because Phase 3B builds a real reveal endpoint (§3), not because the old static hint array was
reinterpreted as "measured."

## 3. Validation telemetry & progressive hints

`POST /api/scenarios/:id/start` — fire-and-forget from the Lab UI when a scenario becomes
active (both task and MCQ types); creates/reopens the attempt with a real `started_at`.

`POST /api/progress/record` (Gateway-relayed from the Lab's existing validate/answer calls,
unchanged mechanism from Phase 2A/2B) now routes through `attempts.recordValidation` instead of
a flat insert, and — on a pass — checks curriculum/capstone/assessment consequences (§4, §6).

`POST /api/scenarios/:id/hint` reveals hints **one at a time, server-enforced**. This required a
real architectural change: Phase 2B's Lab backend (unmodified, per the "Lab image stays
unmodified" convention through Phase 3A) returns the scenario's full `hints` array in
`GET /api/scenarios/:id` — always did, since the Lab has no student/attempt concept at all. A
client could already see hint 3 in the raw response even with the UI collapsed. Making the gate
real meant the Gateway now **strips `hints` from that response** (replacing it with a bare
`hint_count`) for exactly the multi-student, Platform-mediated deployment
(`platform/gateway/src/server.js`, `GET /lab/:token/api/scenarios/:scenarioId`) — the Lab
image's own bytes are still untouched; this is a response rewrite at the boundary, the same
pattern already used for the `hints`-stripping shape as everything else the Gateway mediates.
Standalone/single-container dev mode never goes through the Gateway, so it keeps the original
fully-preloaded array unchanged — the frontend detects which mode it's in
(`scenario.hints === undefined && typeof scenario.hint_count === 'number'`) and switches to
`ProgressiveHints` (`frontend/src/components/ProgressiveHints.jsx`) only in the gated case.

Instructor-visible attempt summary matches the shape in the Phase 3B spec directly:

```
Student: Alice   Scenario: Probe Port Mismatch   Attempts: 2
Validation: Failed 2, Passed 1   Hints used: 1/3   Time: 14m 22s   Final: Completed
```

— all straight off `scenario_attempts` columns, no derived/estimated numbers, and explicitly
**not** turned into a score. See §12 for where instructors see this.

## 4. Curriculum completion rules

`scenarios/rootnode-path.json` gained a `version` field and, per stage, a `completion_rule`:

```json
{ "type": "all_required" }              // every scenario_id in the stage must be completed (default)
{ "type": "threshold", "percent": 80 }  // at least N% completed
{ "type": "prerequisites", "scenario_ids": [...] }  // only these specific ids must be completed
```

The capstone stage is no longer a hardcoded `"locked": true` placeholder — it carries
`"type": "capstone"`, an `"unlock_rule"` (`{"type":"stages_complete","stage_ids":[...]}`), and a
`"capstone_assessment_id"` pointing at the assessment definition that drives it (§6/§7).

`platform/api/src/curriculum-rules.js`'s `computeCurriculumState(path, progress, capstoneStatus)`
is the rule engine — pure, synchronous, no DB access — answering: per-stage locked/complete/
percent/next-scenario, the student's current stage, overall percent complete, and the capstone's
unlock status (server route supplies `capstoneStatus` after checking for a
`capstone_completed` achievement, since that's Postgres state this module doesn't touch).
Exposed at `GET /api/curriculum/state` (new; `GET /api/curriculum/path` from Phase 2A/2B is
unchanged and still used by the pre-existing dashboard summary).

Stage/curriculum completion also triggers achievement recording (§8) automatically, from inside
`POST /api/progress/record` — no separate "check for completion" step the student or frontend
has to remember to call.

## 5 & 6. Assessment model and exam engine

`PRACTICE` is just... practice: the existing curriculum flow, hints freely available (subject to
the scenario actually having any), unrestricted retries, no timer, no session concept.

`ASSESSMENT` (exam or capstone — same underlying engine, §7 covers what makes a capstone
different) is a **definition** (the reusable template: pool, duration, hint policy, completion
rule) producing **sessions** (one per student attempt, immutable snapshot of what was drawn and
under what rules). New tables (migration `013`):

```
assessment_definitions          slug PK, type ('exam'|'capstone'), title, duration_minutes,
                                 hint_policy ('free'|'disabled'|'flagged'), pool_spec JSONB,
                                 completion_rule JSONB, unlock_rule JSONB (capstone only),
                                 version INTEGER, status ('active'|'inactive')
assessment_cohort_assignments   which cohorts a definition is available to
assessment_sessions             one per student attempt — snapshots type/title/duration_minutes/
                                 hint_policy from the definition at draw time, plus
                                 definition_version, started_at, server-computed expires_at,
                                 submitted_at, random_seed, result JSONB
assessment_session_scenarios    the drawn set, ordered, with per-scenario status + a real FK
                                 back to the scenario_attempts row once worked
```

`exam_sessions` (Phase 1/2B, historical-import-only from the old Lab-local SQLite exam
bookkeeping) is **left untouched** — it was already scoped as dead historical data, never the
live mechanism, so there was nothing to migrate into the tables above without inventing data
that was never real. This is a deliberate non-duplication: the assessment tables above *are*
the real thing Phase 2B's note said didn't exist yet.

`pool_spec` shape:

```json
{ "required": ["scenario-id", ...], "random": [{ "count": 1, "from": ["a", "b", "c"] }, ...] }
```

`platform/api/src/assessments.js`'s `drawScenarios` includes every `required` id, plus `count`
random picks from each `random` group, shuffled together into presentation order. The draw uses
a tiny deterministic PRNG (mulberry32, seeded from `crypto.randomBytes` — no new dependency)
purely so the seed can be recorded for audit; a student never sees the seed or the undrawn pool
before starting.

**One assessment at a time per student** — `startSession` refuses if the student already has a
`not_started`/`in_progress` session for *any* definition. This keeps "which scenario is this
validate call happening inside" unambiguous for hint-policy enforcement (§19) without needing
the client to declare a mode.

**Timer**: `expires_at` is computed once, server-side, at `started_at + duration_minutes`, and
never recomputed from anything the client sends. Expiry is checked **lazily** — whenever a
session is read or written, if `now() > expires_at` and it's still `in_progress`, it flips to
`expired` right then. No background cron (explicitly out of scope this phase — see the
non-goals list). Verified live: a 1-minute test definition correctly transitioned to `expired`
on the first request after the minute passed, and unblocked starting a new assessment.

## 7. RootNode Capstone

A capstone is an `assessment_definitions` row with `type='capstone'` — same engine as an exam,
different framing and unlock gate. The shipped default (seeded in migration `013`, idempotent
`INSERT ... ON CONFLICT DO NOTHING`) is `rootnode-capstone`:

- **Required** (always drawn): all four RootNode Real Ops scenarios
  (`rootnode-runaway-monitor`, `broken-deployment`, `rootnode-probe-port-mismatch`,
  `rootnode-enforce-limits`) — "feature prominently," per the spec, satisfied by making them
  non-optional rather than just likely.
- **Random**: one Workloads task, one Configuration task, one Troubleshooting task — task-type
  scenarios preferred over MCQ, since the point is operational ability, not recall.
- 90 minutes, `hint_policy: 'flagged'` (available, but every use is recorded — appropriate for
  a capstone that should mostly reflect independent work without being punitive about needing a
  nudge).
- `unlock_rule`: every non-capstone curriculum stage complete (Foundation through Real Ops).

`GET /api/capstone` is a student-friendly view combining `computeCurriculumState`'s capstone
block with the definition and the student's own session history —
`locked` → `unlocked` → `completed`, with a prerequisite checklist while locked. Verified live,
end to end: locked with all six blocking stage ids, unlocked to zero blocking ids the moment the
last stage completed, a real draw (7 scenarios, all four Real Ops present), completion of all
seven finalizing the session and recording a `capstone_completed` achievement.

## 8. Achievement / completion records

`achievements` (migration `014`): `student_id`, `achievement_type`
(`stage_completed`/`curriculum_completed`/`assessment_completed`/`capstone_completed`),
`curriculum_path_id`/`curriculum_version` (for stage/curriculum types),
`assessment_session_id` (for assessment/capstone types), `achieved_at`, `metadata`.

`recordAchievement` is idempotent via a `WHERE NOT EXISTS` guard (not a unique-index/
`ON CONFLICT` dance — simple and correct at this write volume) — calling it again for something
already recorded is a safe no-op. Triggered automatically: `checkStageAndCurriculumAchievements`
runs inside `POST /api/progress/record` after every pass; `recordAssessmentAchievement` runs
after `assessments.recordScenarioOutcome`/`submitSession` finalizes a session, and only on an
actual pass (defensive — today's only `completion_rule` is `all_scenarios_passed`, which already
guarantees a `completed` session passed, but a future threshold-based rule shouldn't silently
grant achievements for a partial result).

**Not a certificate.** No public/shareable artifact is generated — this is explicitly a Phase
4/5 concern (see the non-goals list); a certificate can be rendered from these records later
without needing new data.

## 9. Assessment/exam Lab behavior

An assessment doesn't provision a separate Lab — the student works through their drawn
scenarios inside the **same** disposable environment their curriculum work already uses. This
is what makes "Lab reset doesn't erase assessment state" nearly free: assessment state
(`assessment_sessions`, `assessment_session_scenarios`, `scenario_attempts`) lives entirely in
Platform Postgres; the Lab environment is purely compute. Resetting/recreating it changes the
container, never the rows. Verified live: reset an environment mid-assessment,
`expires_at`/`status` on the session were unchanged afterward.

**Assessment-mode detection is entirely server-derived** — `findActiveAssessmentContextForScenario`
looks up whether the scenario a `validate`/`hint` call is happening against is currently in the
student's own active session's drawn set. Nothing the client sends declares "I'm in assessment
mode"; the server's own `assessment_session_scenarios` table is the only source of truth (see
§19).

If Lab infrastructure fails mid-assessment, the assessment session itself is unaffected — an
instructor can reset (or, if truly needed, destroy/reprovision) the student's Lab via the
existing admin environment controls (Phase 2B) without touching assessment state at all; there
is no code path that resets an assessment as a side effect of a Lab operation.

## 10 & 11. Student UI and Lab UX

`frontend/src/platform/StudentDashboard.jsx` is now tabbed: **My Learning** (the original
Phase 2A/2B "Up Next" + Launch/Reset Lab flow, extracted unchanged into `MyLearningTab.jsx` and
extended with an overall percent-complete bar from the new rule-driven curriculum state),
**Practice** (scenario/stage status + recent attempt history), **Assessments** (available/
active-with-live-countdown/history), **Capstone** (locked checklist → unlocked → start/resume →
completed, with attempt history), **Achievements** (a flat, dated list — not a leaderboard).

Inside the Lab UI itself (`frontend/src/components/ScenarioPanel.jsx`, `App.jsx`):
`ProgressiveHints` replaces the static hint array render when the Gateway has stripped hints
(§3); an `AssessmentBanner` (new, top-level, polls `/api/assessments/active`) shows a live
countdown and per-scenario completion count whenever an assessment is running — distinct from,
and rendered alongside, the pre-existing Lab-local exam-mode UI (`ExamTimer`/`ExamSession` etc.,
upstream KubeKosh feature, untouched). Both render nothing in standalone/single-container dev
mode (no Platform API to poll there), so neither regresses that deployment path.

## 12. Instructor learning analytics

`/admin` → **Learning**: cohort summary (student count, 7-day active-learner count, average
completion, per-stage completion counts), a **Needs Attention** list, and a per-student
drill-down (curriculum state, full attempt history with validation/hint counts, assessment
history). `/admin` → **Assessments**: create/edit definitions (admin/owner), assign to a cohort,
view/cancel sessions — visible to every instructor role, cohort-scoped server-side for the plain
`instructor` role the same way Phase 3A's other cohort-scoped routes are.

`GET /api/admin/scenario-analytics` (optional `?cohort_id=`): completions, distinct students,
average validation attempts, total failures, total hints used, average completion time — one
`GROUP BY scenario_id` query over `scenario_attempts`, nothing pre-aggregated or cached.

## 13. "Needs Attention"

`GET /api/admin/cohorts/:id/needs-attention` computes four signals per student, on read (no
stored table, no background job): **Repeated Attempts** (an open attempt with ≥3 failed
validations), **May Need Help** (an open attempt where a hint has been used but the scenario
isn't done), **Needs Attention — inactive** (no activity in 7 days), and **Needs Attention —
incomplete assessment** (their most recent assessment session expired without completing). All
four thresholds are named constants in `admin-learning-routes.js`
(`NEEDS_ATTENTION_FAIL_THRESHOLD`, `NEEDS_ATTENTION_STALE_DAYS`), not hardcoded inline. Wording
is deliberately instructional (`Needs Attention`, `May Need Help`, `Repeated Attempts`) — an
explicit test asserts none of the returned labels match punitive language.

## 14 & 15. Assessment/capstone management (admin)

A structured form (`frontend/src/admin/AssessmentsView.jsx`) — id, title, type, duration, hint
policy, and a raw JSON `pool_spec` textarea. Not a drag-and-drop authoring tool, per the explicit
scope line; the JSON textarea is honest about what's actually being edited rather than hiding it
behind partial UI that can't express every pool shape. Editing a definition's
duration/hint_policy/pool_spec/completion_rule bumps its `version` automatically
(`admin-learning-routes.js` diffs the incoming fields against the current row) — cosmetic edits
(title, description) don't. Create/edit is admin/owner only server-side; assign-to-cohort and
session view/cancel are available to a scoped `instructor` too, matching the RBAC matrix (§18).

## 16. Versioning

Deliberately minimal, per the explicit "don't overbuild full CMS versioning" instruction:

- **Curriculum path**: a plain `"version": 1` integer in `rootnode-path.json` itself, recorded
  on `stage_completed`/`curriculum_completed` achievements as a snapshot.
- **Assessment definitions**: a plain incrementing `version` integer, bumped only on a semantic
  change (duration/hint_policy/pool_spec/completion_rule — anything that would change what a
  session is graded against). `assessment_sessions.definition_version` snapshots which version
  was live at draw time, alongside a full redundant snapshot of the fields themselves
  (title/duration/hint_policy) — so a later definition edit, or even deletion, can never change
  what an already-started session is measured against.

No full version-history table exists for either — if that's ever needed, `version` is already a
column to build on, not a retrofit.

## 17. RBAC matrix (Phase 3B additions)

| Action | owner | admin | instructor | student |
|---|---|---|---|---|
| Create/edit assessment definitions | ✅ | ✅ | ❌ | ❌ |
| Assign an assessment to a cohort | ✅ | ✅ | ✅ (own cohorts) | ❌ |
| View/cancel assessment sessions | ✅ (any) | ✅ (any) | ✅ (own cohorts) | ❌ |
| View cohort learning summary / needs-attention / scenario analytics | ✅ (global) | ✅ (global) | ✅ (own cohorts) | ❌ |
| View a student's learning detail | ✅ | ✅ | ✅ (if enrolled in an owned cohort) | own only |
| Start/work an assessment, view own attempts/hints/achievements | ❌ | ❌ | ❌ | ✅ (own only) |

Enforced entirely server-side, reusing Phase 3A's `rbac.js` (`requireAdminOrOwner`,
`requireAnyInstructor`, `requireCohortAccess`, `requireStudentCohortAccess`) — no new
authorization primitives were needed, only new routes wired through the existing ones.

## 18. Security / integrity

- **Timer is server-authoritative** (§6) — `expires_at` fixed at draw time, never accepts a
  client value, checked lazily on every touch.
- **Scenario draw is server-stored** — `assessment_session_scenarios` is the only source of
  truth for what a student is supposed to be working on; nothing the client sends can add,
  remove, or reorder it.
- **A student cannot submit/read another student's session** — every assessment route filters
  by `student_id = req.studentId` server-side; a foreign session id returns 404, not 403 (same
  "don't confirm existence" posture as Phase 3A's RBAC denials) — verified by an explicit test.
- **Gateway ownership** (Phase 2A/2B/3A) is unchanged and still enforced on every proxied Lab
  request, HTTP and WebSocket, including the new `/start` and `/hint` relays.
- **Lab recreation never resets attempt/exam state** (§9) — verified live.
- **Hidden solution/pool data**: undrawn pool alternatives are never returned to the student
  (only the actual draw); hints beyond the currently-unlocked level are never sent (§3, the
  actual mechanism, not just a UI convention).
- **No invasive anti-cheat** — no browser surveillance, no lockdown mode. Explicitly out of
  scope, per the non-goals list.

## 19. Dev/test commands

```bash
cd platform && docker compose -f docker-compose.dev.yml up -d
(cd api && PLATFORM_DB_URL=postgres://postgres:postgres@localhost:5433/rootnode_platform \
  node bootstrap-owner.js [email protected] "Your Name" "a-real-password-12-chars-plus")

OWNER_EMAIL=[email protected] OWNER_PASSWORD=... node test/e2e-phase2a.js
OWNER_EMAIL=[email protected] OWNER_PASSWORD=... node test/e2e-phase2b.js
OWNER_EMAIL=[email protected] OWNER_PASSWORD=... node test/e2e-phase3a.js
OWNER_EMAIL=[email protected] OWNER_PASSWORD=... node test/e2e-phase3b.js
```

`e2e-phase3b.js` additionally shells out to `docker compose restart api` for its restart-
persistence case (set `SKIP_RESTART_TEST=1` to skip that one case if running somewhere that
isn't safe to restart).

## 20. Known limitations

- **No automatic "abandoned" detection** — an attempt a student never returns to stays
  `in_progress` forever; there's no reaper tying Lab destruction back to attempt state. A
  documented, deliberate scope line (would need wiring to Provisioner `environment_events`,
  more machinery than the instructional-visibility goal needs).
- **`cohort_instructors.role_in_cohort`** (Phase 3A) still isn't consulted for finer-grained
  assessment permissions — any cohort assignment grants the full scoped-instructor action set.
- **Needs-attention thresholds are fixed constants**, not per-cohort configurable yet.
- **No pagination** on the audit log (Phase 3A limitation, unchanged) or on scenario analytics.
- **Assessment pool_spec authoring is raw JSON** in the admin UI — correct and complete, but not
  a guided form; a validation-assist UI is a natural but unbuilt next step.
- **One assessment at a time per student** — intentional (§6), but means a student can't have a
  capstone and an exam "in flight" simultaneously even if that were ever desired.

## 21. Phase 4 boundary

Everything in this document runs on a single dev machine against
`platform/docker-compose.dev.yml`. **Not built, and explicitly out of scope this phase:**
production Hetzner/Proxmox deployment, production OIDC provider integration, a parent portal,
payments, public signup, automated email, equipment inventory, a multi-host scheduler,
Grafana/Prometheus production dashboards, public certificates, a marketplace/self-paced commerce
layer, the six held KubeKosh scenarios, invasive proctoring, and AI-driven grading. All of that
stays exactly where the Phase 3A doc's non-goals list and this document's §21 leave it — Phase 4
territory.