docs(phase3b): document the learning and assessment engine

New docs/PHASE3B_LEARNING_AND_ASSESSMENT.md covers the attempt/learning-record model,
progressive hints (including the Gateway response-rewrite mechanism), curriculum completion
rules, the assessment/exam engine and its timer/draw integrity guarantees, the RootNode
Capstone design, achievement records, the deliberately minimal versioning model, instructor
analytics, the Phase 3B RBAC matrix, security/integrity assumptions, dev/test commands, known
limitations, and the Phase 4 boundary.

Updates ENGINEERING_SOP.md (roadmap items 2/7/8 now done, new section 12, corrects a stale
capstone-placeholder description in section 4), LAB_ARCHITECTURE.md's status line,
README.md (corrects the 'Lab image unmodified' claim now that its frontend bundle carries
Phase 3B UX, plus new platform properties), platform/README.md, and platform/api/README.md.
This commit is contained in:
2026-08-18 14:36:13 -05:00
parent 8956a95446
commit 45a147a61f
6 changed files with 503 additions and 66 deletions
+46 -12
View File
@@ -71,14 +71,18 @@ Two separate structures, don't confuse them:
- **RootNode path** (`scenarios/rootnode-path.json`) — the guided onboarding sequence,
separate from bundles on purpose (it's *staged*, bundles are *flat*, and the bundle loader
would choke on a staged file if it were dropped in `scenarios/bundles/`). Loaded by
`loadCurriculumPath()`/`GET /api/curriculum/path`. Stages are Foundation → Workloads →
Networking → Configuration → Troubleshooting → Real Ops → Capstone, currently built by
reusing existing `1-k8s-basics` and `rootnode-real-ops` scenarios grouped by their existing
`category` field — no new scenario content was written for this. **Capstone stage is a
locked placeholder** (`"locked": true`, empty `scenario_ids`) — the capstone engine itself
is a separate, not-yet-built roadmap item (see §7, item 6). Do not put real scenarios in it
until that engine exists, or you'll reproduce the "duplicate beginner scenarios as capstone"
anti-pattern this path was built to avoid.
`loadCurriculumPath()`/`GET /api/curriculum/path` (raw content) and, as of Phase 3B,
`computeCurriculumState()`/`GET /api/curriculum/state` (rule-driven lock/complete/next-scenario
state — see `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` §4). Stages are Foundation → Workloads →
Networking → Configuration → Troubleshooting → Real Ops → Capstone, built by reusing existing
`1-k8s-basics` and `rootnode-real-ops` scenarios grouped by their existing `category` field —
no new scenario content was written for this. Each stage carries a `completion_rule`
(`all_required`/`threshold`/`prerequisites`). **Capstone is no longer a hardcoded placeholder**
— it's `"type": "capstone"` with an `"unlock_rule"` and a `"capstone_assessment_id"` pointing
at the `rootnode-capstone` assessment definition (Postgres, `assessment_definitions` table),
which is where its actual scenario draw lives — see the Phase 3B doc §7. Don't put real
scenarios directly into the JSON stage's `scenario_ids` for capstone; that field stays empty
by design, the draw is defined in the assessment definition instead.
To add/reorder a stage: edit `rootnode-path.json` directly, referencing existing
`scenarios/data/*.json` ids. No backend code change needed — it's read fresh via
@@ -138,7 +142,9 @@ Order agreed 2026-08-17/18. Status as of this doc:
1. RootNode identity/branding pass — **partial.** Shell banner fixed
(`⎈ RootNode Academy Lab — node: k8s-lab`). Upstream KubeKosh attribution left untouched in
footer/README/LICENSE. Broader "make Real Ops first-class" UI work not started.
2. RootNode curriculum mode — **backend done** (§4), frontend not started.
2. RootNode curriculum mode — **done**. Rule-driven stage completion/unlock as of Phase 3B
(§12), student-facing tabs (My Learning/Practice/Assessments/Capstone/Achievements) at
`frontend/src/platform/`.
3. Student accounts / persistent progress — **done**, Postgres-authoritative as of Phase 2B
(§9), sessions persistent as of Phase 3A (§10). Roster name-picking is still classroom/pilot
identification, not strong student auth — documented, not silently glossed over
@@ -156,15 +162,18 @@ Order agreed 2026-08-17/18. Status as of this doc:
documented adapter, not a working integration (docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md
§16).
6. Scenarios → teaching modules (incident context/investigation/prevention) — not started.
7. Capstone/exam engine — not started (placeholder stage only, see §4).
8. Progressive hint levels — not started.
7. Capstone/exam engine — **done** as of Phase 3B (§12). RootNode Capstone (Real Ops-weighted,
unlockable by curriculum-completion rule, not a permanent placeholder) plus a general exam
engine (definitions, cohort assignment, random scenario draw, server-authoritative timer).
8. Progressive hint levels — **done** as of Phase 3B (§12). Server-enforced sequential reveal,
tracked per attempt, policy-aware under assessment mode (free/disabled/flagged).
9. Production observability — not started.
10. CI scenario testing — not started.
11. Container/build optimization — explicitly deprioritized by design (platform works, not urgent).
Next slice: production OIDC/SSO, the production two-VM Proxmox deployment, and teaching-module
scenario content are the biggest remaining items. See
`docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` §18 for the full BUILT NOW vs. FUTURE split.
`docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` §21 for the full Phase 4 boundary.
## 8. Platform Provisioner (Phase 1)
@@ -246,3 +255,28 @@ Going forward: after any `npm install` that changes a service's dependencies, th
`package-lock.json` is part of the same commit as the `package.json` change — never left
untracked, never regenerated-and-ignored. Validate a lockfile actually matches its
`package.json` with `npm ci` (fails loudly on drift) before committing either.
## 12. Learning & Assessment Engine (Phase 3B)
Full design: `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md`.
- **What it is:** the teaching/assessment layer on top of everything Phases 1–3A built —
detailed per-attempt telemetry (`scenario_attempts`, evolved from Phase 2B's one-row-per-click
shape into a real attempt lifecycle), server-enforced progressive hints, rule-driven
curriculum stage/capstone unlock logic, a full exam + capstone assessment engine
(`assessment_definitions`/`assessment_sessions`/`assessment_session_scenarios`), and
persistent `achievements` records.
- **Where the boundary moved:** the Gateway now rewrites one Lab response
(`GET /lab/:token/api/scenarios/:scenarioId`, stripping `hints`) so progressive hints can be
genuinely server-gated rather than just UI-collapsed — the Lab image's own code/bytes are
still untouched; this is a response rewrite at the Gateway boundary, same pattern as every
other Lab-mediation decision made in earlier phases. The Lab's own frontend bundle *did*
change this phase (`ScenarioPanel.jsx`, new `ProgressiveHints`/`AssessmentBanner` components)
— Phase 3B explicitly authorized enhancing Lab UX, unlike prior phases.
- **Test procedure:** `platform/test/e2e-phase3b.js` — covers attempt telemetry, hint
sequencing/persistence-through-Lab-reset, curriculum/capstone lock-unlock, the exam engine
(random draw, timer, expiry, restart persistence), two-student isolation, and instructor
analytics/RBAC.
- **Known limitations:** see the Phase 3B doc §20 — no automatic abandoned-attempt detection, no
per-cohort needs-attention threshold configuration, raw-JSON pool_spec authoring in the admin
UI (correct, not yet a guided form).