docs(phase3b): document the learning and assessment engine
New docs/PHASE3B_LEARNING_AND_ASSESSMENT.md covers the attempt/learning-record model, progressive hints (including the Gateway response-rewrite mechanism), curriculum completion rules, the assessment/exam engine and its timer/draw integrity guarantees, the RootNode Capstone design, achievement records, the deliberately minimal versioning model, instructor analytics, the Phase 3B RBAC matrix, security/integrity assumptions, dev/test commands, known limitations, and the Phase 4 boundary. Updates ENGINEERING_SOP.md (roadmap items 2/7/8 now done, new section 12, corrects a stale capstone-placeholder description in section 4), LAB_ARCHITECTURE.md's status line, README.md (corrects the 'Lab image unmodified' claim now that its frontend bundle carries Phase 3B UX, plus new platform properties), platform/README.md, and platform/api/README.md.
This commit is contained in:
+46
-12
@@ -71,14 +71,18 @@ Two separate structures, don't confuse them:
|
||||
- **RootNode path** (`scenarios/rootnode-path.json`) — the guided onboarding sequence,
|
||||
separate from bundles on purpose (it's *staged*, bundles are *flat*, and the bundle loader
|
||||
would choke on a staged file if it were dropped in `scenarios/bundles/`). Loaded by
|
||||
`loadCurriculumPath()`/`GET /api/curriculum/path`. Stages are Foundation → Workloads →
|
||||
Networking → Configuration → Troubleshooting → Real Ops → Capstone, currently built by
|
||||
reusing existing `1-k8s-basics` and `rootnode-real-ops` scenarios grouped by their existing
|
||||
`category` field — no new scenario content was written for this. **Capstone stage is a
|
||||
locked placeholder** (`"locked": true`, empty `scenario_ids`) — the capstone engine itself
|
||||
is a separate, not-yet-built roadmap item (see §7, item 6). Do not put real scenarios in it
|
||||
until that engine exists, or you'll reproduce the "duplicate beginner scenarios as capstone"
|
||||
anti-pattern this path was built to avoid.
|
||||
`loadCurriculumPath()`/`GET /api/curriculum/path` (raw content) and, as of Phase 3B,
|
||||
`computeCurriculumState()`/`GET /api/curriculum/state` (rule-driven lock/complete/next-scenario
|
||||
state — see `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` §4). Stages are Foundation → Workloads →
|
||||
Networking → Configuration → Troubleshooting → Real Ops → Capstone, built by reusing existing
|
||||
`1-k8s-basics` and `rootnode-real-ops` scenarios grouped by their existing `category` field —
|
||||
no new scenario content was written for this. Each stage carries a `completion_rule`
|
||||
(`all_required`/`threshold`/`prerequisites`). **Capstone is no longer a hardcoded placeholder**
|
||||
— it's `"type": "capstone"` with an `"unlock_rule"` and a `"capstone_assessment_id"` pointing
|
||||
at the `rootnode-capstone` assessment definition (Postgres, `assessment_definitions` table),
|
||||
which is where its actual scenario draw lives — see the Phase 3B doc §7. Don't put real
|
||||
scenarios directly into the JSON stage's `scenario_ids` for capstone; that field stays empty
|
||||
by design, the draw is defined in the assessment definition instead.
|
||||
|
||||
To add/reorder a stage: edit `rootnode-path.json` directly, referencing existing
|
||||
`scenarios/data/*.json` ids. No backend code change needed — it's read fresh via
|
||||
@@ -138,7 +142,9 @@ Order agreed 2026-08-17/18. Status as of this doc:
|
||||
1. RootNode identity/branding pass — **partial.** Shell banner fixed
|
||||
(`⎈ RootNode Academy Lab — node: k8s-lab`). Upstream KubeKosh attribution left untouched in
|
||||
footer/README/LICENSE. Broader "make Real Ops first-class" UI work not started.
|
||||
2. RootNode curriculum mode — **backend done** (§4), frontend not started.
|
||||
2. RootNode curriculum mode — **done**. Rule-driven stage completion/unlock as of Phase 3B
|
||||
(§12), student-facing tabs (My Learning/Practice/Assessments/Capstone/Achievements) at
|
||||
`frontend/src/platform/`.
|
||||
3. Student accounts / persistent progress — **done**, Postgres-authoritative as of Phase 2B
|
||||
(§9), sessions persistent as of Phase 3A (§10). Roster name-picking is still classroom/pilot
|
||||
identification, not strong student auth — documented, not silently glossed over
|
||||
@@ -156,15 +162,18 @@ Order agreed 2026-08-17/18. Status as of this doc:
|
||||
documented adapter, not a working integration (docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md
|
||||
§16).
|
||||
6. Scenarios → teaching modules (incident context/investigation/prevention) — not started.
|
||||
7. Capstone/exam engine — not started (placeholder stage only, see §4).
|
||||
8. Progressive hint levels — not started.
|
||||
7. Capstone/exam engine — **done** as of Phase 3B (§12). RootNode Capstone (Real Ops-weighted,
|
||||
unlockable by curriculum-completion rule, not a permanent placeholder) plus a general exam
|
||||
engine (definitions, cohort assignment, random scenario draw, server-authoritative timer).
|
||||
8. Progressive hint levels — **done** as of Phase 3B (§12). Server-enforced sequential reveal,
|
||||
tracked per attempt, policy-aware under assessment mode (free/disabled/flagged).
|
||||
9. Production observability — not started.
|
||||
10. CI scenario testing — not started.
|
||||
11. Container/build optimization — explicitly deprioritized by design (platform works, not urgent).
|
||||
|
||||
Next slice: production OIDC/SSO, the production two-VM Proxmox deployment, and teaching-module
|
||||
scenario content are the biggest remaining items. See
|
||||
`docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md` §18 for the full BUILT NOW vs. FUTURE split.
|
||||
`docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` §21 for the full Phase 4 boundary.
|
||||
|
||||
## 8. Platform Provisioner (Phase 1)
|
||||
|
||||
@@ -246,3 +255,28 @@ Going forward: after any `npm install` that changes a service's dependencies, th
|
||||
`package-lock.json` is part of the same commit as the `package.json` change — never left
|
||||
untracked, never regenerated-and-ignored. Validate a lockfile actually matches its
|
||||
`package.json` with `npm ci` (fails loudly on drift) before committing either.
|
||||
|
||||
## 12. Learning & Assessment Engine (Phase 3B)
|
||||
|
||||
Full design: `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md`.
|
||||
|
||||
- **What it is:** the teaching/assessment layer on top of everything Phases 1–3A built —
|
||||
detailed per-attempt telemetry (`scenario_attempts`, evolved from Phase 2B's one-row-per-click
|
||||
shape into a real attempt lifecycle), server-enforced progressive hints, rule-driven
|
||||
curriculum stage/capstone unlock logic, a full exam + capstone assessment engine
|
||||
(`assessment_definitions`/`assessment_sessions`/`assessment_session_scenarios`), and
|
||||
persistent `achievements` records.
|
||||
- **Where the boundary moved:** the Gateway now rewrites one Lab response
|
||||
(`GET /lab/:token/api/scenarios/:scenarioId`, stripping `hints`) so progressive hints can be
|
||||
genuinely server-gated rather than just UI-collapsed — the Lab image's own code/bytes are
|
||||
still untouched; this is a response rewrite at the Gateway boundary, same pattern as every
|
||||
other Lab-mediation decision made in earlier phases. The Lab's own frontend bundle *did*
|
||||
change this phase (`ScenarioPanel.jsx`, new `ProgressiveHints`/`AssessmentBanner` components)
|
||||
— Phase 3B explicitly authorized enhancing Lab UX, unlike prior phases.
|
||||
- **Test procedure:** `platform/test/e2e-phase3b.js` — covers attempt telemetry, hint
|
||||
sequencing/persistence-through-Lab-reset, curriculum/capstone lock-unlock, the exam engine
|
||||
(random draw, timer, expiry, restart persistence), two-student isolation, and instructor
|
||||
analytics/RBAC.
|
||||
- **Known limitations:** see the Phase 3B doc §20 — no automatic abandoned-attempt detection, no
|
||||
per-cohort needs-attention threshold configuration, raw-JSON pool_spec authoring in the admin
|
||||
UI (correct, not yet a guided form).
|
||||
|
||||
Reference in New Issue
Block a user