Compare commits
44
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
bd0a10fbfb | ||
|
|
01d3764138 | ||
|
|
903b0ead43 | ||
|
|
7f6e1857c0 | ||
|
|
2db136875f | ||
|
|
505783e5f7 | ||
|
|
e0f954e714 | ||
|
|
54b35e8ad7 | ||
|
|
de6c5b3286 | ||
|
|
831486d664 | ||
|
|
960a638ec9 | ||
|
|
73cea1ba1d | ||
|
|
dbcc210889 | ||
|
|
86ea035e9b | ||
|
|
25409d3bd7 | ||
|
|
046a0fba18 | ||
|
|
bee57d89fd | ||
|
|
82f17a05e8 | ||
|
|
fd38d06055 | ||
|
|
8ffc923a84 | ||
|
|
db927b8081 | ||
|
|
8c09ef8f3e | ||
|
|
28c925069b | ||
|
|
e2298e95de | ||
|
|
fe702754cd | ||
|
|
f152b9a489 | ||
|
|
e4cbcf6c40 | ||
|
|
4c0a950afb | ||
|
|
7baf7074e5 | ||
|
|
777efd168d | ||
|
|
b38c46aaec | ||
|
|
be92be81cd | ||
|
|
f25535027c | ||
|
|
3c7f0ca59c | ||
|
|
9d382c95c5 | ||
|
|
5183a6af09 | ||
|
|
1c08a30c7f | ||
|
|
23f881884d | ||
|
|
f89a769540 | ||
|
|
4bc202e5f3 | ||
|
|
1b6da0a966 | ||
|
|
17160da89e | ||
|
|
611e1ac4ba | ||
|
|
895e16fea4 |
@@ -163,6 +163,19 @@ its core:
|
||||
classes from `/admin` (presence, Lab/learning status, help-request queue, announcements,
|
||||
guidance-only current-activity) — RootNode owns identity/coordination, an external service
|
||||
(Meet/Zoom/Teams/etc.) owns the actual video/voice.
|
||||
- Students can now be enrolled in a self-paced program (as of Phase 5B) — "My Programs" on the
|
||||
student dashboard, entitlement enforced server-side (pending/future/expired/cancelled
|
||||
enrollments cannot launch a Lab no matter what the browser sends), completion server-derived
|
||||
from the existing capstone rules. `/admin` gained a Programs area (program authoring, manual
|
||||
enrollment, activate/cancel). A separate, provider-neutral `POST /api/integrations/enrollments`
|
||||
boundary — server-to-server only, no browser session involved — is what the future public
|
||||
`rootnodeacademy.com` website/checkout will call to create these enrollments; see
|
||||
`docs/PHASE5B_SELF_PACED_ENROLLMENT.md` and `docs/WEBSITE_ENROLLMENT_INTEGRATION.md`.
|
||||
- As of Phase 5C, RootNode actually tells students what's happening — real activation,
|
||||
enrollment-confirmation, and password-reset email, delivered asynchronously through a
|
||||
provider-neutral notification worker (Google Workspace via the Gmail API in production, a
|
||||
deterministic dev provider everywhere else — no test/dev run ever emails a real person). See
|
||||
`docs/PHASE5C_ENROLLMENT_COMMUNICATIONS.md` and `docs/RUNBOOK_GOOGLE_WORKSPACE_EMAIL.md`.
|
||||
|
||||
**Never publish the raw privileged container** — front it with isolation (that's what the
|
||||
Gateway is). The one-container `docker run --privileged -p 7554:80 ...` command above stays
|
||||
@@ -200,7 +213,8 @@ Key properties of the platform (not this image):
|
||||
Production topology and what's built vs. not yet: `docs/LAB_ARCHITECTURE.md`,
|
||||
`docs/PHASE1_PLATFORM_FOUNDATION.md`, `docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md`,
|
||||
`docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md`, `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md`,
|
||||
`docs/PHASE5A_STUDENT_IDENTITY_REMOTE_CLASSROOM.md`.
|
||||
`docs/PHASE5A_STUDENT_IDENTITY_REMOTE_CLASSROOM.md`, `docs/PHASE5B_SELF_PACED_ENROLLMENT.md`,
|
||||
`docs/PHASE5C_ENROLLMENT_COMMUNICATIONS.md`.
|
||||
|
||||
The production infrastructure package (Proxmox/Ansible/Terraform/firewall/monitoring/backup/CI)
|
||||
that runs all of this on real hardware lives in `infra/` — see
|
||||
|
||||
+108
@@ -0,0 +1,108 @@
|
||||
# What Kubernetes Solves & the Desired-State Model
|
||||
|
||||
## Before Kubernetes: the problem
|
||||
|
||||
Imagine you're running a web application on three servers by hand. One server crashes at 2am —
|
||||
someone has to notice, log in, and restart the application. Traffic grows, so you need a fourth
|
||||
server — someone has to provision it, install dependencies, and configure it to match the other
|
||||
three. You ship a new version — someone has to update all four servers, one at a time, without
|
||||
taking the whole application down.
|
||||
|
||||
None of this is hard in isolation. It becomes unmanageable at scale, and it depends entirely on
|
||||
a human noticing and reacting fast enough. Kubernetes exists to make a computer do that
|
||||
noticing and reacting instead — continuously, automatically, and the same way every time.
|
||||
|
||||
## The core idea: desired state
|
||||
|
||||
You don't tell Kubernetes *how* to run your application step by step. You tell it *what you
|
||||
want to be true* — "3 replicas of this container should be running" — and Kubernetes
|
||||
continuously works to make that true, correcting drift whenever reality doesn't match.
|
||||
|
||||
This is the single most important mental shift for someone new to Kubernetes: you're not
|
||||
writing a script that runs once. You're declaring a **desired state**, and a background process
|
||||
never stops comparing the real world against it.
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: web
|
||||
spec:
|
||||
replicas: 3 # <- the desired state: "3 of these, always"
|
||||
...
|
||||
```
|
||||
|
||||
If a Pod crashes, Kubernetes doesn't "restart the crashed one" as a special case — it just
|
||||
notices reality (2 running) no longer matches desired state (3 wanted) and creates a
|
||||
replacement. Same mechanism, no special-casing.
|
||||
|
||||
## The cluster mental model
|
||||
|
||||
A Kubernetes cluster has two conceptually different kinds of machines:
|
||||
|
||||
- **The control plane** — the "brain." It stores the desired state (in `etcd`), watches actual
|
||||
state, and makes decisions (the API server, scheduler, and controllers all live here). You
|
||||
talk to the control plane through `kubectl`; you never SSH into it to manage workloads.
|
||||
- **Worker nodes** — where your containers actually run. Each node runs a `kubelet` (an agent
|
||||
that starts/stops containers as the control plane instructs) and a container runtime.
|
||||
|
||||
You describe desired state to the control plane. The control plane decides which nodes should
|
||||
run what. The nodes' kubelets make it actually happen. You never manually place a Pod on a
|
||||
specific node in normal operation — that placement decision belongs to the scheduler.
|
||||
|
||||
## Declarative, not imperative
|
||||
|
||||
"Imperative" means giving step-by-step instructions ("start a container, then another, then
|
||||
configure networking between them"). "Declarative" means describing an end state and letting
|
||||
the system figure out the steps. Kubernetes' API objects (Pods, Deployments, Services, and
|
||||
everything else you'll learn in ROOT_) are declarative — you write YAML describing what should
|
||||
exist, `kubectl apply` submits it, and the control plane does the rest.
|
||||
|
||||
This is also why Kubernetes YAML looks the way it does: it's not a script, it's a specification.
|
||||
|
||||
## Pods, Namespaces, and labels — the vocabulary you need first
|
||||
|
||||
- **Pod** — the smallest deployable unit. Usually one container, sometimes a small group of
|
||||
tightly-coupled containers that must run together on the same node and share networking.
|
||||
You'll rarely create bare Pods directly in real use (a Deployment manages them for you, see
|
||||
ROOT 06) but understanding the Pod itself is foundational to everything built on top of it.
|
||||
- **Namespace** — a way to partition one cluster into logically separate areas (e.g. `dev`,
|
||||
`staging`, teams, or projects), so names don't collide and access/quotas can be scoped.
|
||||
- **Labels** — arbitrary key/value tags attached to objects (`app: web`, `env: prod`). Labels
|
||||
are how almost everything in Kubernetes finds related objects — a Deployment finds "its" Pods
|
||||
by label selector, not by name pattern matching.
|
||||
|
||||
## kubectl: your one tool for all of this
|
||||
|
||||
`kubectl` is the command-line client that talks to the control plane's API server. Nearly
|
||||
everything you'll do in ROOT_ Labs goes through it: `kubectl get` (list/inspect),
|
||||
`kubectl describe` (detailed status + recent events — your primary troubleshooting tool),
|
||||
`kubectl apply -f` (submit a desired-state YAML file), `kubectl logs` (see what a container
|
||||
printed). You'll build real fluency with these across this module and the ones that follow.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read, 10-12 minutes for a live walkthrough with discussion.
|
||||
|
||||
**Demo plan:** Before the demonstration activity, it's worth drawing the control-plane/worker
|
||||
split on a whiteboard (or sharing a simple diagram) and tracing one request: `kubectl apply` ->
|
||||
API server -> etcd -> scheduler -> kubelet -> container runtime. Students retain the mental
|
||||
model much better when they've seen the path, not just read about it.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often assume `kubectl` talks directly to nodes/containers. It doesn't — it only ever
|
||||
talks to the API server. Worth stating explicitly.
|
||||
- "Declarative" is sometimes read as "automatic" in a vague sense. Be concrete: it means "you
|
||||
describe the end state; a controller loop makes it true," not "Kubernetes guesses what you
|
||||
want."
|
||||
|
||||
**Discussion question:** "If desired state says 3 replicas and you manually run `kubectl delete
|
||||
pod` on one of them, what happens next, and why?" (Answer: a new one is created almost
|
||||
immediately — deleting a Pod doesn't change the Deployment's desired replica count, so the
|
||||
control loop restores it.)
|
||||
|
||||
**Optional extension:** Ask students to predict what would happen if they edited a running Pod's
|
||||
container image directly with `kubectl edit pod` instead of updating the Deployment that manages
|
||||
it (answer: it works until that specific Pod is replaced for any reason, at which point the new
|
||||
Pod reverts to whatever the Deployment's template says — a good early illustration of "the
|
||||
Deployment is the source of truth, not the running Pod").
|
||||
+76
@@ -0,0 +1,76 @@
|
||||
# Demonstration: Desired State Reconciliation
|
||||
|
||||
Watch what actually happens when reality drifts from desired state — this is the mechanism the
|
||||
whole lesson describes, made concrete.
|
||||
|
||||
## Setup
|
||||
|
||||
A Namespace called `demo` exists with a Deployment `web` declaring `replicas: 2`. Right now,
|
||||
reality matches desired state:
|
||||
|
||||
```
|
||||
$ kubectl get pods -n demo
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
web-7d8f9c6b5d-abcde 1/1 Running 0 3m
|
||||
web-7d8f9c6b5d-fghij 1/1 Running 0 3m
|
||||
```
|
||||
|
||||
## Someone (or something) deletes one Pod
|
||||
|
||||
```
|
||||
$ kubectl delete pod web-7d8f9c6b5d-abcde -n demo
|
||||
pod "web-7d8f9c6b5d-abcde" deleted
|
||||
```
|
||||
|
||||
## What happens in the next second or two, without anyone doing anything else
|
||||
|
||||
```
|
||||
$ kubectl get pods -n demo
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
web-7d8f9c6b5d-fghij 1/1 Running 0 3m
|
||||
web-7d8f9c6b5d-klmno 0/1 ContainerCreating 0 1s
|
||||
```
|
||||
|
||||
A brand new Pod (`...-klmno`, a new random suffix — this is NOT the deleted Pod coming back)
|
||||
appeared on its own. Nobody ran `kubectl create` again. The Deployment's controller noticed
|
||||
actual state (1 running Pod matching its label selector) no longer matched desired state
|
||||
(`replicas: 2`), and created exactly one replacement to close the gap — not two, not zero.
|
||||
|
||||
A few seconds later:
|
||||
|
||||
```
|
||||
$ kubectl get pods -n demo
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
web-7d8f9c6b5d-fghij 1/1 Running 0 3m
|
||||
web-7d8f9c6b5d-klmno 1/1 Running 0 8s
|
||||
```
|
||||
|
||||
Back to 2/2 — desired state restored, entirely without human intervention.
|
||||
|
||||
## Why this matters beyond the demo
|
||||
|
||||
This exact mechanism is what makes a node crash, a Pod eviction, or a container being
|
||||
OOM-killed something Kubernetes recovers from automatically instead of something that pages a
|
||||
human at 2am. The controller loop doesn't know or care WHY a Pod disappeared — crash, manual
|
||||
delete, node failure — it only ever asks "does reality match desired state right now?" and acts
|
||||
if not. Understanding this one loop explains a huge amount of Kubernetes' behavior that would
|
||||
otherwise look like unrelated special cases.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Run this live rather than only showing the captured output above — have
|
||||
students watch `kubectl get pods -n demo --watch` in one terminal while you delete a Pod from
|
||||
another. Watching the new Pod name/age appear in real time lands the "this is a NEW Pod, not a
|
||||
restart" point far better than static text.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; 5 minutes as a self-paced read.
|
||||
|
||||
**Common misconception to correct:** students frequently describe this as "Kubernetes restarted
|
||||
the Pod." Correct this explicitly — a deleted Pod is gone permanently; a *new* Pod object was
|
||||
created to satisfy the replica count. This distinction matters later when debugging (the new
|
||||
Pod has zero history from the old one — no old logs, no old restart count).
|
||||
|
||||
**Optional extension:** ask what would happen if you deleted the Deployment itself instead of
|
||||
one Pod (answer: all of its Pods are deleted and NOT replaced, because the thing declaring
|
||||
desired state is now gone — a good way to reinforce that the Deployment, not any individual
|
||||
Pod, is the actual source of truth).
|
||||
@@ -0,0 +1,48 @@
|
||||
# Knowledge Check: The Kubernetes Mental Model
|
||||
|
||||
Answer each question yourself before reading the answer that follows it.
|
||||
|
||||
---
|
||||
|
||||
**1. You run `kubectl delete pod` on one Pod belonging to a Deployment with `replicas: 3`. A
|
||||
few seconds later, `kubectl get pods` shows 3 Pods again, but one has a different name than any
|
||||
of the original three. What actually happened?**
|
||||
|
||||
*Answer:* The deleted Pod is gone permanently. The Deployment's controller noticed actual state
|
||||
(2 Pods) no longer matched desired state (3 Pods) and created ONE new Pod to close the gap. It
|
||||
is not the old Pod restarting — it's a brand new object.
|
||||
|
||||
---
|
||||
|
||||
**2. A Deployment has `replicas: 3`, and there are 5 nodes in the cluster. Which of these
|
||||
decides WHICH node each of the 3 Pods actually lands on?**
|
||||
|
||||
*Answer:* The scheduler (part of the control plane), not the person who wrote the Deployment
|
||||
YAML and not the nodes themselves. You declare desired state; the scheduler makes the placement
|
||||
decision based on available resources and any placement rules configured.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: `kubectl` communicates directly with the kubelet on whichever node your
|
||||
Pod happens to be running on.**
|
||||
|
||||
*Answer:* False. `kubectl` only ever talks to the API server (part of the control plane). The
|
||||
API server is the single entry point for every read and write — it never bypasses the control
|
||||
plane to reach a node directly for a normal `get`/`apply`/`describe` call.
|
||||
|
||||
---
|
||||
|
||||
**4. Why is Kubernetes YAML described as "declarative" rather than a script?**
|
||||
|
||||
*Answer:* A script describes a sequence of steps to execute once. Kubernetes YAML describes an
|
||||
END STATE you want to exist ("3 replicas of this container should be running") — a controller
|
||||
loop then continuously works to keep reality matching that description, correcting drift
|
||||
whenever it appears, indefinitely, not just once at apply time.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Ask these aloud in a live classroom before revealing answers — question 1 in
|
||||
particular tends to surface the "it restarted" misconception the demonstration activity should
|
||||
have already addressed; if it hasn't landed, this is the moment to circle back.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+57
@@ -0,0 +1,57 @@
|
||||
# Guided Exercise: Planning a Namespace Layout
|
||||
|
||||
This is a paper-and-thinking exercise — work through it before the Lab, not during it.
|
||||
|
||||
## The scenario
|
||||
|
||||
You're setting up a small Kubernetes cluster for a class project with three teams (`frontend`,
|
||||
`backend`, `data`), each running their own set of Pods, and you also need a shared area for
|
||||
tools everyone uses (a shared logging dashboard, for example).
|
||||
|
||||
## Your task
|
||||
|
||||
Sketch out, on paper or in a scratch file:
|
||||
|
||||
1. What Namespaces would you create, and why? (Hint: think about who needs to see what, and
|
||||
whether any team's mistake — e.g. deleting the wrong Pod by accident — should be able to
|
||||
affect another team.)
|
||||
2. If `frontend` team members should be able to manage their own Pods but NOT touch `backend`'s
|
||||
or `data`'s, what does that suggest about how Namespaces relate to access control? (You'll
|
||||
learn the actual mechanism, RBAC, in ROOT 09 — for now just reason about why Namespace
|
||||
boundaries make that kind of restriction possible at all.)
|
||||
3. Would you put the shared logging dashboard in one of the three team Namespaces, or somewhere
|
||||
else? Justify your answer.
|
||||
4. If two teams both need to run something they each call `web`, what would happen if they were
|
||||
in the SAME Namespace? What changes if they're in separate Namespaces?
|
||||
|
||||
## A reasonable answer to check yourself against
|
||||
|
||||
1. Three Namespaces (`frontend`, `backend`, `data`) — one per team, so each team's objects are
|
||||
isolated from the others and a mistake in one Namespace (deleting the wrong Pod, applying a
|
||||
bad manifest) can't accidentally affect a different team's workloads.
|
||||
2. Namespace boundaries are what RBAC rules can be scoped TO — "this person can manage Pods in
|
||||
the `frontend` Namespace" is a real, common permission model. Without separate Namespaces,
|
||||
you'd have to distinguish access by resource name pattern instead of by boundary, which is
|
||||
far more error-prone.
|
||||
3. A separate Namespace (e.g. `shared-tools` or `platform`) — it isn't owned by any one team,
|
||||
and mixing it into a team's Namespace would make it unclear who's responsible for it or
|
||||
accidentally couple its lifecycle to that team's.
|
||||
4. In the SAME Namespace, object names must be unique — a second `web` Deployment would either
|
||||
fail to create or overwrite the first, depending on how it's applied. In SEPARATE
|
||||
Namespaces, both `frontend/web` and `backend/web` can coexist without conflict — the
|
||||
Namespace is part of what makes a name unique, not just the name itself.
|
||||
|
||||
If your answer differed meaningfully from this, re-read the lesson's Namespace section before
|
||||
continuing to the Lab.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Good as a 2-3 person small-group discussion in a live classroom before
|
||||
revealing the reasonable answer — question 4 in particular often produces a genuine "oh" moment
|
||||
once students realize name uniqueness is Namespace-scoped, not cluster-wide.
|
||||
|
||||
**Approximate timing:** 10 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what would need to be true for the `data` team to be ALLOWED to read
|
||||
(but not modify) something in `shared-tools` — a good bridge to RBAC's read vs. write verbs,
|
||||
covered properly in ROOT 09.
|
||||
@@ -0,0 +1,44 @@
|
||||
# Launch Lab: Working with Namespaces
|
||||
|
||||
Time to apply the mental model for real — you'll create an actual isolated Namespace and deploy
|
||||
into it, in your own disposable Lab environment.
|
||||
|
||||
## What you're doing
|
||||
|
||||
You'll create a Namespace (`team-alpha`) and deploy an application into it with a specific
|
||||
replica count — directly applying the "isolated area within one cluster" idea from the guided
|
||||
exercise, and the desired-state model from the lesson (once you declare `replicas: 2`, that
|
||||
becomes the standing desired state for that Deployment, enforced continuously).
|
||||
|
||||
## Before you start
|
||||
|
||||
Think back to the demonstration: once your Deployment exists with its desired replica count,
|
||||
Kubernetes will keep it there on its own. You don't need to "check back" and manually fix
|
||||
anything — if you want to prove this to yourself, try deleting one of your Pods after you've
|
||||
finished the task and watch what happens.
|
||||
|
||||
## Your task in this Lab
|
||||
|
||||
Create the Namespace, then deploy the application inside it with the exact replica count
|
||||
specified. Use `kubectl get namespace` and `kubectl get deployment -n <namespace>` to confirm
|
||||
your work — the same verification pattern you'll use constantly throughout ROOT_.
|
||||
|
||||
This Lab has its own in-context hints if you get stuck. Try working from the lesson and guided
|
||||
exercise first — the commands you need are exactly the ones described there.
|
||||
|
||||
When you're done, return to this module to close out with a reflection on why this isolation
|
||||
matters beyond just "keeping things organized."
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Classroom flow:** This is most students' first real `kubectl apply`/`create` in ROOT_ beyond
|
||||
whatever pilot exposure they've had — a good moment to circulate and check that basic
|
||||
`kubectl` command syntax isn't the blocker (as opposed to the Namespace concept itself, which
|
||||
the guided exercise should have already addressed).
|
||||
|
||||
**Approximate timing:** 10-15 minutes.
|
||||
|
||||
**Common mistake to watch for:** creating the Deployment in the DEFAULT namespace instead of
|
||||
`team-alpha` (forgetting `-n team-alpha` or the namespace field in the manifest) — this is a
|
||||
very common first mistake and a good real-world illustration of why Namespace isolation only
|
||||
works if you're actually specifying it every time.
|
||||
@@ -0,0 +1,44 @@
|
||||
# Reflection: Why Desired State Matters in Production
|
||||
|
||||
You just created an isolated Namespace and a Deployment with a declared replica count. Take a
|
||||
few minutes to connect this to why it matters beyond passing this Lab's validation.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. If your `team-alpha` Namespace were a real team's production environment, what's one mistake
|
||||
in a DIFFERENT namespace that Namespace isolation would have protected you from?
|
||||
2. You declared `replicas: 2` rather than manually creating two Pods. What would you have had
|
||||
to do manually, over and over, if Kubernetes only supported creating individual Pods with no
|
||||
concept of desired state?
|
||||
3. Think of one non-Kubernetes system you've used (a thermostat, a spell-checker, an
|
||||
auto-save feature) that also works by continuously comparing "what is" to "what should be."
|
||||
What made you think of it?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**Namespaces are your first real production safety boundary, and they're free.** A
|
||||
misconfigured `kubectl delete` scoped to the wrong Namespace is a common, entirely preventable
|
||||
class of incident — teams that consistently namespace their environments (dev/staging/prod, or
|
||||
per-team) limit the blast radius of almost any operator mistake to just that Namespace. This
|
||||
costs nothing to set up and prevents a real, common category of outage.
|
||||
|
||||
**Desired state is what makes self-healing possible at all.** Every "the platform recovered on
|
||||
its own" story you'll hear about Kubernetes in production traces back to exactly the mechanism
|
||||
you just watched in the demonstration: a controller comparing actual state to desired state and
|
||||
correcting the gap. Understanding this loop is what lets you predict Kubernetes' behavior
|
||||
instead of being surprised by it.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** use all three reflection questions as discussion
|
||||
prompts. Question 3 in particular tends to produce good analogies that make the abstract
|
||||
"controller loop" concept concrete and memorable.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to name one place in their own life (not computing) where
|
||||
"declare the goal and let a system enforce it" would work better than "manually do the thing
|
||||
every time" — a genuinely reflective prompt that reinforces the mental model beyond Kubernetes
|
||||
specifically.
|
||||
+49
@@ -0,0 +1,49 @@
|
||||
# Building kubectl Fluency: Why Twenty Small Reps Beat One Big Lecture
|
||||
|
||||
## What this module is, and why it looks different from the last one
|
||||
|
||||
ROOT 05's mental model module (m00) taught you the CONCEPT of desired state and gave you your
|
||||
first `kubectl` commands. This module does something deliberately different: it's a breadth
|
||||
drill across the everyday `kubectl` vocabulary you'll type dozens of times a day as a working
|
||||
Kubernetes engineer — labels, JSONPath, dry-run manifests, patch, field selectors, sort-by,
|
||||
diff, init containers, static Pods, and more.
|
||||
|
||||
There's no single narrative thread connecting these twenty tasks, and that's intentional. Real
|
||||
`kubectl` fluency isn't one big insight — it's having enough of these small, specific commands
|
||||
already in muscle memory that you're not stopping to look up syntax while you're mid-incident.
|
||||
Each task below explains the specific concept it covers in its own description; your job is to
|
||||
work through them, not to memorize a lecture first.
|
||||
|
||||
## How to approach this module
|
||||
|
||||
- **Don't try to do them in one sitting if you don't need to.** These are independent — skip
|
||||
around, come back to ones you found awkward, and don't feel obligated to treat this as one
|
||||
continuous session.
|
||||
- **Read each task's own explanation before reaching for the hint.** Every one of these
|
||||
twenty tasks includes real context in its description — that context IS the teaching content
|
||||
for that specific command. The hint is there for when you're stuck, not as the first thing to
|
||||
read.
|
||||
- **Notice which commands you reach for confidently versus which ones you have to think about.**
|
||||
That gap is exactly what the reflection at the end of this module asks you to name.
|
||||
- **This breadth matters more than it looks like it does.** `kubectl explain`, `--dry-run`,
|
||||
JSONPath output, and `kubectl patch` in particular are commands experienced engineers use
|
||||
constantly and newer engineers often don't know exist at all — this module exists specifically
|
||||
to close that gap early.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Lesson goal:** Frame this module correctly before students hit it — it's a breadth-practice
|
||||
bank, not a single-concept lesson, and the individual task descriptions carry the teaching
|
||||
content. Without this framing, students sometimes expect a lecture and are confused when they
|
||||
land directly on twenty short tasks.
|
||||
|
||||
**Approximate timing:** 5 minutes to introduce; the module itself runs long (20 tasks) and is
|
||||
meant to be worked through at each student's own pace, live or between sessions.
|
||||
|
||||
**Facilitation for a live cohort:** Consider live-demoing 2-3 of the less obvious ones
|
||||
(`kubectl explain`, JSONPath output, `kubectl patch`) rather than all twenty — the goal is to
|
||||
unblock students on the commands they're least likely to have seen before, not to walk through
|
||||
every task.
|
||||
|
||||
**Expected takeaway:** Kubernetes fluency is built from a large vocabulary of small, reliable
|
||||
commands, not just conceptual understanding — this module is where that vocabulary gets built.
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
# Reflection: Naming Your Gaps While They're Still Cheap to Close
|
||||
|
||||
You just worked through twenty independent `kubectl` tasks. Before moving on, take a few
|
||||
minutes to consolidate what actually stuck versus what you'd need to look up again.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Which THREE of these twenty commands do you feel genuinely confident using again right now,
|
||||
without a hint? Name them specifically.
|
||||
2. Which ONE command surprised you the most — either because you didn't know it existed, or
|
||||
because it did something different than you expected?
|
||||
3. `kubectl explain`, `kubectl patch`, and JSONPath output are commands a lot of engineers never
|
||||
learn early and end up rediscovering the hard way, mid-incident, years later. Now that you've
|
||||
used all three, where do you think you'd actually reach for one of them in a real debugging
|
||||
situation?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**Fluency gaps are cheap to close now and expensive to close later.** Right now, not knowing
|
||||
`kubectl patch` costs you a few minutes rereading this module. In a live incident six months
|
||||
from now, not knowing it costs you minutes you don't have while a Service is down. Naming your
|
||||
gaps here, while the stakes are zero, is how you make sure you close them before they matter.
|
||||
|
||||
**Breadth practice is not busywork.** It's easy to dismiss twenty small tasks as less valuable
|
||||
than one big project — but the twenty small tasks are what make the big project fast to execute
|
||||
later. This is the same discipline you'll rely on for the rest of ROOT_: build the vocabulary
|
||||
now, apply it under pressure later.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 3 works well as a round-robin — go
|
||||
person to person and have each name a different command, so the group collectively surfaces
|
||||
most of the twenty rather than converging on the same one or two obvious answers.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Expected takeaway:** Students should leave this module able to name their own remaining gaps
|
||||
specifically, not just feel generally "done" with kubectl basics.
|
||||
+81
@@ -0,0 +1,81 @@
|
||||
# Why Deployments Exist & How a Rolling Update Progresses
|
||||
|
||||
## The problem a bare Pod doesn't solve
|
||||
|
||||
A Pod, on its own, has no self-healing and no concept of "there should be more than one of
|
||||
these." If a Pod crashes, nothing recreates it. If you need 3 copies for capacity, you'd have to
|
||||
create 3 separately-named Pods by hand and keep track of all of them yourself. If you need to
|
||||
update the container image, you'd have to delete and recreate every one of them manually,
|
||||
almost certainly causing a visible outage while you do it.
|
||||
|
||||
**A Deployment solves all three problems at once**: it declares how many replicas of a Pod
|
||||
template should exist, keeps that count true continuously (the exact desired-state mechanism
|
||||
from ROOT 05), and knows how to change the Pod template over time — a version upgrade — without
|
||||
just deleting everything and starting over.
|
||||
|
||||
## ReplicaSets: the layer underneath, briefly
|
||||
|
||||
You'll rarely touch a ReplicaSet directly, but understanding it clarifies what a Deployment
|
||||
actually does. A Deployment doesn't manage Pods directly — it manages a ReplicaSet, which is
|
||||
what actually keeps N Pods matching a label selector running. When you update a Deployment's
|
||||
Pod template (like changing the image), the Deployment creates a NEW ReplicaSet with the new
|
||||
template and gradually shifts replicas from the old ReplicaSet to the new one. The old
|
||||
ReplicaSet isn't deleted — it's scaled to zero and kept around, which is exactly what makes
|
||||
rollback possible (more on that below).
|
||||
|
||||
## How a rolling update actually progresses
|
||||
|
||||
By default, a Deployment updates using the `RollingUpdate` strategy: it doesn't take all old
|
||||
Pods down and bring all new ones up at once (that would cause an outage). Instead, it
|
||||
incrementally creates new-version Pods and removes old-version ones, always keeping enough
|
||||
capacity available:
|
||||
|
||||
```
|
||||
Before: [v1] [v1] [v1]
|
||||
Step 1: [v1] [v1] [v1] [v2] <- one new Pod created
|
||||
Step 2: [v1] [v1] [v2] <- one old Pod removed once new one is Ready
|
||||
Step 3: [v1] [v1] [v2] [v2]
|
||||
Step 4: [v1] [v2] [v2]
|
||||
...
|
||||
After: [v2] [v2] [v2]
|
||||
```
|
||||
|
||||
Crucially, a new Pod only counts as "up" for the purposes of proceeding once it passes its
|
||||
readiness probe (see ROOT 10's gold-standard module) — this is exactly why a misconfigured
|
||||
readiness probe can make a rollout hang indefinitely instead of failing fast: Kubernetes is
|
||||
correctly waiting for a "ready" signal that will never come.
|
||||
|
||||
## Rollback: what it actually restores
|
||||
|
||||
`kubectl rollout undo` doesn't magically know how to "undo" arbitrary changes — it works because
|
||||
of the old ReplicaSet still sitting around at zero replicas, from the mechanism described above.
|
||||
Rolling back means: scale the OLD ReplicaSet back up and the current (broken) one back down,
|
||||
using the exact same rolling mechanism as a forward update. This is why rollback is fast and
|
||||
safe — it's not a special operation, it's the same well-tested rolling-update machinery running
|
||||
in reverse, against a Pod template that's already known to have worked before.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read, 10-12 minutes live with the ReplicaSet diagram.
|
||||
|
||||
**Demo plan:** Draw (or show) the ReplicaSet-swap diagram above on a whiteboard before the
|
||||
demonstration activity — students who see the "old ReplicaSet scaled to zero, not deleted"
|
||||
detail up front connect rollback to rollout much faster.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often think rollback "restores a backup" or reverts a Git-style diff. Correct this:
|
||||
it's scaling an existing (still-present) ReplicaSet back up, nothing is being restored from
|
||||
storage.
|
||||
- "Rolling update" is sometimes assumed to mean "one Pod at a time, always." The actual pace is
|
||||
controlled by `maxSurge`/`maxUnavailable` (not covered in depth here, but worth naming so
|
||||
students don't over-generalize from the simplified diagram).
|
||||
|
||||
**Discussion question:** "Why does a rolling update need readiness probes to work correctly,
|
||||
specifically?" (Answer: without a readiness signal, Kubernetes has no way to know when a new
|
||||
Pod is actually safe to receive traffic before removing an old one — it would either guess via a
|
||||
fixed delay, unreliable, or not wait at all, unsafe.)
|
||||
|
||||
**Optional extension:** ask what would happen to an in-progress rollout if the new image simply
|
||||
doesn't exist (a typo in the tag) — a good bridge to real troubleshooting: the rollout stalls
|
||||
indefinitely with `ImagePullBackOff` on the new Pods, never completing and never rolling back on
|
||||
its own.
|
||||
+77
@@ -0,0 +1,77 @@
|
||||
# Demonstration: Watching a Rollout, Good and Bad
|
||||
|
||||
## A healthy rollout
|
||||
|
||||
A Deployment `app-v1` running `nginx:1.24` at 3 replicas gets updated to `nginx:1.25`:
|
||||
|
||||
```
|
||||
$ kubectl set image deployment/app-v1 nginx=nginx:1.25
|
||||
deployment.apps/app-v1 image updated
|
||||
|
||||
$ kubectl rollout status deployment/app-v1
|
||||
Waiting for deployment "app-v1" rollout to finish: 1 out of 3 new replicas have been updated...
|
||||
Waiting for deployment "app-v1" rollout to finish: 2 out of 3 new replicas have been updated...
|
||||
Waiting for deployment "app-v1" rollout to finish: 1 old replicas are pending termination...
|
||||
deployment "app-v1" successfully rolled out
|
||||
```
|
||||
|
||||
`kubectl rollout status` blocks and reports exactly the incremental progress described in the
|
||||
lesson — this is your primary tool for watching (or scripting a wait on) a rollout, rather than
|
||||
repeatedly polling `kubectl get pods` yourself.
|
||||
|
||||
## Checking revision history
|
||||
|
||||
```
|
||||
$ kubectl rollout history deployment/app-v1
|
||||
REVISION CHANGE-CAUSE
|
||||
1 <none>
|
||||
2 <none>
|
||||
```
|
||||
|
||||
Revision 1 is the original ReplicaSet (now scaled to zero, still present); revision 2 is the
|
||||
current one. This history is exactly what `kubectl rollout undo` uses.
|
||||
|
||||
## A stalled rollout
|
||||
|
||||
Now imagine the same update, but to a typo'd tag, `nginx:1.25-typo`, which doesn't exist:
|
||||
|
||||
```
|
||||
$ kubectl rollout status deployment/app-v1
|
||||
Waiting for deployment "app-v1" rollout to finish: 1 out of 3 new replicas have been updated...
|
||||
```
|
||||
|
||||
...and it just sits there. `kubectl get pods` shows why:
|
||||
|
||||
```
|
||||
$ kubectl get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
app-v1-7d8f9c6b5d-abcde 1/1 Running 0 5m
|
||||
app-v1-7d8f9c6b5d-fghij 1/1 Running 0 5m
|
||||
app-v1-9c5b7d8f6b-klmno 0/1 ImagePullBackOff 0 30s
|
||||
```
|
||||
|
||||
The new Pod can never become Ready because its image can't even be pulled — so the rollout
|
||||
correctly refuses to proceed further (removing more old Pods would reduce capacity for no
|
||||
benefit, since the replacement will never come up). Nothing is "broken" in the platform; the
|
||||
platform is behaving exactly correctly given bad input.
|
||||
|
||||
**The fix here is `kubectl rollout undo`, not waiting longer** — this exact scenario is what
|
||||
`deployment-rollback`'s Lab, later in ROOT_, gives you real hands-on practice with.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** If time allows, run the healthy rollout live against a throwaway Deployment,
|
||||
then intentionally break the second one with a bad tag so students see `ImagePullBackOff` occur
|
||||
in real time rather than only in static text.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; 5-7 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students sometimes assume a stalled rollout will
|
||||
eventually time out and auto-rollback on its own. By default it will not — a Deployment has no
|
||||
automatic rollback trigger; a human (or automation) has to run `kubectl rollout undo` (or fix
|
||||
the underlying issue) explicitly. This is worth stating directly since it's a genuine and
|
||||
consequential production gotcha.
|
||||
|
||||
**Optional extension:** ask what `kubectl rollout status --timeout=2m` would change about this
|
||||
scenario (answer: the command itself would exit with an error after 2 minutes instead of
|
||||
blocking forever — useful in CI/CD pipelines that need to fail a deploy step rather than hang).
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
# Knowledge Check: Deployments & Rollouts
|
||||
|
||||
---
|
||||
|
||||
**1. A rolling update to a new image version has been "in progress" for 20 minutes with no
|
||||
error message, and `kubectl rollout status` never returns. What's the first thing you should
|
||||
check, and why?**
|
||||
|
||||
*Answer:* `kubectl get pods` to see the new Pod's status, then `kubectl describe pod` on it if
|
||||
it isn't `Running`. A stalled rollout with no explicit error almost always means a new Pod can't
|
||||
become Ready — commonly `ImagePullBackOff` (bad image/tag) or a failing readiness probe. The
|
||||
rollout isn't broken; it's correctly refusing to proceed until the new Pod proves healthy.
|
||||
|
||||
---
|
||||
|
||||
**2. True or false: `kubectl rollout undo` restores your application from a backup of its
|
||||
previous state.**
|
||||
|
||||
*Answer:* False. It scales the previous (still-present, just scaled-to-zero) ReplicaSet back up
|
||||
and the current one back down — the same rolling mechanism as a forward update, just pointed at
|
||||
an older, already-proven-working Pod template. Nothing is being restored from storage.
|
||||
|
||||
---
|
||||
|
||||
**3. A Deployment has `replicas: 3`. You need it to handle more traffic. Which is the correct
|
||||
action: edit the Pod template's resource limits, or change the replica count? Justify your
|
||||
answer for a stateless web application.**
|
||||
|
||||
*Answer:* Change the replica count (scale out) for a stateless application handling more
|
||||
concurrent traffic — more identical copies sharing the load. Changing resource limits affects
|
||||
how much each individual Pod can use, not how many Pods exist; it doesn't add capacity for more
|
||||
concurrent requests the way scaling does.
|
||||
|
||||
---
|
||||
|
||||
**4. Why does a Deployment keep an old, scaled-to-zero ReplicaSet around instead of deleting it
|
||||
immediately after a successful rollout?**
|
||||
|
||||
*Answer:* That scaled-down ReplicaSet is exactly what makes fast, safe rollback possible later —
|
||||
deleting it immediately would mean a "rollback" would have to rebuild the old Pod template from
|
||||
scratch (or fail entirely) instead of simply scaling existing, already-defined Pods back up.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 1 is the most important one to discuss out loud — it's the exact
|
||||
diagnostic instinct students will need repeatedly in ROOT 10.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+54
@@ -0,0 +1,54 @@
|
||||
# Guided Exercise: Planning a Safe Rollout Strategy
|
||||
|
||||
Paper-and-thinking exercise — work through this before the Lab.
|
||||
|
||||
## The scenario
|
||||
|
||||
Your team runs `api-server`, a Deployment with 6 replicas serving live traffic. You need to
|
||||
ship a new image version. The team lead asks: "what could go wrong, and how would we know
|
||||
before it takes down the whole service?"
|
||||
|
||||
## Your task
|
||||
|
||||
Work through these questions:
|
||||
|
||||
1. If the new image has a bug that crashes the container immediately on startup, at what point
|
||||
in the rolling update would you expect to notice — after 1 new Pod, after all 6, or somewhere
|
||||
in between? Why?
|
||||
2. What's the difference in blast radius between that crash-on-startup bug and a subtler bug
|
||||
that only fails under load, several minutes after a Pod starts serving traffic?
|
||||
3. Given your answer to question 2, what's the practical value of watching `kubectl rollout
|
||||
status` and `kubectl get pods` DURING a rollout, rather than only checking after it reports
|
||||
"successfully rolled out"?
|
||||
4. If you noticed a problem partway through, what's the single command that gets you back to
|
||||
the last known-good state fastest?
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. After the FIRST new Pod — a crash-on-startup bug means the new Pod never becomes Ready
|
||||
(`CrashLoopBackOff`), so the rollout stalls immediately, well before more old Pods are
|
||||
removed. This is actually the SAFEST kind of bug, because the blast radius stays tiny.
|
||||
2. A crash-on-startup bug is naturally self-limiting (the rollout stalls itself, as above). A
|
||||
bug that only appears under load is NOT self-limiting the same way — by the time it's
|
||||
noticeable, several old (working) replicas may already have been replaced, meaning a larger
|
||||
fraction of real traffic is already hitting the broken version.
|
||||
3. Watching during the rollout, not just checking the final "successfully rolled out" message,
|
||||
is what lets you catch the load-dependent case in question 2 early — waiting for the final
|
||||
status alone would let the rollout complete entirely (100% on the broken version) before you
|
||||
noticed anything wrong.
|
||||
4. `kubectl rollout undo deployment/api-server` — assuming the previous ReplicaSet is still
|
||||
present (the default), this is faster than trying to hand-diagnose and hot-fix the new
|
||||
version under pressure.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2's distinction (self-limiting vs. not) is the core insight worth
|
||||
making sure lands — it's why "the rollout said success" is necessary but not sufficient
|
||||
evidence that a deploy actually went well.
|
||||
|
||||
**Approximate timing:** 10-15 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what monitoring/observability signal (beyond `kubectl` output) a
|
||||
real team would want during a rollout to catch the load-dependent case even faster (a good
|
||||
forward pointer to ROOT 10's observability themes, without needing to build any actual
|
||||
monitoring infrastructure here).
|
||||
+39
@@ -0,0 +1,39 @@
|
||||
# Launch Lab: Perform a Rolling Update
|
||||
|
||||
Time to run a real rolling update against a real Deployment in your disposable Lab.
|
||||
|
||||
## What you're doing
|
||||
|
||||
A Deployment `app-v1` is running an older image version. Your task is to update it to a newer
|
||||
version and confirm every replica ends up running the new image — exactly the mechanism from
|
||||
the lesson and demonstration, on a real cluster this time.
|
||||
|
||||
## Before you start
|
||||
|
||||
Recall the demonstration: `kubectl rollout status` is your tool for watching progress rather
|
||||
than repeatedly polling `kubectl get pods` yourself, and a stalled rollout with no obvious error
|
||||
usually means a new Pod can't become Ready. If your update seems to hang, that's exactly the
|
||||
diagnostic instinct from the knowledge check to apply.
|
||||
|
||||
## Your task in this Lab
|
||||
|
||||
Update the Deployment's image and verify all replicas are running the new version. Use
|
||||
`kubectl rollout status` to confirm completion, not just a single `kubectl get pods` snapshot.
|
||||
|
||||
This Lab has its own in-context hints if you get stuck.
|
||||
|
||||
When you're done, return to this module for a short reflection on what "successfully rolled
|
||||
out" actually guarantees — and doesn't.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Classroom flow:** A good moment to have students keep `kubectl get pods --watch` open in a
|
||||
second terminal (if their Lab environment supports multiple panes) so they see the incremental
|
||||
old-Pod/new-Pod transition happen live, reinforcing the demonstration.
|
||||
|
||||
**Approximate timing:** 10-15 minutes.
|
||||
|
||||
**Common mistake to watch for:** running `kubectl edit deployment` and forgetting to save/apply
|
||||
correctly, or editing the wrong field (e.g. `replicas` instead of the container image) — a good
|
||||
moment to reinforce reading the task description precisely rather than pattern-matching to a
|
||||
similar-looking previous Lab.
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
# Reflection: Rollback Is a Plan, Not a Panic Button
|
||||
|
||||
You just completed a real rolling update. Take a few minutes to connect it to how this plays out
|
||||
in a real production team.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. "Successfully rolled out" (from `kubectl rollout status`) tells you every replica is Running
|
||||
and Ready. What does it NOT tell you about whether the new version is actually correct?
|
||||
2. If you were on-call and a rollout you just ran seemed to be causing problems, would you reach
|
||||
for `kubectl rollout undo` immediately or try to hot-fix the new version first? What would
|
||||
change your answer?
|
||||
3. Think back to the guided exercise's crash-on-startup vs. load-dependent distinction. Which
|
||||
one would a typical automated health check (like a readiness probe, ROOT 10) catch, and
|
||||
which one might slip through even with probes configured correctly?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**A rollout completing is not the same as a deploy succeeding.** "Every Pod is Running and
|
||||
Ready" only proves the new code started up and passed whatever checks are configured — it says
|
||||
nothing about whether the new version is functionally correct under real traffic. Real teams
|
||||
pair a rollout with actual monitoring (error rates, latency) before calling a deploy genuinely
|
||||
successful, not just a green `kubectl rollout status`.
|
||||
|
||||
**Rollback should be a rehearsed, low-drama action, not a last resort.** Teams that treat
|
||||
`kubectl rollout undo` as a normal, expected tool — something you reach for calmly the moment
|
||||
something looks wrong — recover from bad deploys in seconds. Teams that treat rollback as an
|
||||
admission of failure hesitate, try to hot-fix forward under pressure, and turn a small,
|
||||
recoverable mistake into a longer outage.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 2 is worth pushing on — many students'
|
||||
instinct is to try to fix forward, and it's worth surfacing WHY that instinct is often wrong
|
||||
under time pressure (unknown time-to-fix vs. known, fast rollback).
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to write the one-sentence Slack message they'd post to
|
||||
their team the moment they decide to roll back a bad deploy — reinforces that rollback is a
|
||||
normal, communicable operational action, not a private failure.
|
||||
+85
@@ -0,0 +1,85 @@
|
||||
# Jobs, CronJobs, and DaemonSets: The Other Workload Shapes
|
||||
|
||||
Everything in the previous module assumed a long-running application that should always have N
|
||||
replicas up. Not every workload looks like that. Kubernetes has three other primitives for
|
||||
genuinely different shapes of work — using a Deployment for any of these would be the wrong
|
||||
tool, not just a less elegant one.
|
||||
|
||||
## Job: run to completion, exactly once (or a fixed number of times)
|
||||
|
||||
A Job creates one or more Pods and considers itself done when they exit successfully — it does
|
||||
NOT restart a Pod that exits with success, and it DOES retry one that fails, up to a configured
|
||||
limit. This is fundamentally different from a Deployment, which expects its Pods to run forever
|
||||
and treats any exit as something to replace.
|
||||
|
||||
Use a Job for: a database migration, a batch data-processing task, a one-off cleanup script —
|
||||
anything with a defined "finished" state.
|
||||
|
||||
```yaml
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: migrate-db
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: migrate
|
||||
image: myapp:latest
|
||||
command: ["./migrate.sh"]
|
||||
restartPolicy: Never
|
||||
backoffLimit: 3
|
||||
```
|
||||
|
||||
## CronJob: a Job, on a schedule
|
||||
|
||||
A CronJob doesn't run Pods itself — it creates a new Job on a cron schedule (`schedule:
|
||||
"0 2 * * *"` for "every day at 2am"), and that Job behaves exactly as described above. Use a
|
||||
CronJob for anything that should run periodically and finish: nightly backups, scheduled
|
||||
reports, periodic cleanup.
|
||||
|
||||
A CronJob that reliably fails to complete (an image that always crashes, for example) will keep
|
||||
creating new Jobs on schedule, each failing the same way — this is a real, common way to
|
||||
silently exhaust cluster resources (job/pod history accumulating) if nobody's watching.
|
||||
|
||||
## DaemonSet: exactly one Pod per node, automatically
|
||||
|
||||
A DaemonSet doesn't declare a replica count at all — it ensures exactly one matching Pod runs
|
||||
on every node in the cluster (or every node matching a selector), and automatically adds a Pod
|
||||
when a new node joins. Use a DaemonSet for per-node infrastructure: a log-shipping agent, a
|
||||
node-level monitoring exporter, a network plugin component — things that fundamentally need to
|
||||
exist once per machine, not "N copies somewhere."
|
||||
|
||||
## The decision in one pass
|
||||
|
||||
| You need... | Use |
|
||||
|---|---|
|
||||
| Always-on, N identical copies, restarted forever | Deployment |
|
||||
| Run once (or a few times) until it finishes successfully | Job |
|
||||
| The above, but on a recurring schedule | CronJob |
|
||||
| Exactly one copy on every node | DaemonSet |
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live.
|
||||
|
||||
**Demo plan (if extending this PARTIAL module later):** live-create a Job with a command that
|
||||
deliberately fails once then succeeds (`backoffLimit: 2`), and watch `kubectl get pods` show
|
||||
the failed attempt alongside the eventual successful one — makes the retry-until-backoffLimit
|
||||
behavior concrete.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often assume a Job that finishes successfully gets cleaned up automatically. It
|
||||
doesn't by default — the completed Pod/Job objects remain until manually deleted or a
|
||||
`ttlSecondsAfterFinished` is configured (not covered here, worth a one-line mention if asked).
|
||||
- "DaemonSet" is sometimes confused with "one Pod, cluster-wide" (singular). It's the opposite —
|
||||
one Pod PER NODE, which can be many Pods total in a multi-node cluster.
|
||||
|
||||
**Discussion question:** "Why doesn't a Deployment support 'run once and be done'?" (Answer: a
|
||||
Deployment's whole contract is maintaining a standing replica count forever — repurposing it for
|
||||
one-shot work would fight that contract rather than use a primitive designed for it.)
|
||||
|
||||
**Optional extension:** ask what would go wrong if a CronJob's schedule ran more often than a
|
||||
single Job execution takes to complete (answer: overlapping Job runs, unless
|
||||
`concurrencyPolicy` is set to `Forbid` or `Replace` — a real gotcha worth naming even without
|
||||
building a full lab around it).
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
# Knowledge Check: Choosing the Right Workload Primitive
|
||||
|
||||
---
|
||||
|
||||
**1. You need to run a database migration script exactly once when a new version deploys. Which
|
||||
primitive fits, and why not a Deployment?**
|
||||
|
||||
*Answer:* A Job. A Deployment expects its Pods to run forever and replaces any that exit; a
|
||||
migration script is supposed to exit successfully when done, which a Deployment would treat as
|
||||
a failure to "fix" by restarting it — the wrong behavior for a task that's supposed to finish.
|
||||
|
||||
---
|
||||
|
||||
**2. A CronJob scheduled for "every 5 minutes" keeps creating Jobs that fail immediately due to
|
||||
a bad image tag. What actually happens over the next hour if nobody notices?**
|
||||
|
||||
*Answer:* A new failing Job (and its Pod) gets created roughly every 5 minutes, accumulating
|
||||
failed Job/Pod objects and consuming cluster resources — nothing stops this automatically. This
|
||||
is exactly the kind of slow, easy-to-miss resource exhaustion a real on-call engineer should
|
||||
watch for.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: a DaemonSet's Pod count scales with a `replicas` field, the same as a
|
||||
Deployment.**
|
||||
|
||||
*Answer:* False. A DaemonSet has no `replicas` field — its Pod count is implicitly "one per
|
||||
matching node," and it grows or shrinks automatically as nodes join or leave the cluster.
|
||||
|
||||
---
|
||||
|
||||
**4. You need a log-shipping agent that must run on every node, including any new node added
|
||||
later. Which primitive, and what would go wrong if you used a Deployment with a high replica
|
||||
count instead?**
|
||||
|
||||
*Answer:* A DaemonSet. A Deployment's replicas are placed by the scheduler wherever it decides
|
||||
is best — nothing guarantees one per node, and a newly-added node wouldn't automatically get a
|
||||
copy the way a DaemonSet guarantees.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2 is worth emphasizing — it's a genuinely realistic "how does a
|
||||
cluster quietly get into trouble" scenario, not just trivia.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+72
@@ -0,0 +1,72 @@
|
||||
# Demonstration: Watching a Job Actually Complete
|
||||
|
||||
## Creating the Job
|
||||
|
||||
```
|
||||
$ kubectl apply -f pi-job.yaml
|
||||
job.batch/pi-job created
|
||||
|
||||
$ kubectl get jobs
|
||||
NAME COMPLETIONS DURATION AGE
|
||||
pi-job 0/1 3s 3s
|
||||
|
||||
$ kubectl get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
pi-job-8x2kq 1/1 Running 0 3s
|
||||
```
|
||||
|
||||
The Job's own status (`0/1` completions) is tracked separately from its Pod's status — this
|
||||
matters because a Job can retry a FAILED Pod attempt without that looking like the healthy
|
||||
restart behavior you'd see on a Deployment.
|
||||
|
||||
## Once the Pod's process actually exits successfully
|
||||
|
||||
```
|
||||
$ kubectl get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
pi-job-8x2kq 0/1 Completed 0 8s
|
||||
|
||||
$ kubectl get jobs
|
||||
NAME COMPLETIONS DURATION AGE
|
||||
pi-job 1/1 7s 8s
|
||||
```
|
||||
|
||||
`STATUS: Completed` — not `Running`, not `CrashLoopBackOff`. This is the ONE status a
|
||||
long-running Deployment's Pod is never supposed to reach on its own; for a Job, it's success.
|
||||
The Job object itself now reports `1/1` and stops — nothing recreates this Pod, because
|
||||
"finished successfully" is the desired end state here, unlike a Deployment where exiting is
|
||||
always treated as something to fix.
|
||||
|
||||
## What a FAILING Job attempt looks like, briefly
|
||||
|
||||
```
|
||||
$ kubectl get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
bad-job-abc12 0/1 Error 0 5s
|
||||
bad-job-def34 0/1 Error 0 2s
|
||||
|
||||
$ kubectl describe job bad-job
|
||||
...
|
||||
Events:
|
||||
Warning BackoffLimitExceeded Job has reached the specified backoff limit
|
||||
```
|
||||
|
||||
Two DIFFERENT Pods (`abc12`, `def34`), not one Pod restarting twice — each failed attempt gets
|
||||
its own new Pod, up to `backoffLimit`. Once that limit is hit, the Job stops retrying and
|
||||
reports the failure via Events rather than trying forever.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Run the successful case live; showing the FAILING case is optional if time is
|
||||
short, but the `Completed` vs `Running`/`Error` distinction is worth pausing on regardless.
|
||||
|
||||
**Approximate timing:** 8 minutes live; 5 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students who've only seen Deployments sometimes expect
|
||||
`kubectl get pods` to show the Job's Pod disappear once it's done. It doesn't, by default — it
|
||||
stays as `Completed` until manually cleaned up or a TTL is configured, which is worth mentioning
|
||||
briefly (not a hands-on topic here).
|
||||
|
||||
**Optional extension:** ask what `kubectl get pods` would show for a CronJob's Pods across
|
||||
several scheduled runs (answer: one Completed Pod per past run, accumulating — the same
|
||||
"nothing auto-cleans this up" behavior, multiplied by the schedule).
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
# Guided Exercise: Choosing the Right Workload Primitive
|
||||
|
||||
Paper-and-thinking exercise before the Labs.
|
||||
|
||||
## The scenario
|
||||
|
||||
Your team needs four things running in the cluster:
|
||||
|
||||
1. A nightly database backup that must run at 2am and finish before morning.
|
||||
2. A log-shipping agent that must run on every node, including nodes added later.
|
||||
3. A one-time script to backfill a new database column, run once during this deploy.
|
||||
4. The main web application, always up, serving live traffic.
|
||||
|
||||
## Your task
|
||||
|
||||
For each of the four, name the correct primitive (Deployment, Job, CronJob, or DaemonSet) and
|
||||
answer: what would go wrong if you used a Deployment for it instead?
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. **CronJob.** A Deployment has no concept of "run on a schedule" — you'd need external
|
||||
tooling to trigger it repeatedly, reinventing what CronJob already does. Using a bare Job
|
||||
would run it once; CronJob is what recreates that Job on schedule.
|
||||
2. **DaemonSet.** A Deployment's replica count is a number, not "one per node" — nothing
|
||||
guarantees even coverage, and a newly added node wouldn't automatically get a copy the way a
|
||||
DaemonSet guarantees.
|
||||
3. **Job.** A Deployment expects its Pod to run forever; a script that's SUPPOSED to exit when
|
||||
done would be treated as a crash and restarted forever, which is wrong for a one-time task.
|
||||
4. **Deployment.** This is the actual always-on, N-replicas case the other three are explicitly
|
||||
NOT designed for.
|
||||
|
||||
## Self-check
|
||||
|
||||
If you answered "Job" for #4 or "Deployment" for #1-3, revisit the lesson's decision table
|
||||
before the Labs — getting this choice right the first time avoids building something that
|
||||
technically runs but behaves wrong in a way that's easy to miss until it causes a real problem
|
||||
(a backup CronJob accidentally deployed as a Job that only ever runs once, for example).
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Good as a quick individual exercise before revealing the answer — the "what
|
||||
would go wrong" framing forces reasoning about the CONSEQUENCE of a wrong choice, not just
|
||||
recall of a definition.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live with discussion; 8 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what would need to be true for #1 (the nightly backup) to actually
|
||||
be SAFE to run as a plain Job triggered manually once, instead of a CronJob — a good check that
|
||||
students understand CronJob's value is specifically the SCHEDULING, not something inherent to
|
||||
backups.
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
# Reflection: The Right Primitive Is a Correctness Decision, Not a Style Choice
|
||||
|
||||
You just built a one-shot Job and a scheduled CronJob for real. Take a few minutes to connect
|
||||
this to production thinking.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. If your team accidentally deployed the database backup script as a Deployment instead of a
|
||||
CronJob, what's the FIRST symptom you'd likely notice — and would it look like a bug in the
|
||||
backup script itself, or something else?
|
||||
2. A CronJob that fails silently every night for a week is a real, common incident pattern. What
|
||||
would you want monitored to catch this quickly?
|
||||
3. Why does "it ran once and worked" not prove a CronJob is configured correctly?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**Workload-primitive mistakes often look like application bugs at first.** A backup script
|
||||
"crashing" every few minutes on a Deployment isn't a bug in the script — the script did exactly
|
||||
what it was told to do (exit when finished) and the Deployment is "correctly" restarting it
|
||||
according to its own contract. Recognizing "wrong primitive," not "buggy code," as the root
|
||||
cause saves real debugging time.
|
||||
|
||||
**A schedule you never verify is a schedule you can't trust.** A CronJob failing every single
|
||||
run since deployment is easy to miss if nobody checks — unlike a Deployment's crash-looping
|
||||
Pods, which are visible in `kubectl get pods` by default, a CronJob's failed history requires
|
||||
someone to actually look at `kubectl get jobs` or the CronJob's own status. Real teams monitor
|
||||
CronJob success/failure explicitly rather than assuming silence means success.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 1 is worth pushing on — many students'
|
||||
first instinct is to debug the "crashing" script rather than question the workload type it's
|
||||
running under.
|
||||
|
||||
**Approximate timing:** 8 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to name one alerting rule they'd add for a critical nightly
|
||||
CronJob (e.g. "alert if no successful completion recorded in the last 25 hours").
|
||||
+84
@@ -0,0 +1,84 @@
|
||||
# Externalizing Configuration: ConfigMaps & Secrets
|
||||
|
||||
## The problem: hardcoded configuration doesn't scale
|
||||
|
||||
If your application's database URL, log level, or feature flags are baked directly into its
|
||||
container image, changing any of them means rebuilding and redeploying the image — for a
|
||||
one-line config change. Worse, the SAME image can't be reused across dev/staging/production if
|
||||
each environment needs different settings compiled in.
|
||||
|
||||
Kubernetes solves this by letting you inject configuration into a container at RUNTIME,
|
||||
separately from the image itself — the same image, different config per environment.
|
||||
|
||||
## ConfigMap: configuration that isn't secret
|
||||
|
||||
A **ConfigMap** holds arbitrary key/value configuration data — log levels, feature flags, URLs,
|
||||
whole config files. It's a plain Kubernetes object, readable by anyone with API access to that
|
||||
Namespace, so it's for configuration that ISN'T sensitive.
|
||||
|
||||
Two ways to consume a ConfigMap in a Pod:
|
||||
|
||||
- **As environment variables** — each key becomes an env var the application reads normally.
|
||||
- **As a mounted volume** — each key becomes a FILE inside the container, useful for
|
||||
applications that expect to read configuration from disk rather than environment variables.
|
||||
|
||||
```yaml
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: app-env
|
||||
```
|
||||
|
||||
## Secret: the same idea, for sensitive data
|
||||
|
||||
A **Secret** has the identical shape and consumption pattern as a ConfigMap (env vars or a
|
||||
mounted volume) but is meant for sensitive values — passwords, API keys, tokens. Kubernetes
|
||||
stores Secret data base64-ENCODED, not encrypted — this is an important distinction:
|
||||
base64 is trivially reversible, so a Secret's real protection comes from Kubernetes' RBAC
|
||||
(who can even read Secret objects at all, ROOT 09) and, in a properly configured cluster,
|
||||
encryption at rest — not from the encoding itself.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: app-secret
|
||||
type: Opaque
|
||||
data:
|
||||
api-key: c3VwZXJzZWNyZXQ= # base64 of "supersecret" — NOT encrypted
|
||||
```
|
||||
|
||||
## Why the same mechanism, two different objects
|
||||
|
||||
You might ask why Kubernetes doesn't just have one "Config" object with a "sensitive" flag.
|
||||
Keeping them separate makes the DISTINCTION visible and enforceable — RBAC rules can grant
|
||||
"read ConfigMaps" without granting "read Secrets," and a quick `kubectl get configmaps` never
|
||||
accidentally exposes something that should have been a Secret. The separation is a deliberate
|
||||
safety boundary, not redundant API surface.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live.
|
||||
|
||||
**Demo plan:** Before the demonstration activity, it's worth showing `kubectl get secret
|
||||
app-secret -o yaml` live and pointing out the base64 value, then decoding it on the spot
|
||||
(`echo <value> | base64 -d`) — the "this isn't actually encrypted" realization lands much better
|
||||
seen than described.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students very commonly assume Secrets are encrypted by Kubernetes automatically. Correct this
|
||||
explicitly and early — it's a genuine, consequential security misunderstanding.
|
||||
- The env-var vs. volume-mount choice is sometimes seen as arbitrary. Note one real
|
||||
consideration: env vars are visible via `kubectl describe pod` and process inspection tools
|
||||
inside the container more easily than a mounted file's contents — not a hard rule, but a
|
||||
reason some teams prefer volume mounts specifically for Secrets.
|
||||
|
||||
**Discussion question:** "If someone with `kubectl get secrets` access on your namespace can
|
||||
trivially read every Secret's real value, what is a Secret actually protecting you from?"
|
||||
(Answer: it protects against accidental exposure through normal channels like
|
||||
`kubectl get configmaps`, source control, or logs — not against a determined actor with direct
|
||||
API read access, which is exactly why RBAC restricting who has that access is the real control.)
|
||||
|
||||
**Optional extension:** ask what changes about this picture on a cluster with encryption-at-rest
|
||||
enabled for Secrets (answer: the ETCD-stored data itself becomes encrypted, protecting against
|
||||
someone who gains access to the underlying etcd storage directly — still separate from and in
|
||||
addition to RBAC controlling API-level access).
|
||||
+66
@@ -0,0 +1,66 @@
|
||||
# Demonstration: A Secret Is Encoded, Not Encrypted
|
||||
|
||||
## Creating a Secret
|
||||
|
||||
```
|
||||
$ kubectl create secret generic app-secret --from-literal=api-key=supersecret
|
||||
secret/app-secret created
|
||||
```
|
||||
|
||||
## Looking at it
|
||||
|
||||
```
|
||||
$ kubectl get secret app-secret -o yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: app-secret
|
||||
data:
|
||||
api-key: c3VwZXJzZWNyZXQ=
|
||||
type: Opaque
|
||||
```
|
||||
|
||||
## Decoding it — no special permission needed beyond reading the Secret itself
|
||||
|
||||
```
|
||||
$ echo "c3VwZXJzZWNyZXQ=" | base64 -d
|
||||
supersecret
|
||||
```
|
||||
|
||||
Anyone who can run `kubectl get secret app-secret -o yaml` (i.e. anyone RBAC allows to READ
|
||||
Secrets in this Namespace) can trivially recover the real value with a standard, built-in
|
||||
command — no cracking, no special tooling. This is exactly why the lesson's point matters: the
|
||||
protection here is WHO can read the Secret object at all (RBAC), not the base64 encoding itself.
|
||||
|
||||
## Comparing to a ConfigMap
|
||||
|
||||
```
|
||||
$ kubectl get configmap app-config -o yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: app-config
|
||||
data:
|
||||
environment: production
|
||||
log_level: info
|
||||
```
|
||||
|
||||
A ConfigMap's values are stored in PLAIN TEXT — no encoding step at all. This is consistent with
|
||||
ConfigMaps being explicitly for non-sensitive data; there's no pretense of protection to begin
|
||||
with.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Run this live, actually decoding the value in front of the class — the gap
|
||||
between "Secret" (a name implying protection) and what the command output shows is the entire
|
||||
point, and it lands far better performed than read.
|
||||
|
||||
**Approximate timing:** 6-8 minutes live; 5 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** this demonstration directly targets the single most common
|
||||
misconception about Kubernetes Secrets — do not skip it even if time is short, since a wrong
|
||||
belief here has real security consequences later.
|
||||
|
||||
**Optional extension:** ask what a `SealedSecret` or an external secrets manager (mentioned by
|
||||
name only, not built as a hands-on topic here) is trying to solve that a plain Kubernetes Secret
|
||||
does not — a good forward pointer without requiring new infrastructure in this course.
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
# Knowledge Check: Configuration & Secrets
|
||||
|
||||
---
|
||||
|
||||
**1. A teammate says "our database password is safe because it's stored as a Kubernetes
|
||||
Secret." What's missing from that statement?**
|
||||
|
||||
*Answer:* A Secret's data is base64-ENCODED, not encrypted — anyone who can read the Secret
|
||||
object at all can trivially decode it. The real protection is RBAC (who is allowed to read
|
||||
Secrets in that namespace) and, on a properly configured cluster, encryption at rest — not the
|
||||
Secret object type itself.
|
||||
|
||||
---
|
||||
|
||||
**2. Why does Kubernetes have two separate objects (ConfigMap and Secret) instead of one
|
||||
"Config" object with a sensitivity flag?**
|
||||
|
||||
*Answer:* Keeping them separate makes the distinction enforceable via RBAC — a role can grant
|
||||
"read ConfigMaps" without granting "read Secrets" — and makes it visible at a glance
|
||||
(`kubectl get configmaps` never risks accidentally listing something sensitive).
|
||||
|
||||
---
|
||||
|
||||
**3. An application needs its configuration to change without rebuilding its container image
|
||||
across dev/staging/production. Which Kubernetes mechanism solves this, and why not just bake
|
||||
different configs into three separate images?**
|
||||
|
||||
*Answer:* ConfigMaps (and Secrets, for sensitive values) injected at runtime via env vars or a
|
||||
mounted volume. Baking config into the image means maintaining three separate images for what
|
||||
should be identical application code — externalizing configuration lets the SAME image run
|
||||
correctly in every environment.
|
||||
|
||||
---
|
||||
|
||||
**4. True or false: mounting a ConfigMap as a volume and consuming it as environment variables
|
||||
are functionally identical from the application's perspective.**
|
||||
|
||||
*Answer:* False. An application reading environment variables needs code that reads env vars; an
|
||||
application expecting a config FILE on disk needs the volume-mount approach. The right choice
|
||||
depends on how the application itself is written to read configuration, not just personal
|
||||
preference.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 1 is the single most important takeaway from this module — confirm
|
||||
it's genuinely understood, not just correctly guessed, before moving to the Labs.
|
||||
|
||||
**Approximate timing:** 5-7 minutes.
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
# Guided Exercise: ConfigMap, Secret, Env Var, or Volume?
|
||||
|
||||
Paper-and-thinking exercise before the Labs.
|
||||
|
||||
## The scenario
|
||||
|
||||
Your application needs four pieces of configuration:
|
||||
|
||||
1. A log level (`debug`/`info`/`warn`) that ops sometimes needs to flip without redeploying.
|
||||
2. A database password.
|
||||
3. A large JSON config file the application reads from a specific path at startup.
|
||||
4. A feature flag read once at container startup via an environment variable, per the
|
||||
application's existing code.
|
||||
|
||||
## Your task
|
||||
|
||||
For each, decide: ConfigMap or Secret, and env var or volume mount — then justify each choice.
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. **ConfigMap, env var.** Not sensitive; the application likely already reads it as an env var
|
||||
(a common convention), and env vars are simple to change and trigger a rolling restart with.
|
||||
2. **Secret, either — but env var is common here too**, since most applications expect
|
||||
credentials via environment variables by convention. The KEY decision is Secret (sensitive),
|
||||
not the delivery mechanism.
|
||||
3. **ConfigMap, volume mount.** The application expects a FILE at a path, not an environment
|
||||
variable — volume mount is the only mechanism that presents a ConfigMap as an actual file.
|
||||
4. **ConfigMap (assuming not sensitive) or Secret (if it is), env var** — the application's
|
||||
EXISTING code dictates this; you don't get to choose a mechanism the application wasn't
|
||||
written to read.
|
||||
|
||||
## Self-check
|
||||
|
||||
If you chose "volume mount" for #4 or "env var" for #3, revisit the lesson — the deciding
|
||||
factor here isn't which mechanism seems more "correct" in the abstract, it's what the
|
||||
application actually expects. Getting this backwards means the application simply won't find
|
||||
its configuration at all.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Emphasize that #4's answer depends entirely on EXISTING application
|
||||
behavior — a good moment to note that platform engineers frequently have to adapt Kubernetes
|
||||
config delivery to match an application's assumptions, not the reverse.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live with discussion; 8 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what would happen if a ConfigMap consumed as a volume mount is
|
||||
UPDATED after the Pod is already running — a good bridge to the real (if not hands-on-tested
|
||||
here) behavior that mounted ConfigMap files update automatically after a short delay, while
|
||||
env-var-injected values do NOT update without a Pod restart.
|
||||
+39
@@ -0,0 +1,39 @@
|
||||
# Reflection: Configuration Is Infrastructure, Secrets Are a Trust Boundary
|
||||
|
||||
You just externalized configuration and handled a real Secret. Take a few minutes to connect
|
||||
this to production practice.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Before this module, would you have assumed a Kubernetes Secret was encrypted? What's the
|
||||
real-world consequence of a team believing that when it isn't true?
|
||||
2. If your team's RBAC accidentally granted a low-trust role `get secrets` access across an
|
||||
entire namespace, what's actually exposed — and would your team necessarily notice?
|
||||
3. Why does separating configuration from the container image matter beyond convenience — think
|
||||
about what it enables for testing or incident response.
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**A misunderstood Secret is a false sense of security.** Teams that believe Secrets are
|
||||
encrypted by default sometimes skip additional controls (proper RBAC scoping, external secrets
|
||||
management, encryption at rest) that they'd otherwise recognize as necessary. The gap between
|
||||
perceived and actual protection is exactly where real incidents happen.
|
||||
|
||||
**Externalized configuration is what makes rollback and incident response fast.** If a bad
|
||||
config value ships (a wrong feature flag, an incorrect log level flooding storage), fixing it
|
||||
means updating a ConfigMap/Secret and letting Pods pick up the change — not rebuilding and
|
||||
redeploying an entire image. This is a direct, practical payoff of the separation this module
|
||||
taught, not just an architectural nicety.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 2 works well as a small-group exercise —
|
||||
have students actually list what a namespace's Secrets typically contain in a real application
|
||||
(DB creds, API keys, TLS certs) to make the exposure concrete.
|
||||
|
||||
**Approximate timing:** 8 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to draft a one-line policy statement their hypothetical
|
||||
team would adopt about who is allowed `get`/`list` access to Secrets in production namespaces.
|
||||
+87
@@ -0,0 +1,87 @@
|
||||
# The Service/Endpoints/DNS Request Path
|
||||
|
||||
## The problem: Pod IPs are not stable
|
||||
|
||||
Every Pod gets its own IP address, but that IP is NOT durable — when a Pod is replaced (a
|
||||
rollout, a crash recovery, a reschedule), the new Pod gets a brand new IP. If another part of
|
||||
your application had that old IP hardcoded, it would silently start talking to nothing. This is
|
||||
the exact problem in `expose-service-basics`'s scenario: "other pods cannot reliably connect
|
||||
because pod IPs change when they restart."
|
||||
|
||||
## A Service: a stable address in front of a moving set of Pods
|
||||
|
||||
A Service is a stable virtual IP and DNS name that automatically routes to whichever Pods
|
||||
currently match its label selector — regardless of how many there are or how often they change.
|
||||
You never talk to a Pod's IP directly in normal operation; you talk to the Service, and the
|
||||
Service's Endpoints (a live, continuously-updated list of the matching Pods' IPs) determines
|
||||
where traffic actually goes.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: redis-svc
|
||||
spec:
|
||||
selector:
|
||||
app: redis # <- matches Pods by label, same mechanism from ROOT 05
|
||||
ports:
|
||||
- port: 6379
|
||||
```
|
||||
|
||||
This is the same label-selector mechanism you learned in ROOT 05 — a Service doesn't know or
|
||||
care about specific Pod names, only which Pods currently carry a matching label.
|
||||
|
||||
## The three Service types you'll use most
|
||||
|
||||
- **ClusterIP** (the default) — reachable only from inside the cluster. Use this for
|
||||
internal-only communication, like one microservice talking to another or to a database.
|
||||
- **NodePort** — additionally exposes the Service on a static port on every node's own IP,
|
||||
reachable from outside the cluster. Mostly used for development/testing or when there's no
|
||||
cloud load balancer available.
|
||||
- **LoadBalancer** — asks the cloud provider to provision a real external load balancer pointed
|
||||
at the Service. This is the standard way to expose something to the public internet on a
|
||||
cloud-hosted cluster.
|
||||
|
||||
Choosing wrong isn't just a style preference — a ClusterIP Service is architecturally
|
||||
unreachable from outside the cluster no matter how correctly everything else is configured.
|
||||
|
||||
## DNS: how Pods actually find a Service by name
|
||||
|
||||
Every Service automatically gets a DNS name inside the cluster
|
||||
(`<service-name>.<namespace>.svc.cluster.local`, or usually just `<service-name>` from within
|
||||
the same Namespace) — this is why application code can use a plain hostname like `redis-svc`
|
||||
instead of an IP address at all. Kubernetes' internal DNS resolves that name to the Service's
|
||||
stable virtual IP, which then routes to whichever Pods are currently healthy.
|
||||
|
||||
## The full request path, end to end
|
||||
|
||||
```
|
||||
Client -> DNS lookup "redis-svc" -> Service's stable virtual IP
|
||||
-> Service consults its Endpoints (live list of matching, ready Pods)
|
||||
-> traffic forwarded to one of those Pod IPs
|
||||
```
|
||||
|
||||
Every one of these steps can fail independently, which is exactly why "the app can't reach
|
||||
redis" requires checking each layer rather than guessing — the next activity shows you what
|
||||
that looks like when the Endpoints layer is empty.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read, 10-12 minutes live with the request-path diagram.
|
||||
|
||||
**Demo plan:** Before the next activity, it's worth live-running `kubectl get endpoints
|
||||
<service-name>` against a healthy Service so students see a populated Endpoints list before
|
||||
they see an empty one in the demonstration.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often think a Service "is" a load balancer process running somewhere. It's closer to
|
||||
a routing rule maintained by `kube-proxy` on every node — worth clarifying that there's no
|
||||
single "Service Pod" to go find and debug.
|
||||
- ClusterIP vs. NodePort vs. LoadBalancer is sometimes memorized without the "why" — the
|
||||
knowledge check's Service-type question is designed to test the underlying reasoning, not
|
||||
just recall.
|
||||
|
||||
**Discussion question:** "If a Service's selector matches zero Pods, does the Service itself
|
||||
fail to exist?" (Answer: no — the Service object exists fine; its Endpoints list is just empty,
|
||||
which looks like connection failure from the client's side without any error at the Service
|
||||
level itself. This is exactly the next activity's topic.)
|
||||
+82
@@ -0,0 +1,82 @@
|
||||
# Demonstration: A Service With Zero Endpoints
|
||||
|
||||
This is one of the most common "it just doesn't work" incidents in real Kubernetes operation —
|
||||
and it produces no error message at all at the Service level.
|
||||
|
||||
## The setup
|
||||
|
||||
A Service `api-svc` selects Pods labeled `app: api`. A Deployment was recently updated and its
|
||||
Pods now carry the label `app: api-v2` instead (a rename during a refactor, deliberately or
|
||||
accidentally).
|
||||
|
||||
## What you'd see from the client side
|
||||
|
||||
```
|
||||
$ curl http://api-svc:8080/health
|
||||
curl: (7) Failed to connect to api-svc port 8080: Connection refused
|
||||
```
|
||||
|
||||
Just a generic connection failure — nothing here says "your Service's selector doesn't match
|
||||
any Pods."
|
||||
|
||||
## What the Service object itself shows
|
||||
|
||||
```
|
||||
$ kubectl get service api-svc
|
||||
NAME TYPE CLUSTER-IP PORT(S) AGE
|
||||
api-svc ClusterIP 10.96.34.201 8080/TCP 2h
|
||||
```
|
||||
|
||||
Completely normal-looking. The Service exists, has a stable IP, is not in any kind of "error"
|
||||
state — because from the Service's own perspective, nothing is wrong. It's doing exactly what
|
||||
it's configured to do: route to whatever matches its selector, which currently happens to be
|
||||
nothing.
|
||||
|
||||
## Where the real evidence is: Endpoints
|
||||
|
||||
```
|
||||
$ kubectl get endpoints api-svc
|
||||
NAME ENDPOINTS AGE
|
||||
api-svc <none> 2h
|
||||
```
|
||||
|
||||
This is the actual signal. `<none>` means the Service's selector currently matches zero Pods —
|
||||
this is the single fastest check for "is this a Service-to-Pod mismatch" any time something
|
||||
reachable-in-principle looks like a connection failure.
|
||||
|
||||
## Confirming the mismatch
|
||||
|
||||
```
|
||||
$ kubectl get pods --show-labels
|
||||
NAME READY STATUS LABELS
|
||||
api-abc123 1/1 Running app=api-v2
|
||||
```
|
||||
|
||||
The Pod is healthy and running — `app=api-v2`, not `app=api`. The Service's selector
|
||||
(`app: api`) matches nothing, and nothing about the Pod itself is broken.
|
||||
|
||||
## The general diagnostic shape
|
||||
|
||||
`curl`/application-level connection failure tells you *that* something's wrong, but nothing
|
||||
about *where*. `kubectl get endpoints <service>` is the fastest way to distinguish "the Service
|
||||
has no matching Pods" from "the Pods exist and are ready but something else is wrong" —
|
||||
reach for it early whenever a Service-fronted connection fails for no obvious reason.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Live-create this exact mismatch (a Service with one selector, a Deployment
|
||||
with a slightly different label) and walk through `kubectl get service` -> `kubectl get
|
||||
endpoints` -> `kubectl get pods --show-labels` in front of the class, narrating what each
|
||||
command is and isn't telling you.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; 5-7 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students often check the Service object first and, seeing
|
||||
nothing obviously wrong, assume the problem must be elsewhere entirely (DNS, firewall, the
|
||||
application itself) — reinforce that `kubectl get endpoints` is the specific, fast check for
|
||||
this exact failure mode, before reaching for anything more complex.
|
||||
|
||||
**Optional extension:** ask what `kubectl describe service api-svc` would additionally show
|
||||
compared to `kubectl get service` (answer: the same selector the Service is using, which lets
|
||||
you directly compare it against `kubectl get pods --show-labels`'s actual Pod labels side by
|
||||
side, rather than inferring the mismatch).
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
# Knowledge Check: Services & Service Discovery
|
||||
|
||||
---
|
||||
|
||||
**1. A Deployment has three replicas but `kubectl get endpoints` shows `<none>` for its Service.
|
||||
Which part of the system should you inspect first, and why?**
|
||||
|
||||
*Answer:* Compare the Service's selector (`kubectl describe service`) against the actual Pods'
|
||||
labels (`kubectl get pods --show-labels`) — a Service with a healthy backend Deployment but zero
|
||||
Endpoints almost always means the selector doesn't match the Pods' current labels, not that the
|
||||
Pods themselves are unhealthy.
|
||||
|
||||
---
|
||||
|
||||
**2. You need a microservice reachable ONLY from other services inside the same cluster, never
|
||||
from the public internet. Which Service type, and why not LoadBalancer?**
|
||||
|
||||
*Answer:* ClusterIP — it's reachable only from inside the cluster by design, which is exactly
|
||||
the requirement. A LoadBalancer would provision a real external endpoint reachable from the
|
||||
public internet, which is more exposure than the requirement calls for, and depends on cloud
|
||||
provider support this scenario doesn't need.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: application code that connects to `redis-svc` by that plain hostname is
|
||||
relying on Kubernetes' internal DNS to resolve it.**
|
||||
|
||||
*Answer:* True. Every Service automatically gets a DNS name; that's specifically what lets code
|
||||
use a stable hostname instead of hardcoding a Pod IP that could change at any time.
|
||||
|
||||
---
|
||||
|
||||
**4. A Service's Endpoints list is non-empty, but the application still can't connect. Does
|
||||
this rule out a Service-level problem entirely?**
|
||||
|
||||
*Answer:* Not entirely — a non-empty Endpoints list means at least one Pod currently matches the
|
||||
selector, but the connection could still fail for other reasons (wrong port number on the
|
||||
Service, a NetworkPolicy blocking the traffic, the application inside the Pod not actually
|
||||
listening). A populated Endpoints list narrows the problem, it doesn't prove the whole path
|
||||
works.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 1 is the single most transferable diagnostic habit from this module —
|
||||
worth explicitly calling out as "the first thing to check" for this entire class of problem.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+49
@@ -0,0 +1,49 @@
|
||||
# Guided Exercise: Choosing the Right Service Type
|
||||
|
||||
Paper-and-thinking exercise before the Lab.
|
||||
|
||||
## The scenario
|
||||
|
||||
You're deploying three components for a class project on a cloud-hosted cluster:
|
||||
|
||||
1. A Postgres database that only your own backend application should ever talk to.
|
||||
2. A backend API that the frontend (also running in the cluster) calls internally.
|
||||
3. The frontend web application itself, which needs to be reachable by users on the public
|
||||
internet.
|
||||
|
||||
## Your task
|
||||
|
||||
For each of the three components, decide: ClusterIP, NodePort, or LoadBalancer — and justify
|
||||
each choice in one sentence referencing WHO needs to reach it.
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. **Postgres — ClusterIP.** Only your own backend (running inside the same cluster) needs to
|
||||
reach it; there is no reason for it to be reachable from outside the cluster at all, and
|
||||
exposing a database externally is a real, avoidable security risk.
|
||||
2. **Backend API — ClusterIP.** The frontend calls it from inside the cluster too — same
|
||||
reasoning as the database. External users never talk to the backend directly; they only ever
|
||||
go through the frontend.
|
||||
3. **Frontend — LoadBalancer** (on a real cloud-hosted cluster). Public internet users need to
|
||||
reach it, and a cloud provider's LoadBalancer is the standard mechanism for that. NodePort
|
||||
would technically work but is typically used for development/testing rather than a real
|
||||
public-facing service on a cloud cluster.
|
||||
|
||||
## Self-check
|
||||
|
||||
If you chose LoadBalancer for the database or the backend API, reconsider — "internet-reachable
|
||||
by default" is not the safe default for anything that doesn't need to be, and getting this
|
||||
choice right the first time avoids a real security exposure later, not just a style issue.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Push on the "why not just LoadBalancer everything, it works" instinct some
|
||||
students will have — this is a good early moment to connect Service-type choice to the security
|
||||
mindset that ROOT 09 builds on more fully (exposure should be intentional, not the path of
|
||||
least resistance).
|
||||
|
||||
**Approximate timing:** 10 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what would need to be added (not covered in this module) to let
|
||||
public users reach the frontend at a real domain name over HTTPS rather than a bare
|
||||
LoadBalancer IP — a natural forward pointer to Ingress, covered in the next module.
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
# Launch Lab: Exposing an Application
|
||||
|
||||
Time to create a real Service in your disposable Lab and connect it to a real Deployment.
|
||||
|
||||
## What you're doing
|
||||
|
||||
A `redis` Deployment is running, but other Pods can't reliably reach it because Pod IPs change
|
||||
whenever a Pod restarts — exactly the problem the lesson opened with. Your task is to create a
|
||||
stable ClusterIP Service in front of it.
|
||||
|
||||
## Before you start
|
||||
|
||||
Recall the request path from the lesson: your Service's `selector` has to match the Deployment
|
||||
Pods' actual labels, or you'll end up with the empty-Endpoints situation from the
|
||||
demonstration. Check the Deployment's Pod labels before writing your Service's selector, don't
|
||||
guess at them.
|
||||
|
||||
## Your task in this Lab
|
||||
|
||||
Create a ClusterIP Service exposing the `redis` Deployment on the correct port. Once it's
|
||||
created, verify it the same way the demonstration did: `kubectl get service`, then `kubectl get
|
||||
endpoints` to confirm it actually has a non-empty Endpoints list, not just that the Service
|
||||
object exists.
|
||||
|
||||
This Lab has its own in-context hints if you get stuck.
|
||||
|
||||
When you're done, return to this module for a reflection on why service discovery is
|
||||
infrastructure, not magic.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Classroom flow:** Encourage students to run `kubectl get endpoints` as their OWN verification
|
||||
step even though the Lab's automated validation will check this too — building the habit of
|
||||
checking Endpoints, not just "the Service exists," is the actual transferable skill here.
|
||||
|
||||
**Approximate timing:** 10-15 minutes.
|
||||
|
||||
**Common mistake to watch for:** a selector that matches on the wrong label key/value, or a
|
||||
port/targetPort mismatch (the Service's `port` is what clients connect to; `targetPort` is what
|
||||
the container actually listens on — conflating the two is a very common first-time error).
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
# Reflection: Service Discovery Is Infrastructure, Not Magic
|
||||
|
||||
You just created a Service that gives a moving set of Pods a stable address. Take a few minutes
|
||||
to connect this to real operational thinking.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Before this module, "the app can just connect to redis by name" might have seemed obvious or
|
||||
automatic. What specific mechanism actually makes that true, and what would break it?
|
||||
2. If your Service's selector had been wrong (matching zero Pods), what would the SYMPTOM have
|
||||
looked like from the application's side, versus what the actual CAUSE would have been?
|
||||
3. Why is checking `kubectl get endpoints` before assuming a networking/DNS/firewall problem a
|
||||
good default habit, given how common a selector mismatch actually is?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**"It's probably a networking issue" is often wrong, and Endpoints is how you find out fast.** A
|
||||
huge fraction of real "service X can't reach service Y" incidents in Kubernetes are Service
|
||||
selector mismatches, not actual network problems — checking `kubectl get endpoints` first, before
|
||||
escalating to firewall rules or DNS debugging, saves real time precisely because this failure
|
||||
mode produces zero errors at the layers people instinctively check first.
|
||||
|
||||
**Stable addressing is what makes rolling updates and self-healing actually usable.** Everything
|
||||
you learned in ROOT 06 about Pods being replaced during a rollout or after a crash only works
|
||||
smoothly in practice because Services give callers a stable target that survives those
|
||||
replacements — without this layer, every Pod replacement would require updating every caller's
|
||||
configuration by hand.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** use question 2 as a discussion prompt — the
|
||||
symptom/cause gap is exactly why this class of bug is often under-diagnosed in real teams.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to write the one-line runbook entry they'd want a future
|
||||
on-call engineer to have for "Service X seems unreachable" — a good exercise in turning a
|
||||
lesson into something operationally reusable.
|
||||
+90
@@ -0,0 +1,90 @@
|
||||
# Ingress and NetworkPolicy: What They Architecturally Do
|
||||
|
||||
## Ingress: routing external HTTP(S) traffic by host/path
|
||||
|
||||
You already know a LoadBalancer Service can expose something to the public internet. But a real
|
||||
application usually needs more than "one raw IP to one Service" — you might want
|
||||
`api.example.com` and `app.example.com` to route to different backend Services, all through one
|
||||
entry point, with TLS termination handled in one place instead of per-Service.
|
||||
|
||||
**Ingress** is the Kubernetes object for exactly this: a set of HTTP routing rules (by hostname
|
||||
and/or URL path) that an Ingress controller reads and turns into actual load-balancer/proxy
|
||||
configuration.
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: app-ingress
|
||||
spec:
|
||||
rules:
|
||||
- host: app.example.com
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: frontend-svc
|
||||
port: { number: 80 }
|
||||
```
|
||||
|
||||
An important distinction: Ingress is a set of ROUTING RULES, not a load balancer itself — an
|
||||
Ingress controller (a separate piece of software running in the cluster) is what actually reads
|
||||
these rules and does the routing. Without a controller installed, an Ingress object exists in
|
||||
the API but nothing acts on it.
|
||||
|
||||
## NetworkPolicy: traffic reachability as something you declare, not assume
|
||||
|
||||
By default, every Pod in a Kubernetes cluster can reach every other Pod — there is no built-in
|
||||
network isolation between them at all. This surprises people coming from a "obviously the
|
||||
network is locked down unless configured otherwise" assumption. It is exactly backwards from
|
||||
that assumption: **everything is reachable until you say otherwise.**
|
||||
|
||||
A NetworkPolicy changes this for the Pods it selects, by declaring which traffic IS allowed —
|
||||
everything not explicitly allowed to a Pod covered by at least one NetworkPolicy is then denied.
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: allow-frontend-to-backend
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels: { app: backend }
|
||||
ingress:
|
||||
- from:
|
||||
- podSelector:
|
||||
matchLabels: { app: frontend }
|
||||
```
|
||||
|
||||
This example allows traffic to `backend` Pods only from `frontend` Pods — once this policy
|
||||
exists and selects `backend`, every OTHER Pod trying to reach `backend` is now blocked, even
|
||||
though nothing was true about that before the policy existed.
|
||||
|
||||
## Why this matters architecturally, before it matters as a security control
|
||||
|
||||
Right now, think of NetworkPolicy purely as "declaring intended reachability, not assuming it."
|
||||
ROOT 09 comes back to NetworkPolicy specifically as a SECURITY control (default-deny postures,
|
||||
least-privilege network access) — this module is deliberately just the architectural mental
|
||||
model: Kubernetes doesn't isolate network traffic for you by default, and NetworkPolicy is the
|
||||
mechanism for declaring the reachability you actually intend.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often assume Ingress by itself "does" the routing. Emphasize the controller
|
||||
requirement — an Ingress object with no controller installed does nothing.
|
||||
- The "everything reachable by default" fact genuinely surprises most students the first time
|
||||
they hear it — it's worth stating plainly and letting it land rather than rushing past it.
|
||||
|
||||
**Discussion question:** "If NO NetworkPolicy exists anywhere in a Namespace, what's the actual
|
||||
reachability between Pods in that Namespace?" (Answer: completely open — every Pod can reach
|
||||
every other Pod, matching the "everything reachable by default" rule.)
|
||||
|
||||
**Optional extension:** ask what would happen if you created a NetworkPolicy that selects
|
||||
`frontend` Pods but declares no `ingress` rules at all (answer: `frontend` Pods would become
|
||||
completely unreachable by anything — an empty ingress list inside a NetworkPolicy that HAS an
|
||||
ingress policyType means "deny all incoming," not "no policy applies").
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
# Knowledge Check: Ingress & NetworkPolicy Foundations
|
||||
|
||||
---
|
||||
|
||||
**1. You create an Ingress object with correct routing rules, but no traffic is ever routed
|
||||
according to them. What's the most likely reason?**
|
||||
|
||||
*Answer:* No Ingress controller is installed/running in the cluster. Ingress objects are just
|
||||
routing rules; a separate controller has to actually read and act on them, or nothing happens.
|
||||
|
||||
---
|
||||
|
||||
**2. True or false: by default, a Pod in Namespace A cannot reach a Pod in Namespace B unless a
|
||||
NetworkPolicy explicitly allows it.**
|
||||
|
||||
*Answer:* False — the opposite is true by default. With NO NetworkPolicy in place at all,
|
||||
every Pod can reach every other Pod regardless of Namespace. NetworkPolicy is what RESTRICTS
|
||||
this default-open behavior, not what's required to enable connectivity in the first place.
|
||||
|
||||
---
|
||||
|
||||
**3. A NetworkPolicy selects Pods labeled `app: backend` and declares one ingress rule allowing
|
||||
traffic from Pods labeled `app: frontend`. What happens to traffic from a Pod labeled
|
||||
`app: admin-tool` trying to reach `backend`?**
|
||||
|
||||
*Answer:* It's blocked. Once a NetworkPolicy selects a Pod, only traffic matching one of its
|
||||
explicit rules is allowed — everything else to that Pod is denied, including traffic that would
|
||||
have worked fine before the policy existed.
|
||||
|
||||
---
|
||||
|
||||
**4. Why is Ingress the right tool for exposing an HTTP application by hostname, rather than
|
||||
creating a separate LoadBalancer Service per hostname?**
|
||||
|
||||
*Answer:* A LoadBalancer Service typically provisions its own external IP/load balancer per
|
||||
Service — one per hostname would mean paying for and managing N separate external endpoints.
|
||||
Ingress lets one entry point route to many backend Services by hostname/path, which is both
|
||||
cheaper and centralizes concerns like TLS termination.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2 catches a very common and consequential misconception — worth
|
||||
confirming out loud that everyone actually updated their mental model, not just guessed
|
||||
correctly.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+72
@@ -0,0 +1,72 @@
|
||||
# Demonstration: Tracing a Request Through Ingress
|
||||
|
||||
## The setup
|
||||
|
||||
An Ingress `web-ingress` routes `app.example.com/` to Service `web-svc`, which fronts a
|
||||
Deployment `web-app`. Nothing works yet from outside the cluster — let's see why, step by step.
|
||||
|
||||
## Step 1: is the Ingress object even doing anything?
|
||||
|
||||
```
|
||||
$ kubectl get ingress web-ingress
|
||||
NAME CLASS HOSTS ADDRESS PORTS AGE
|
||||
web-ingress traefik app.example.com 80 2m
|
||||
```
|
||||
|
||||
`ADDRESS` is empty. On a real cluster, this column fills in once the Ingress controller has
|
||||
actually processed the object and assigned it a reachable address — an empty address after a
|
||||
couple of minutes is the first sign to check the controller itself, not your YAML.
|
||||
|
||||
## Step 2: is the controller running at all?
|
||||
|
||||
```
|
||||
$ kubectl get pods -n kube-system -l app.kubernetes.io/name=traefik
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
traefik-7d8f9c6b5d-abcde 1/1 Running 0 10d
|
||||
```
|
||||
|
||||
Running fine — so the ADDRESS delay in this case is transient, not a missing controller. (If
|
||||
this returned no Pods at all, THAT would be the actual root cause — no controller means no
|
||||
Ingress object ever gets acted on, exactly like the demonstration in ROOT 07's Ingress lesson
|
||||
described.)
|
||||
|
||||
## Step 3: once ADDRESS populates, does the route actually work?
|
||||
|
||||
```
|
||||
$ kubectl get ingress web-ingress
|
||||
NAME CLASS HOSTS ADDRESS PORTS AGE
|
||||
web-ingress traefik app.example.com 10.0.1.50 80 3m
|
||||
|
||||
$ curl -H "Host: app.example.com" http://10.0.1.50/
|
||||
<html>...web-app's real response...</html>
|
||||
```
|
||||
|
||||
Success — but notice the `-H "Host: app.example.com"` header. Ingress routes by HOSTNAME, not
|
||||
just by IP — hitting the Ingress controller's address WITHOUT the matching Host header would
|
||||
route differently (or not at all), because that's literally what the Ingress rule is matching
|
||||
on.
|
||||
|
||||
## The general shape
|
||||
|
||||
Symptom (nothing reachable from outside) has at least three DIFFERENT possible layers: the
|
||||
Ingress object itself, the controller processing it, and the Service/Pod chain underneath it
|
||||
(already covered in ROOT 07's Services module). Check them in that order — Ingress ADDRESS,
|
||||
then controller health, then the underlying Service — rather than guessing at DNS or firewall
|
||||
issues first.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** If a real Ingress controller is available in the demo environment, run this
|
||||
live including the Host-header nuance — students are frequently surprised that `curl <ip>`
|
||||
alone doesn't work the way they expect.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; 5-7 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students often expect an Ingress's IP/address to work like
|
||||
a Service's ClusterIP (reachable directly, no extra header needed). Reinforce that Ingress
|
||||
routing is fundamentally HOSTNAME/PATH-based, which is exactly what makes one Ingress able to
|
||||
route many different domains through one entry point.
|
||||
|
||||
**Optional extension:** ask what would happen if TWO Ingress objects both claimed the same host
|
||||
+ path — a good forward pointer to why Ingress configuration conflicts are a real operational
|
||||
concern on a shared cluster, without needing to build a hands-on lab around it here.
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
# Guided Exercise: Planning Network Isolation
|
||||
|
||||
Paper-and-thinking exercise before the Lab.
|
||||
|
||||
## The scenario
|
||||
|
||||
A `database` Pod (label `app=database`) in the `netpol` namespace should only ever be reachable
|
||||
by Pods labeled `role=backend` — not the frontend, not anything else in the cluster.
|
||||
|
||||
## Your task
|
||||
|
||||
1. Before writing any YAML: right now, with NO NetworkPolicy in place, can the frontend reach
|
||||
the database? Why?
|
||||
2. What would a NetworkPolicy that ONLY specifies `podSelector: { matchLabels: { app: database
|
||||
} }` and no `ingress` rules at all do to traffic reaching the database — including from
|
||||
`role=backend` Pods?
|
||||
3. Sketch (in words, not full YAML) what the `ingress` rule needs to say to allow ONLY
|
||||
`role=backend` traffic in.
|
||||
4. If a new Pod is later created with `role=backend` in a DIFFERENT namespace, would your policy
|
||||
as sketched in #3 allow it? What would need to change if cross-namespace access should be
|
||||
allowed?
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. Yes — by default, every Pod can reach every other Pod regardless of labels or namespace
|
||||
(ROOT 07's Ingress & NetworkPolicy lesson). Nothing restricts this until a policy exists.
|
||||
2. It would block ALL incoming traffic to the database, including from `role=backend` — an
|
||||
empty `ingress` list under a policy that selects the Pod means "no traffic is allowed in,"
|
||||
not "no policy applies."
|
||||
3. An `ingress` rule with `from: [{ podSelector: { matchLabels: { role: backend } } }]` — this
|
||||
allows traffic specifically from Pods carrying that label, and nothing else.
|
||||
4. Not automatically — a bare `podSelector` in the `from` clause matches Pods in the SAME
|
||||
namespace as the policy by default. Allowing a different namespace requires adding a
|
||||
`namespaceSelector` alongside the `podSelector` in the same `from` entry.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2 is the one most likely to trip students up — it directly reinforces
|
||||
the "empty ingress rules means deny-all, not no-op" point from the lesson, which is worth
|
||||
confirming actually landed before the Lab.
|
||||
|
||||
**Approximate timing:** 10-12 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what a security-minded team would ALSO want to restrict about this
|
||||
database Pod's OWN outbound (egress) traffic, beyond who can reach it — a light forward pointer
|
||||
to ROOT 09's security-control framing of NetworkPolicy, without requiring students to build an
|
||||
egress rule here.
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
# Reflection: Reachability Should Be a Decision, Not a Default
|
||||
|
||||
You just configured real Ingress routing and real network isolation. Take a few minutes to
|
||||
connect this to production thinking.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Before this module, would you have assumed a fresh Kubernetes cluster isolates Pods from
|
||||
each other by default? Has that changed?
|
||||
2. If your team runs a multi-tenant cluster (several unrelated applications sharing it), what's
|
||||
the risk of NEVER applying any NetworkPolicy at all?
|
||||
3. Ingress lets many different domains share one entry point. What's the operational benefit of
|
||||
that beyond just "fewer objects to manage"?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**"No NetworkPolicy" is not neutral — it's full reachability.** Exactly like an unhardened
|
||||
Pod's SecurityContext (ROOT 09 covers this in depth), the absence of a NetworkPolicy isn't a
|
||||
safe default; it's the MOST permissive configuration available. On any cluster where more than
|
||||
one team or application shares infrastructure, that's a real, often-overlooked exposure.
|
||||
|
||||
**Centralizing external entry points through Ingress reduces attack surface, not just object
|
||||
count.** Every separate LoadBalancer Service is a separate externally-reachable endpoint to
|
||||
secure, monitor, and patch. Routing through one (or a small number of) Ingress controllers means
|
||||
security controls — TLS termination, rate limiting, WAF rules — are configured and audited in
|
||||
one place instead of scattered across every application team's own Service definitions.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 2 is worth pushing on with a concrete
|
||||
scenario — ask what a compromised low-value application (a demo app, a staging environment)
|
||||
could reach on a cluster with zero NetworkPolicies, versus one with sensible defaults.
|
||||
|
||||
**Approximate timing:** 8 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to sketch (in words) what a reasonable DEFAULT-DENY policy
|
||||
for an entire new namespace would look like, before any application-specific allow rules are
|
||||
added — previews the security-control framing ROOT 09 builds on.
|
||||
+74
@@ -0,0 +1,74 @@
|
||||
# Ephemeral vs. Persistent Storage
|
||||
|
||||
## The default: a container's filesystem is thrown away
|
||||
|
||||
By default, anything a container writes to its own filesystem disappears the moment that
|
||||
container is replaced — a crash, a rollout, a reschedule, anything. This is a deliberate design
|
||||
choice (it's part of what makes a Pod safely disposable/replaceable), but it means "just write
|
||||
files to disk" silently loses data unless you use one of Kubernetes' actual storage mechanisms.
|
||||
|
||||
## emptyDir: shared, but still temporary
|
||||
|
||||
An `emptyDir` volume gives a Pod's containers a shared directory that survives a container
|
||||
restart WITHIN the same Pod, but is deleted the moment the Pod itself is removed. It's the right
|
||||
tool for two containers in one Pod that need to share files with each other (a sidecar writing
|
||||
logs another container reads, for example) — not for anything that needs to outlive the Pod.
|
||||
|
||||
```yaml
|
||||
volumes:
|
||||
- name: shared-data
|
||||
emptyDir: {}
|
||||
```
|
||||
|
||||
## PersistentVolume & PersistentVolumeClaim: storage that outlives the Pod
|
||||
|
||||
A **PersistentVolume (PV)** represents an actual piece of storage (a cloud disk, an NFS share,
|
||||
whatever the cluster's infrastructure provides) as a Kubernetes object. A
|
||||
**PersistentVolumeClaim (PVC)** is a Pod's REQUEST for storage matching some criteria (size,
|
||||
access mode) — the Pod references the PVC, not a PV directly, and Kubernetes binds the PVC to a
|
||||
matching PV.
|
||||
|
||||
```
|
||||
Pod ──references──> PVC ("I need 5Gi, ReadWriteOnce") ──binds to──> PV (actual storage)
|
||||
```
|
||||
|
||||
This indirection matters: a Pod's manifest doesn't need to know or care exactly which physical
|
||||
storage backs its PVC — that's resolved separately, and the same PVC keeps working even if the
|
||||
Pod behind it is replaced.
|
||||
|
||||
## StorageClass & dynamic provisioning
|
||||
|
||||
Manually creating a PV for every PVC doesn't scale. A **StorageClass** describes a category of
|
||||
storage (e.g. "fast SSD" vs. "cheap archival") and, when a PVC requests that class, the cluster
|
||||
can automatically provision a matching PV on demand — this is "dynamic provisioning," and it's
|
||||
the normal way storage works on any real cloud-hosted cluster. A StorageClass also controls the
|
||||
**reclaim policy** — what happens to the underlying storage after its PVC is deleted (`Delete`
|
||||
the data, or `Retain` it for manual recovery).
|
||||
|
||||
## The decision in one pass
|
||||
|
||||
| You need... | Use |
|
||||
|---|---|
|
||||
| Temporary scratch space shared between containers in ONE Pod | `emptyDir` |
|
||||
| Storage that survives Pod replacement, manually provisioned | PV + PVC |
|
||||
| The above, provisioned automatically on demand | PVC + StorageClass (dynamic provisioning) |
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live with the PV/PVC/StorageClass diagram.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often assume `emptyDir` is "persistent" because the name doesn't obviously say
|
||||
otherwise. Be explicit: it survives a CONTAINER restart, not a POD deletion.
|
||||
- The PV/PVC indirection can seem like unnecessary ceremony at first — frame it as the same
|
||||
"declare what you want, let the system resolve it" pattern from ROOT 05's desired-state
|
||||
lesson, applied to storage instead of Pod count.
|
||||
|
||||
**Discussion question:** "If a Pod using a PVC is deleted and replaced, what happens to its
|
||||
data?" (Answer: it's preserved — the PVC and its bound PV are separate objects from the Pod,
|
||||
and a new Pod referencing the same PVC picks up right where the data was left, which is the
|
||||
entire point of using persistent storage over `emptyDir`.)
|
||||
|
||||
**Optional extension:** ask what a `Retain` reclaim policy is protecting against, compared to
|
||||
`Delete` (answer: accidental data loss from deleting a PVC by mistake — `Retain` keeps the
|
||||
underlying storage around for manual recovery instead of immediately destroying it).
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
# Knowledge Check: Storage Fundamentals
|
||||
|
||||
---
|
||||
|
||||
**1. Two containers in the same Pod need to share files with each other, but nothing needs to
|
||||
survive the Pod being replaced. What's the simplest correct choice?**
|
||||
|
||||
*Answer:* `emptyDir` — shared within the Pod, deliberately temporary, exactly matching the
|
||||
requirement. Reaching for a PV/PVC here would be more machinery than the problem needs.
|
||||
|
||||
---
|
||||
|
||||
**2. A Pod using a PVC is deleted and a new Pod is created referencing the SAME PVC. What
|
||||
happens to the data?**
|
||||
|
||||
*Answer:* It's preserved. The PVC and its bound PV exist independently of any specific Pod — the
|
||||
new Pod picks up exactly where the data was left, which is the entire reason to use persistent
|
||||
storage instead of `emptyDir` for anything that needs to outlive a single Pod's lifetime.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: a Pod's manifest must specify exactly which PersistentVolume backs its
|
||||
storage request.**
|
||||
|
||||
*Answer:* False. A Pod references a PVC (a request), not a PV directly — the binding between
|
||||
PVC and an actual PV is resolved separately, which is exactly the indirection that lets the same
|
||||
Pod manifest work regardless of which physical storage ends up behind it.
|
||||
|
||||
---
|
||||
|
||||
**4. What does a StorageClass's reclaim policy actually control, and why would a team choose
|
||||
`Retain` over `Delete` for a production database's storage?**
|
||||
|
||||
*Answer:* It controls what happens to the underlying storage after its PVC is deleted. `Retain`
|
||||
keeps the data around for manual recovery instead of immediately destroying it — a safety
|
||||
margin against an accidental `kubectl delete pvc` on something genuinely important, at the cost
|
||||
of needing manual cleanup later.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2 is worth confirming lands correctly — it's the single fact that
|
||||
justifies the entire PV/PVC indirection existing at all.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+86
@@ -0,0 +1,86 @@
|
||||
# How the Scheduler Places Pods, and How Governance Shapes It
|
||||
|
||||
## The scheduler's job
|
||||
|
||||
Every time a new Pod needs to run, the scheduler decides which node it lands on — you never
|
||||
manually assign a Pod to a node in normal operation (ROOT 05's mental model). By default, the
|
||||
scheduler picks based on available resources (does this node have enough free CPU/memory), but
|
||||
several mechanisms let you INFLUENCE or CONSTRAIN that decision when the default isn't good
|
||||
enough.
|
||||
|
||||
## Steering placement: nodeSelector, taints/tolerations, affinity
|
||||
|
||||
- **nodeSelector** — the simplest constraint: "only schedule this Pod on nodes with THIS
|
||||
label" (e.g. `disktype: ssd`). A blunt but clear tool.
|
||||
- **Taints and tolerations** — the inverse relationship. A NODE is tainted (marked "don't
|
||||
schedule here unless you specifically tolerate this taint") and a Pod TOLERATES a specific
|
||||
taint to be allowed to land there anyway. Commonly used to reserve nodes for specific
|
||||
workloads (e.g. GPU nodes tainted so only GPU-requiring Pods land there) or to keep general
|
||||
workloads off nodes reserved for something else.
|
||||
- **Affinity/anti-affinity** — more expressive rules: "prefer/require scheduling near Pods with
|
||||
label X" (affinity) or "prefer/require scheduling AWAY from Pods with label X"
|
||||
(anti-affinity). A common real use: anti-affinity between replicas of the same application, so
|
||||
a single node failure can't take down every replica at once.
|
||||
|
||||
## Priority: which Pods matter more when resources are tight
|
||||
|
||||
A **PriorityClass** lets you mark some Pods as more important than others. When the cluster is
|
||||
under resource pressure, a higher-priority pending Pod can even cause a lower-priority RUNNING
|
||||
Pod to be evicted (preempted) to make room — this is a real, consequential mechanism, not just a
|
||||
scheduling hint, and misconfiguring it can mean an unimportant batch job unexpectedly evicts
|
||||
something that actually mattered.
|
||||
|
||||
## PodDisruptionBudget: protecting against VOLUNTARY disruption
|
||||
|
||||
A **PodDisruptionBudget (PDB)** limits how many replicas of an application can be taken down
|
||||
AT ONCE during voluntary disruptions — a node drain for maintenance, a cluster upgrade — by
|
||||
declaring a minimum number (or percentage) that must stay available. This does NOT protect
|
||||
against involuntary disruption (a node crashing unexpectedly) — it specifically constrains
|
||||
operations that Kubernetes itself controls the timing of.
|
||||
|
||||
```yaml
|
||||
apiVersion: policy/v1
|
||||
kind: PodDisruptionBudget
|
||||
metadata:
|
||||
name: web-pdb
|
||||
spec:
|
||||
minAvailable: 2
|
||||
selector:
|
||||
matchLabels: { app: web }
|
||||
```
|
||||
|
||||
Without a PDB, a node drain during routine maintenance could legally take down every replica of
|
||||
`web` simultaneously if they all happened to be on nodes being drained together — a PDB is what
|
||||
prevents that from being "technically allowed."
|
||||
|
||||
## ResourceQuota & LimitRange: governance at the Namespace level
|
||||
|
||||
A **ResourceQuota** caps the TOTAL resources (CPU, memory, object counts) a Namespace can
|
||||
consume — useful for multi-team clusters where one team's runaway workload shouldn't be able to
|
||||
starve everyone else. A **LimitRange** sets DEFAULT and/or min/max resource requests/limits for
|
||||
individual containers within a Namespace, so a Pod created without explicit resource settings
|
||||
still gets sensible ones rather than none at all.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 20 minutes read; 15 minutes live given the number of concepts covered.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Taints/tolerations are frequently confused with affinity — clarify the directionality: a
|
||||
taint REPELS by default (node says no); a toleration is what lets a specific Pod through
|
||||
anyway. Affinity is the Pod expressing a preference, not the node restricting access.
|
||||
- Students sometimes assume a PDB prevents ALL disruption. Be explicit that it only constrains
|
||||
VOLUNTARY disruption (things Kubernetes/an operator controls the timing of) — a hardware
|
||||
failure ignores PDBs entirely, because nobody is "asking permission" for a crash.
|
||||
|
||||
**Discussion question:** "Why would a real team use anti-affinity between replicas of the same
|
||||
application instead of just trusting the scheduler's default spread?" (Answer: the default
|
||||
scheduler does consider spread as one factor among several, but doesn't GUARANTEE it the way an
|
||||
explicit anti-affinity rule does — for something where "don't lose all replicas to one node
|
||||
failure" is a hard requirement, not just a nice-to-have, an explicit rule is safer than an
|
||||
implicit tendency.)
|
||||
|
||||
**Optional extension:** ask what would happen to a Namespace's Pods if a ResourceQuota were
|
||||
added AFTER Pods without explicit resource requests already existed there (a good bridge to why
|
||||
LimitRange defaults matter — a Quota alone doesn't retroactively fix Pods created without
|
||||
resource requests, it just constrains what NEW Pods can consume going forward).
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
# Knowledge Check: Scheduling & Resource Governance
|
||||
|
||||
---
|
||||
|
||||
**1. A node is tainted `dedicated=gpu:NoSchedule`. A Pod with no matching toleration is
|
||||
pending and never gets scheduled anywhere else, even though other untainted nodes have free
|
||||
capacity. What's actually going on?**
|
||||
|
||||
*Answer:* Nothing is wrong — the taint only affects the tainted node specifically; it doesn't
|
||||
explain why the Pod isn't landing elsewhere. If it's stuck pending entirely, the cause is
|
||||
something else (insufficient resources cluster-wide, a nodeSelector/affinity rule that can't be
|
||||
satisfied anywhere) — the taint itself only ever repels from the ONE node it's applied to.
|
||||
|
||||
---
|
||||
|
||||
**2. During a planned node drain for maintenance, would you expect a PodDisruptionBudget to
|
||||
prevent an UNRELATED node from crashing unexpectedly at the same time?**
|
||||
|
||||
*Answer:* No. A PDB only constrains VOLUNTARY disruptions — operations Kubernetes/an operator
|
||||
controls the timing of, like a drain. An unrelated hardware failure is involuntary and ignores
|
||||
PDBs entirely; nothing is "asking permission" for a crash.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: a higher-priority pending Pod can cause a lower-priority, already-RUNNING
|
||||
Pod to be evicted to make room.**
|
||||
|
||||
*Answer:* True — this is what Pod priority/preemption actually does under resource pressure,
|
||||
not just a scheduling hint. Misconfigured priorities can have real, disruptive consequences for
|
||||
this reason.
|
||||
|
||||
---
|
||||
|
||||
**4. A team notices new Pods in their Namespace keep getting created with no CPU/memory
|
||||
requests at all, even though nobody set that intentionally. What governance mechanism would fix
|
||||
this going forward, and what would it NOT fix?**
|
||||
|
||||
*Answer:* A LimitRange with default requests/limits would apply sensible defaults to future
|
||||
Pods created without explicit values. It would NOT retroactively change Pods that already exist
|
||||
— it only affects new Pod creation from the point it's added.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2 reinforces a genuinely important, often-missed distinction
|
||||
(voluntary vs. involuntary disruption) — worth confirming it lands, not just that the answer was
|
||||
guessed correctly.
|
||||
|
||||
**Approximate timing:** 5-7 minutes given the density of this module's lesson.
|
||||
+87
@@ -0,0 +1,87 @@
|
||||
# Defense in Depth & the Kubernetes Security Model
|
||||
|
||||
## Security is a posture, not a single setting
|
||||
|
||||
There is no single "make it secure" switch in Kubernetes. Real security comes from **defense in
|
||||
depth** — multiple independent layers, each reducing what an attacker (or a bug) can do if an
|
||||
earlier layer fails. If a container has a vulnerability that lets an attacker execute arbitrary
|
||||
code, the question that actually matters is: what can that code DO once it's running? A
|
||||
container running as root, with every Linux capability, on a writable filesystem, with no
|
||||
restrictions at all, can do far more damage than one deliberately hardened to need almost
|
||||
nothing.
|
||||
|
||||
## Least privilege: grant only what's actually needed
|
||||
|
||||
The organizing principle behind everything in this course is **least privilege**: a workload
|
||||
should have exactly the permissions and capabilities it needs to do its job, and nothing more.
|
||||
This sounds obvious stated plainly, but the DEFAULT behavior of a plain container is closer to
|
||||
"maximum privilege unless told otherwise" — root user, all standard Linux capabilities, a
|
||||
writable filesystem. Hardening a Pod means deliberately narrowing that down.
|
||||
|
||||
## SecurityContext: where you actually configure this
|
||||
|
||||
`securityContext` (settable at the Pod level and per-container) is where you declare these
|
||||
restrictions:
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
containers:
|
||||
- name: app
|
||||
securityContext:
|
||||
readOnlyRootFilesystem: true
|
||||
capabilities:
|
||||
drop: ["ALL"]
|
||||
```
|
||||
|
||||
- **`runAsNonRoot` / `runAsUser`** — the container process runs as a specific non-root user
|
||||
instead of root. If a container image was built to require root and you set this, it will
|
||||
fail to start — which is a good, loud failure, not a silent security gap. Hardening
|
||||
sometimes surfaces an image's OWN bad assumptions.
|
||||
- **`capabilities: drop: ["ALL"]`** — Linux capabilities are fine-grained permissions
|
||||
(`NET_BIND_SERVICE`, `SYS_ADMIN`, dozens more) a process can hold even without full root.
|
||||
Dropping all of them, then adding back only the specific ones actually needed, is the
|
||||
least-privilege pattern applied to capabilities specifically.
|
||||
- **`readOnlyRootFilesystem: true`** — the container's own filesystem cannot be written to at
|
||||
all (use an explicit `emptyDir` volume, ROOT 08, for anything that genuinely needs to write
|
||||
temporary files). This closes off a whole category of attack that relies on writing a
|
||||
malicious file to disk.
|
||||
|
||||
## Why this matters even when "nothing is currently wrong"
|
||||
|
||||
None of these settings fix a bug or a vulnerability — they limit the BLAST RADIUS if one exists
|
||||
that you don't yet know about. A container hardened with non-root, dropped capabilities, and a
|
||||
read-only filesystem gives a successful exploit dramatically less to work with than an
|
||||
unhardened one, even against the exact same vulnerability. This is the entire point of defense
|
||||
in depth: you don't rely on any single layer being perfect.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10-12 minutes live with the demonstration.
|
||||
|
||||
**Demo plan:** Have the demonstration activity ready to run right after this lesson — seeing a
|
||||
concrete before/after Pod definition lands the abstract "least privilege" principle far better
|
||||
than the lesson text alone.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students sometimes think hardening settings are only relevant "if you expect to be attacked."
|
||||
Reframe: defense in depth assumes SOMETHING will eventually go wrong somewhere (a
|
||||
vulnerability, a bug, a misconfiguration) and prepares for that, rather than betting
|
||||
everything on nothing ever going wrong.
|
||||
- `runAsNonRoot: true` failing a Pod's startup is sometimes read as "the security setting is
|
||||
broken." It's the opposite — it's correctly refusing to let a root-requiring image start,
|
||||
which is valuable, actionable information about the image itself.
|
||||
|
||||
**Discussion question:** "If dropping all capabilities breaks a container that turns out to
|
||||
genuinely need one specific capability, what's the right fix — removing the drop, or something
|
||||
more targeted?" (Answer: add back only that ONE specific capability rather than removing the
|
||||
blanket drop — least privilege means "exactly what's needed," not "give up and grant
|
||||
everything again.")
|
||||
|
||||
**Optional extension:** ask students to identify one setting from this lesson that would have
|
||||
made zero difference against a vulnerability that lets an attacker read environment variables
|
||||
(a good check that they understand these are layered, not universal, protections — the answer
|
||||
is that none of SecurityContext's settings here directly protect secrets stored in env vars;
|
||||
that's a different concern, covered later in this course).
|
||||
+87
@@ -0,0 +1,87 @@
|
||||
# Demonstration: A Pod Before and After Hardening
|
||||
|
||||
## Before: the default, unhardened Pod
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: app-unhardened
|
||||
spec:
|
||||
containers:
|
||||
- name: app
|
||||
image: myapp:latest
|
||||
```
|
||||
|
||||
Checking what this actually runs as:
|
||||
|
||||
```
|
||||
$ kubectl exec app-unhardened -- id
|
||||
uid=0(root) gid=0(root) groups=0(root)
|
||||
|
||||
$ kubectl exec app-unhardened -- touch /etc/anything-i-want
|
||||
$ kubectl exec app-unhardened -- ls -la /etc/anything-i-want
|
||||
-rw-r--r-- 1 root root 0 Jan 1 00:00 /etc/anything-i-want
|
||||
```
|
||||
|
||||
Running as root, with a fully writable filesystem, including system directories. This is not a
|
||||
misconfiguration — it's simply what happens when nothing restricts it.
|
||||
|
||||
## After: the same application, hardened
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: app-hardened
|
||||
spec:
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
containers:
|
||||
- name: app
|
||||
image: myapp:latest
|
||||
securityContext:
|
||||
readOnlyRootFilesystem: true
|
||||
capabilities:
|
||||
drop: ["ALL"]
|
||||
```
|
||||
|
||||
```
|
||||
$ kubectl exec app-hardened -- id
|
||||
uid=1000 gid=0 groups=0
|
||||
|
||||
$ kubectl exec app-hardened -- touch /etc/anything-i-want
|
||||
touch: /etc/anything-i-want: Read-only file system
|
||||
```
|
||||
|
||||
Same application image, same functionality (assuming the app doesn't itself require root or
|
||||
filesystem writes it wasn't told about) — but a compromise of this container now can't write
|
||||
to the filesystem, can't use the elevated capabilities that were dropped, and never had root to
|
||||
begin with.
|
||||
|
||||
## What this actually buys you
|
||||
|
||||
Nothing here fixed a bug in `myapp:latest` — the image is identical in both cases. What changed
|
||||
is what an attacker (or a bug behaving unexpectedly) could DO if something inside that container
|
||||
went wrong. The unhardened version had almost no limits; the hardened version has several
|
||||
independent ones, any of which alone would have blocked the `touch` command above.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Run both Pods live if time allows — watching `touch` succeed on one and fail
|
||||
on the other in real time is far more convincing than reading it.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; 5-7 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students sometimes read the hardened Pod's failed `touch`
|
||||
as "hardening broke something." Reinforce: the FUNCTIONALITY didn't change (the application
|
||||
itself never needed to write to `/etc`) — only an unnecessary, unused capability was removed.
|
||||
If hardening breaks something the application genuinely needs, that's real information about
|
||||
the application, not a reason to skip hardening.
|
||||
|
||||
**Optional extension:** ask what would happen if `myapp:latest` were built assuming it could
|
||||
write a cache file to `/tmp` inside the container — a good bridge to the fact that
|
||||
`readOnlyRootFilesystem` often needs an explicit `emptyDir` volume mounted at the specific path
|
||||
an application legitimately needs to write to (ROOT 08), rather than being an all-or-nothing
|
||||
choice.
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
# Knowledge Check: Security Model & SecurityContext
|
||||
|
||||
---
|
||||
|
||||
**1. A container is hardened with `capabilities: drop: ["ALL"]` and then fails because it
|
||||
genuinely needs to bind to a low-numbered network port, which requires `NET_BIND_SERVICE`.
|
||||
What's the least-privilege fix?**
|
||||
|
||||
*Answer:* Add back only `NET_BIND_SERVICE` specifically (`capabilities: add:
|
||||
["NET_BIND_SERVICE"]` alongside the drop-all), not remove the blanket drop entirely.
|
||||
Least privilege means granting exactly what's needed, not reverting to granting everything.
|
||||
|
||||
---
|
||||
|
||||
**2. True or false: setting `runAsNonRoot: true` fixes a vulnerability inside the application
|
||||
itself.**
|
||||
|
||||
*Answer:* False. It doesn't fix anything about the application's code — it limits what a
|
||||
compromise of that code could DO, by ensuring the process never has root privileges to begin
|
||||
with. This is the defense-in-depth distinction: reducing blast radius, not eliminating bugs.
|
||||
|
||||
---
|
||||
|
||||
**3. Why might `runAsNonRoot: true` cause a Pod to fail to start entirely, and is that a
|
||||
problem with the setting?**
|
||||
|
||||
*Answer:* If the container image was built assuming it runs as root (e.g. it tries to bind a
|
||||
privileged port, or write to a root-owned directory), the Pod will fail to start under
|
||||
`runAsNonRoot`. This is NOT a problem with the setting — it's valuable, actionable information
|
||||
that the image itself makes an assumption that should probably be fixed.
|
||||
|
||||
---
|
||||
|
||||
**4. A container has `readOnlyRootFilesystem: true` and needs to write temporary cache files.
|
||||
What's the correct way to allow that without abandoning the read-only filesystem entirely?**
|
||||
|
||||
*Answer:* Mount an explicit `emptyDir` volume (ROOT 08) at the specific path the application
|
||||
needs to write to. This keeps the rest of the filesystem read-only while allowing the one
|
||||
legitimately-needed writable location.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 1 and question 4 both reinforce the same "narrow the exception, don't
|
||||
remove the protection" pattern — worth calling out that they're the same principle applied
|
||||
twice.
|
||||
|
||||
**Approximate timing:** 5-7 minutes.
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
# Guided Exercise: Planning a Pod Hardening Checklist
|
||||
|
||||
Paper-and-thinking exercise before the Labs.
|
||||
|
||||
## The scenario
|
||||
|
||||
You're handed a Pod spec for a small internal API with NO `securityContext` set at all — the
|
||||
fully-default, unhardened case from the demonstration. Your team lead asks you to harden it
|
||||
before it ships.
|
||||
|
||||
## Your task
|
||||
|
||||
Without writing the actual YAML yet, answer these:
|
||||
|
||||
1. List the settings you'd add, in the order you'd verify them (hint: there are at least three
|
||||
distinct concerns from the lesson — user/root, filesystem, capabilities).
|
||||
2. For each setting, name ONE way you'd notice if you'd broken the application by adding it
|
||||
(what would the failure actually look like?).
|
||||
3. If the application genuinely needs to write a cache file to disk, does that mean you should
|
||||
skip `readOnlyRootFilesystem` entirely? What's the more precise fix?
|
||||
4. Suppose after hardening, the container still runs fine but a security scan flags that it
|
||||
still has the `NET_RAW` capability. What's the most likely reason, and what's the fix?
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. In a reasonable order: (a) `runAsNonRoot`/`runAsUser` — a fundamental identity change,
|
||||
surfaces image assumptions fastest; (b) `capabilities: drop: ["ALL"]` — narrow, usually safe
|
||||
for a typical web API; (c) `readOnlyRootFilesystem: true` — most likely to need an
|
||||
accompanying volume, so verify last once the others are confirmed working.
|
||||
2. (a) Pod fails to start entirely (`CrashLoopBackOff` or an immediate error) if the image
|
||||
assumed root. (b) The application errors on a specific operation that needed a capability
|
||||
you dropped (e.g. failing to bind a privileged port). (c) A write operation fails at runtime
|
||||
(a log line, an exception) the first time the app tries to write to disk.
|
||||
3. No — the fix is mounting an explicit `emptyDir` volume at the specific path the cache needs,
|
||||
keeping the rest of the filesystem read-only rather than abandoning the protection entirely.
|
||||
4. Most likely: `capabilities: drop: ["ALL"]` was never actually applied (a YAML indentation
|
||||
mistake, or it was added to the Pod's securityContext instead of the container's — capability
|
||||
drops are container-level, not Pod-level). The fix is verifying the drop is actually present
|
||||
on the CONTAINER's securityContext specifically.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 4 is deliberately tricky — Pod-level vs. container-level
|
||||
`securityContext` fields is a real, common source of "I set this and it didn't work" confusion
|
||||
worth surfacing here before students hit it in a Lab.
|
||||
|
||||
**Approximate timing:** 15 minutes live with discussion; 12-15 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask students to identify which of the settings in this exercise would
|
||||
be MOST valuable to apply first if they could only add one to a large, existing fleet of
|
||||
unhardened Pods under time pressure, and why.
|
||||
+42
@@ -0,0 +1,42 @@
|
||||
# Reflection: Least Privilege Is a Default, Not an Afterthought
|
||||
|
||||
You just hardened real Pods across several Labs — non-root, dropped capabilities, a read-only
|
||||
filesystem, and a full SecurityContext review. Take a few minutes to connect this to real
|
||||
practice.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Before this module, would you have described "runs as root with a fully writable
|
||||
filesystem" as insecure, or just as "the default, unremarkable configuration"? Has that
|
||||
changed?
|
||||
2. None of the settings you configured fixed a bug in any application. What did they actually
|
||||
change, precisely?
|
||||
3. If your team had to choose between hardening every new Pod by default (as a policy) versus
|
||||
hardening Pods only after an incident reveals a gap, which produces less total risk over
|
||||
time, and why?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**Unhardened is not neutral — it's the maximum-privilege default.** A container with no
|
||||
SecurityContext configured isn't "unopinionated"; it's root, every capability, fully writable —
|
||||
the MOST privileged configuration possible, not a safe middle ground. Teams that treat hardening
|
||||
as optional are choosing maximum blast radius by default, usually without realizing they're
|
||||
choosing anything at all.
|
||||
|
||||
**Defense in depth means no single mistake should be catastrophic.** The value of stacking
|
||||
non-root + dropped capabilities + read-only filesystem isn't that any ONE of them is perfect —
|
||||
it's that a real compromise would have to defeat several independent restrictions simultaneously
|
||||
instead of exploiting one gap that grants everything. This is exactly why security-minded teams
|
||||
apply all of these by default rather than picking "the one that matters most."
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 3 is worth pushing on directly — it's
|
||||
the actual business case for security-by-default, not just a technical preference.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to draft a one-paragraph "default Pod hardening policy"
|
||||
their hypothetical team would adopt, based on what they've learned in this module.
|
||||
+90
@@ -0,0 +1,90 @@
|
||||
# Identity for Workloads: ServiceAccounts & RBAC
|
||||
|
||||
## Workloads need an identity too, separate from yours
|
||||
|
||||
When YOU run `kubectl`, you're authenticated as yourself. But a Pod running INSIDE the cluster
|
||||
that needs to call the Kubernetes API (to list other Pods, read a ConfigMap, whatever) needs its
|
||||
own identity — it can't and shouldn't use your personal credentials. That identity is a
|
||||
**ServiceAccount**.
|
||||
|
||||
Every Pod runs as SOME ServiceAccount — if you don't specify one, it uses the `default`
|
||||
ServiceAccount in its Namespace, which by default has essentially no permissions. A Pod's
|
||||
ServiceAccount token is automatically mounted into the container unless you explicitly disable
|
||||
that (`automountServiceAccountToken: false`) — worth doing for any Pod that never actually needs
|
||||
to call the Kubernetes API at all, since an unused, mounted credential is still something an
|
||||
attacker could steal and use.
|
||||
|
||||
## RBAC: what that identity is actually allowed to do
|
||||
|
||||
Having an identity (a ServiceAccount) doesn't grant any permissions by itself — **RBAC
|
||||
(Role-Based Access Control)** is the separate system that defines what an identity can DO.
|
||||
|
||||
- A **Role** (namespace-scoped) or **ClusterRole** (cluster-wide) defines a set of permissions:
|
||||
which API resources, which verbs (get/list/watch/create/update/delete).
|
||||
- A **RoleBinding** (or **ClusterRoleBinding**) connects a Role/ClusterRole to a specific
|
||||
identity (a ServiceAccount, a user, or a group) — the Role alone grants nothing to anyone
|
||||
until it's bound.
|
||||
|
||||
```yaml
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: pod-reader
|
||||
namespace: default
|
||||
rules:
|
||||
- apiGroups: [""]
|
||||
resources: ["pods"]
|
||||
verbs: ["get", "list"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: read-pods
|
||||
namespace: default
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: my-app
|
||||
roleRef:
|
||||
kind: Role
|
||||
name: pod-reader
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
```
|
||||
|
||||
This grants the `my-app` ServiceAccount permission to `get`/`list` Pods in the `default`
|
||||
Namespace — and nothing else. It cannot delete Pods, cannot touch Secrets, cannot do anything
|
||||
not explicitly listed.
|
||||
|
||||
## Role vs. ClusterRole: scope matters
|
||||
|
||||
A Role only grants permissions within the ONE Namespace it's defined in. A ClusterRole can grant
|
||||
permissions across the entire cluster (or, bound via a RoleBinding instead of a
|
||||
ClusterRoleBinding, can also be reused namespace-by-namespace) — reach for a ClusterRole only
|
||||
when the workload genuinely needs cluster-wide visibility, not by default.
|
||||
|
||||
## Least privilege, applied to permissions specifically
|
||||
|
||||
The same principle from the previous module applies directly here: grant the narrowest set of
|
||||
verbs on the narrowest set of resources that the workload actually needs. A Role granting
|
||||
`get`/`list` on Pods is very different from one granting `*` on `*` — the latter is
|
||||
"root-equivalent" API access, and should be treated with exactly that level of caution.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often conflate "ServiceAccount" with "user account for logging in." A ServiceAccount
|
||||
is specifically an identity for WORKLOADS, not a human login mechanism.
|
||||
- The Role/RoleBinding two-step (define permissions, then separately grant them to someone) is
|
||||
sometimes seen as redundant. Emphasize the reuse benefit: the same Role can be bound to
|
||||
multiple identities without redefining the permission set each time.
|
||||
|
||||
**Discussion question:** "Why might you explicitly disable automounting a ServiceAccount token
|
||||
for a Pod that never calls the Kubernetes API?" (Answer: an unused but present credential is
|
||||
still something an attacker who compromises the container could steal and use elsewhere —
|
||||
removing it entirely removes that risk for a Pod that never needed it.)
|
||||
|
||||
**Optional extension:** ask what the practical difference is between granting a ClusterRole via
|
||||
a RoleBinding (scoped to one Namespace) versus a ClusterRoleBinding (cluster-wide) — a good
|
||||
check that students understand the Role/ClusterRole and Binding-type distinctions are
|
||||
independent axes, not the same choice.
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
# Knowledge Check: ServiceAccounts & RBAC
|
||||
|
||||
---
|
||||
|
||||
**1. A Pod is created with no ServiceAccount specified. What identity does it actually run as,
|
||||
and what can that identity do by default?**
|
||||
|
||||
*Answer:* The `default` ServiceAccount in its Namespace, which by default has essentially no
|
||||
permissions — it can authenticate to the API but is granted nothing until an RBAC binding says
|
||||
otherwise.
|
||||
|
||||
---
|
||||
|
||||
**2. You create a Role granting `get`/`list` on Pods, but a workload still can't read Pods.
|
||||
What's the most likely missing piece?**
|
||||
|
||||
*Answer:* A RoleBinding (or ClusterRoleBinding) connecting that Role to the workload's actual
|
||||
ServiceAccount. A Role by itself grants nothing to anyone — it has to be bound to an identity.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: a ClusterRole can only ever be applied cluster-wide.**
|
||||
|
||||
*Answer:* False. A ClusterRole can be bound via a RoleBinding to apply within just ONE
|
||||
Namespace (reusing a cluster-wide-DEFINED permission set in a namespace-SCOPED way), or via a
|
||||
ClusterRoleBinding to apply cluster-wide. The Role-type and Binding-type are independent
|
||||
choices.
|
||||
|
||||
---
|
||||
|
||||
**4. Why would a security review flag a Role granting `verbs: ["*"]` on `resources: ["*"]`
|
||||
even if the workload "seems to work fine"?**
|
||||
|
||||
*Answer:* That grant is root-equivalent API access — it violates least privilege regardless of
|
||||
whether the workload happens to only use a small subset of it today. "Works fine" doesn't mean
|
||||
"correctly scoped" — an over-broad grant is a real risk the moment that workload (or its
|
||||
credentials) is compromised, whether or not it's ever exploited in practice.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 4 is worth discussing even if everyone answers correctly — the "it
|
||||
works, so what's the problem" instinct is exactly what least-privilege review exists to counter.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+62
@@ -0,0 +1,62 @@
|
||||
# Demonstration: Watching RBAC Actually Deny Something
|
||||
|
||||
## The setup
|
||||
|
||||
A ServiceAccount `readonly-bot` has a Role granting only `get`/`list` on `pods` — nothing else.
|
||||
|
||||
## What it CAN do
|
||||
|
||||
```
|
||||
$ kubectl auth can-i list pods --as=system:serviceaccount:default:readonly-bot
|
||||
yes
|
||||
|
||||
$ kubectl get pods --as=system:serviceaccount:default:readonly-bot
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
web-abc123 1/1 Running 0 2h
|
||||
```
|
||||
|
||||
## What it CANNOT do
|
||||
|
||||
```
|
||||
$ kubectl auth can-i delete pods --as=system:serviceaccount:default:readonly-bot
|
||||
no
|
||||
|
||||
$ kubectl delete pod web-abc123 --as=system:serviceaccount:default:readonly-bot
|
||||
Error from server (Forbidden): pods "web-abc123" is forbidden:
|
||||
User "system:serviceaccount:default:readonly-bot" cannot delete resource "pods"
|
||||
in API group "" in the namespace "default"
|
||||
```
|
||||
|
||||
This is what least privilege actually looks like in practice — not a policy document, a real,
|
||||
enforced denial with a specific, readable error naming exactly which permission was missing.
|
||||
`kubectl auth can-i` is your fastest way to CHECK a permission before attempting an action that
|
||||
might fail — useful both for debugging "why can't my workload do X" and for verifying a Role is
|
||||
actually as narrow as intended.
|
||||
|
||||
## What happens when the ServiceAccount tries something it was never even considered for
|
||||
|
||||
```
|
||||
$ kubectl auth can-i get secrets --as=system:serviceaccount:default:readonly-bot
|
||||
no
|
||||
```
|
||||
|
||||
Notice this returns `no` the same way `delete pods` did — RBAC has no concept of "not yet
|
||||
decided," only "explicitly allowed" or "everything else, denied." This is the practical meaning
|
||||
of default-deny applied to permissions: nothing is grey area.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Run all three `kubectl auth can-i`/action pairs live — the FORBIDDEN error
|
||||
message's exact wording (naming the verb, resource, and namespace) is worth reading aloud, since
|
||||
that's exactly the evidence a student would use to debug a real permissions issue later.
|
||||
|
||||
**Approximate timing:** 8 minutes live; 5 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students sometimes think `kubectl auth can-i` REQUIRES the
|
||||
action to actually be attempted to know the answer. It doesn't — it's a dry-run permission
|
||||
check against the same RBAC rules, safe to run repeatedly without side effects.
|
||||
|
||||
**Optional extension:** ask what the error message would look like if `readonly-bot` tried to
|
||||
act on a DIFFERENT namespace than the one its Role/RoleBinding applies to — a good check that
|
||||
students understand Role scope is namespace-specific unless a ClusterRole+ClusterRoleBinding is
|
||||
used instead.
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
# Guided Exercise: Writing a Least-Privilege Role
|
||||
|
||||
Paper-and-thinking exercise before the Labs.
|
||||
|
||||
## The scenario
|
||||
|
||||
A monitoring agent needs to run inside your cluster. Its ONLY job is to periodically list Pods
|
||||
and read their status, across the whole cluster, to report health metrics elsewhere. It never
|
||||
needs to create, modify, or delete anything, and it never needs to touch Secrets, ConfigMaps, or
|
||||
any object type other than Pods.
|
||||
|
||||
## Your task
|
||||
|
||||
1. Should this use a Role or a ClusterRole? Justify your answer based on scope.
|
||||
2. Which verbs does it actually need? Which common verbs (create, update, delete, patch) should
|
||||
be deliberately left OUT?
|
||||
3. Does it need access to Secrets? Why is it tempting to grant broader access "just in case,"
|
||||
and why should you resist that?
|
||||
4. What would you check, using `kubectl auth can-i`, to CONFIRM your Role is exactly as narrow
|
||||
as intended before deploying it?
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. **ClusterRole** (bound via a ClusterRoleBinding) — it needs to see Pods across the WHOLE
|
||||
cluster, not just one namespace, so a namespace-scoped Role can't satisfy the requirement.
|
||||
2. **`get`, `list`, `watch`** on `pods` — read-only verbs only. Deliberately exclude `create`,
|
||||
`update`, `patch`, `delete` — this workload never legitimately needs to change anything.
|
||||
3. No — it never touches Secrets, so it should get zero permission on them. The temptation to
|
||||
"just in case" grant broader access is exactly what least privilege exists to resist: every
|
||||
unused permission is pure risk with no corresponding benefit if this workload is ever
|
||||
compromised.
|
||||
4. Run `kubectl auth can-i delete pods --as=<the ServiceAccount>` and confirm it returns `no` —
|
||||
and similarly check a Secrets verb returns `no` — proving the Role is bounded, not just
|
||||
assuming it from reading the YAML.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 3 is the core judgment call this exercise is testing — push on WHY
|
||||
"just in case" access is a real anti-pattern, not just point out that it is one.
|
||||
|
||||
**Approximate timing:** 10-12 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what would change about the answer if this monitoring agent later
|
||||
needed to also restart crash-looping Pods automatically — a good check that students can reason
|
||||
about incrementally expanding a Role's scope only as far as a genuinely new requirement demands.
|
||||
+42
@@ -0,0 +1,42 @@
|
||||
# Reflection: Permissions Should Reflect What a Workload Actually Needs
|
||||
|
||||
You just configured real ServiceAccounts and RBAC rules. Take a few minutes to connect this to
|
||||
production security practice.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Before this module, would you have thought twice about a workload's ServiceAccount having
|
||||
broad, unused permissions "just in case"? Has that changed?
|
||||
2. If a workload with cluster-admin-equivalent RBAC access is ever compromised (a vulnerable
|
||||
dependency, a supply-chain issue), what's the realistic blast radius compared to one scoped
|
||||
to exactly what it needs?
|
||||
3. How would you discover, on a real cluster you didn't build, whether any ServiceAccount has
|
||||
more access than it actually uses?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**RBAC over-permissioning is a silent, cumulative risk.** Unlike a misconfigured probe or a
|
||||
crashing Pod, an over-broad Role doesn't cause a visible symptom on a healthy day — it just sits
|
||||
there as unused risk until the one day a workload using it is compromised. This makes RBAC
|
||||
review a discipline that has to be practiced deliberately, not one that gets forced on you by
|
||||
an obvious failure.
|
||||
|
||||
**Least privilege is a default you set once, not a fix you apply after an incident.** Every
|
||||
example in this module started from "grant nothing, add exactly what's needed" rather than
|
||||
"grant broadly, restrict later" — the second approach almost never actually happens once
|
||||
something works, because there's no pressure forcing the cleanup.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 3 is a good one to leave partially open
|
||||
— acknowledge that real answers involve tooling beyond plain `kubectl` (RBAC audit tools) that
|
||||
this course doesn't build hands-on, while still reinforcing that the QUESTION itself is the
|
||||
right one to ask.
|
||||
|
||||
**Approximate timing:** 8 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to identify one ServiceAccount permission from earlier in
|
||||
ROOT_ (any Lab so far) that, in a real production system, they'd want to double-check was
|
||||
actually necessary.
|
||||
+76
@@ -0,0 +1,76 @@
|
||||
# NetworkPolicy as a Security Control & Supply Chain Trust
|
||||
|
||||
## NetworkPolicy, revisited as a security posture
|
||||
|
||||
ROOT 07 introduced NetworkPolicy architecturally: Kubernetes doesn't isolate Pod-to-Pod traffic
|
||||
by default, and NetworkPolicy is how you declare intended reachability. Here, the same
|
||||
mechanism is the point of a deliberate SECURITY practice: **default deny**.
|
||||
|
||||
A default-deny policy blocks all traffic to selected Pods, and you then explicitly allow only
|
||||
what's actually needed:
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-deny
|
||||
spec:
|
||||
podSelector: {} # selects EVERY Pod in the namespace
|
||||
policyTypes: ["Ingress"]
|
||||
```
|
||||
|
||||
An empty `podSelector: {}` with no ingress rules means "select every Pod, allow nothing in" —
|
||||
the network equivalent of the SecurityContext principle from earlier in this course: start
|
||||
from nothing, add back only what's proven necessary. This is exactly the opposite of
|
||||
Kubernetes' un-configured default (everything open), which is precisely why it has to be
|
||||
declared explicitly rather than relying on any built-in setting.
|
||||
|
||||
**Egress restriction** works the same way in reverse — restricting what a Pod can call OUT to,
|
||||
not just what can call IN. A compromised Pod that can freely make outbound connections to
|
||||
anything is far more dangerous (data exfiltration, calling out to an attacker's server) than one
|
||||
restricted to only the specific destinations it legitimately needs.
|
||||
|
||||
## Supply chain: trusting what you run before it ever starts
|
||||
|
||||
Everything covered so far in this course assumes the container image itself is trustworthy.
|
||||
Supply-chain security is about that assumption specifically:
|
||||
|
||||
- **Image-pull secrets** — when pulling from a private registry, the cluster needs credentials
|
||||
to authenticate. Storing and using these correctly (rather than, say, a publicly-readable
|
||||
registry with no access control) is a basic supply-chain control: only images from a trusted,
|
||||
access-controlled source get pulled at all.
|
||||
- **Minimizing image footprint** (referenced here, not a hands-on Lab in this module) — a
|
||||
smaller image means fewer packages, fewer known CVEs, and a smaller attack surface if the
|
||||
image is ever compromised.
|
||||
|
||||
## TLS at the Ingress layer: protecting data in transit
|
||||
|
||||
Recall ROOT 07's Ingress lesson — routing external HTTP(S) traffic by hostname/path. Configuring
|
||||
TLS termination at that same Ingress layer means traffic between users and your cluster is
|
||||
encrypted, protecting against interception on the network path. This is a security concern
|
||||
layered on top of the same architectural mechanism, the same relationship NetworkPolicy has to
|
||||
its ROOT 07 introduction.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students sometimes think "default deny" means blocking ALL traffic including what the
|
||||
application needs to function. Clarify: default deny is the STARTING point; you then add
|
||||
specific allow rules for exactly what's needed — the goal state is fully functional AND fully
|
||||
intentional, not non-functional.
|
||||
- Egress restriction is sometimes seen as less important than ingress. Emphasize the
|
||||
exfiltration angle explicitly — a compromised Pod's outbound reach is just as much a real
|
||||
risk as inbound exposure.
|
||||
|
||||
**Discussion question:** "Why is 'default deny, then explicitly allow' a better starting
|
||||
posture than 'default allow, then explicitly block known-bad traffic'?" (Answer: default-deny
|
||||
means a NEW, unanticipated attempt to reach or leave a Pod is blocked by default; default-allow
|
||||
means anything not YET identified as bad gets through — you're always one step behind an
|
||||
attacker under the second model.)
|
||||
|
||||
**Optional extension:** ask what a real team would want to verify before trusting a
|
||||
publicly-available container image from an unfamiliar source, beyond just "does it work" — a
|
||||
good bridge into image scanning/provenance concepts (not built as a hands-on Lab in this course,
|
||||
but worth naming as a real practice).
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
# Knowledge Check: Network & Supply Chain Security
|
||||
|
||||
---
|
||||
|
||||
**1. A NetworkPolicy has `podSelector: {}` with `policyTypes: ["Ingress"]` and no ingress rules
|
||||
defined. What's the practical effect on every Pod in that namespace?**
|
||||
|
||||
*Answer:* Every Pod is selected, and since no ingress rule allows anything, all incoming traffic
|
||||
to every Pod is denied — this is the "default deny" starting posture, applied namespace-wide.
|
||||
|
||||
---
|
||||
|
||||
**2. Why is restricting EGRESS (outbound) traffic from a Pod a real security control, not just
|
||||
a mirror-image nicety of ingress restriction?**
|
||||
|
||||
*Answer:* A compromised Pod with unrestricted egress can exfiltrate data to an attacker-controlled
|
||||
destination or call out to download additional malicious payloads. Restricting egress to only
|
||||
what's legitimately needed limits what a compromise can accomplish, independent of whether
|
||||
inbound access was ever an issue.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: "default deny, then explicitly allow" and "default allow, then explicitly
|
||||
block known-bad traffic" provide equivalent security given the same set of rules.**
|
||||
|
||||
*Answer:* False. Default-deny blocks anything NOT yet explicitly allowed, including new/
|
||||
unanticipated attempts. Default-allow only blocks what's already been identified as bad — an
|
||||
attacker doing something not yet on the block list gets through under the second model but not
|
||||
the first.
|
||||
|
||||
---
|
||||
|
||||
**4. Why does minimizing a container image's footprint (fewer packages) improve supply-chain
|
||||
security specifically, rather than just making the image smaller/faster to pull?**
|
||||
|
||||
*Answer:* Fewer installed packages means fewer potential known vulnerabilities (CVEs) present in
|
||||
the image at all, and a smaller attack surface if the image is ever compromised — it's a
|
||||
security property, not merely a performance optimization.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 3 is the core conceptual point of this module — worth confirming it's
|
||||
genuinely understood, not just correctly guessed.
|
||||
|
||||
**Approximate timing:** 5-7 minutes.
|
||||
+89
@@ -0,0 +1,89 @@
|
||||
# Demonstration: Default-Deny, Before and After
|
||||
|
||||
## Before: everything can reach everything
|
||||
|
||||
```
|
||||
$ kubectl run test-pod --image=busybox --rm -it -- wget -qO- --timeout=2 http://backend-svc
|
||||
<real response from backend>
|
||||
```
|
||||
|
||||
Any Pod in the namespace can reach `backend-svc`, regardless of what it is or who deployed it —
|
||||
this is the un-configured default the ROOT 07 lesson described, now made concrete from a
|
||||
security angle.
|
||||
|
||||
## Applying a default-deny policy
|
||||
|
||||
```
|
||||
$ kubectl apply -f default-deny.yaml
|
||||
networkpolicy.networking.k8s.io/default-deny created
|
||||
```
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-deny
|
||||
spec:
|
||||
podSelector: {}
|
||||
policyTypes: ["Ingress"]
|
||||
```
|
||||
|
||||
## After: the same request, now blocked
|
||||
|
||||
```
|
||||
$ kubectl run test-pod --image=busybox --rm -it -- wget -qO- --timeout=2 http://backend-svc
|
||||
wget: download timed out
|
||||
```
|
||||
|
||||
No error naming a permission (that's RBAC's job, a different layer) — just a timeout, because
|
||||
the packets are being dropped at the network layer before they ever reach the Pod. This is a
|
||||
DIFFERENT failure signature than an RBAC denial, worth being able to tell apart: RBAC denials
|
||||
are explicit and immediate; NetworkPolicy denials look like the destination is simply
|
||||
unreachable.
|
||||
|
||||
## Explicitly allowing only what's needed
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: allow-frontend-to-backend
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels: { app: backend }
|
||||
ingress:
|
||||
- from:
|
||||
- podSelector:
|
||||
matchLabels: { app: frontend }
|
||||
```
|
||||
|
||||
```
|
||||
$ kubectl run frontend-test --labels="app=frontend" --image=busybox --rm -it -- wget -qO- --timeout=2 http://backend-svc
|
||||
<real response from backend>
|
||||
|
||||
$ kubectl run other-test --labels="app=other" --image=busybox --rm -it -- wget -qO- --timeout=2 http://backend-svc
|
||||
wget: download timed out
|
||||
```
|
||||
|
||||
Same destination, same request — the ONLY difference is the caller's label. This is
|
||||
least-privilege network access made concrete: reachability now reflects an explicit decision,
|
||||
not an accident of both Pods existing in the same cluster.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Run all three states live if a real cluster is available — the timeout (not an
|
||||
error) for the denied case is worth calling out explicitly, since it looks identical to a
|
||||
genuinely broken/unreachable service from the caller's perspective.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; 5-7 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students sometimes expect a NetworkPolicy denial to
|
||||
produce a clear "forbidden" error like RBAC does. It doesn't — reinforce that a NetworkPolicy-
|
||||
caused timeout is genuinely hard to distinguish from an actual network/DNS problem without
|
||||
already suspecting NetworkPolicy, which is exactly why checking existing policies is a real
|
||||
diagnostic step worth knowing.
|
||||
|
||||
**Optional extension:** ask how a student would distinguish "blocked by NetworkPolicy" from
|
||||
"the backend Pod is actually down" purely from the caller's side (answer: check the backend
|
||||
Pod's own status/logs directly — if it's healthy and Ready, but still unreachable from one
|
||||
caller and reachable from another with a different label, NetworkPolicy is the likely cause).
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
# Guided Exercise: Planning Egress Restriction
|
||||
|
||||
Paper-and-thinking exercise before the Labs.
|
||||
|
||||
## The scenario
|
||||
|
||||
A `payment-processor` Pod handles sensitive transactions. It legitimately needs to reach:
|
||||
- an internal `payments-db` Service, and
|
||||
- one external payment gateway API.
|
||||
|
||||
It should NOT be able to reach anything else — including the public internet in general, other
|
||||
internal services, or any other namespace.
|
||||
|
||||
## Your task
|
||||
|
||||
1. Why is restricting this Pod's OUTBOUND (egress) traffic just as important as restricting who
|
||||
can reach IT (ingress), given what this Pod handles?
|
||||
2. If this Pod were compromised via a vulnerable dependency, what would UNRESTRICTED egress let
|
||||
an attacker do that restricted egress would prevent?
|
||||
3. Sketch (in words) what an egress policy for this Pod needs to allow, and — just as
|
||||
important — what it must NOT allow by omission.
|
||||
4. Why might a security review flag a Pod with `policyTypes: ["Egress"]` but an EMPTY `egress`
|
||||
list as suspicious, rather than assume it's simply "not configured yet"?
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. A Pod handling sensitive data that's compromised is far more dangerous if it can freely
|
||||
exfiltrate that data anywhere (unrestricted egress) than if its only reachable destinations
|
||||
are pre-approved and narrow.
|
||||
2. Unrestricted egress would let an attacker exfiltrate transaction data to an external server,
|
||||
or download additional malicious tooling. Restricted egress limits it to only the
|
||||
pre-approved destinations, even after compromise.
|
||||
3. Allow: egress to `payments-db` (internal, by label/namespace selector) and the specific
|
||||
external payment gateway (by IP range or, if supported, a domain-based egress rule). Must NOT
|
||||
allow: general internet access, other internal Services, or any other namespace by default.
|
||||
4. An empty `egress` list under `policyTypes: ["Egress"]` means DENY ALL outbound — exactly like
|
||||
the ingress case from the lesson. A reviewer seeing this should confirm it's intentional
|
||||
(a Pod that genuinely needs zero outbound network access) rather than assume it's an
|
||||
oversight, since the two look identical in the YAML.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 4 reinforces the same "empty rules list means deny-all, not
|
||||
unconfigured" point from earlier lessons — a good moment to confirm it generalizes correctly to
|
||||
egress, not just ingress.
|
||||
|
||||
**Approximate timing:** 10-12 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what additional control (beyond NetworkPolicy) a team might want for
|
||||
truly sensitive egress, like restricting DNS resolution itself for this Pod — a forward pointer
|
||||
to defense-in-depth without requiring a hands-on Lab here.
|
||||
+41
@@ -0,0 +1,41 @@
|
||||
# Reflection: Reachability and Trust Are Decisions You Make on Purpose
|
||||
|
||||
You just applied default-deny NetworkPolicy, egress restriction, TLS ingress, and image-pull
|
||||
authentication for real. Take a few minutes to connect these to a single underlying principle.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Four different mechanisms in this module (NetworkPolicy ingress, NetworkPolicy egress, TLS,
|
||||
image-pull secrets) all express the same underlying idea in a different layer. What is that
|
||||
idea, stated in one sentence?
|
||||
2. If you had to prioritize implementing ONLY ONE of this module's controls on a brand-new,
|
||||
unsecured cluster, which would you pick first, and why?
|
||||
3. How would you explain, to someone who thinks "our cluster is internal, so it doesn't need
|
||||
this," why internal-only still benefits from all four controls?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**Trust should be explicit, narrow, and verifiable — never assumed from proximity.** A Pod on
|
||||
the same cluster, a request from inside the corporate network, an image from "a registry we
|
||||
usually use" — none of these are actually trustworthy by default. Every control in this module
|
||||
replaces an assumption ("it's internal, it's probably fine") with an explicit, checkable
|
||||
decision (this specific Pod can reach this specific destination; this specific registry is
|
||||
authenticated; this specific connection is encrypted).
|
||||
|
||||
**Supply-chain and network security compound.** A compromised image with unrestricted egress on
|
||||
an open network is a far worse incident than the same compromised image on a hardened one — the
|
||||
controls in this module don't individually prevent every incident, but together they
|
||||
dramatically shrink what any single failure can turn into.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 1 is the module's thesis — make sure
|
||||
students can actually articulate it in their own words, not just recognize it when stated.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to rank this module's four controls by how much
|
||||
IMPLEMENTATION EFFORT each typically requires on a real team, separately from security value —
|
||||
a good exercise in real-world prioritization tradeoffs.
|
||||
+64
@@ -0,0 +1,64 @@
|
||||
# Pod Security Admission & a Survey of Runtime Hardening
|
||||
|
||||
This module is deliberately a survey, not a deep dive — the topics here are real and important,
|
||||
but lean toward cluster-operator/platform-team concerns rather than the daily-driver skills most
|
||||
ROOT_ learners will use constantly. Treat the labs in this module as good-to-know, not required
|
||||
mastery, before moving on.
|
||||
|
||||
## Pod Security Admission: enforcing a baseline across a Namespace
|
||||
|
||||
Everything you configured by hand in earlier modules (non-root, dropped capabilities, read-only
|
||||
filesystem) can also be ENFORCED automatically at the Namespace level via **Pod Security
|
||||
Admission (PSA)** — a label on the Namespace itself that rejects any Pod violating a chosen
|
||||
security standard (`privileged`, `baseline`, or `restricted`, from least to most strict):
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: production
|
||||
labels:
|
||||
pod-security.kubernetes.io/enforce: restricted
|
||||
```
|
||||
|
||||
This replaces the older PodSecurityPolicy (PSP) mechanism, which was removed from Kubernetes —
|
||||
if you encounter PSP in older documentation or an older cluster, know that PSA is its current
|
||||
replacement.
|
||||
|
||||
## Immutable Secrets: preventing a class of accidental change
|
||||
|
||||
A Secret can be marked `immutable: true`, which prevents its data from being changed after
|
||||
creation (a new Secret with a new name has to be created instead, and workloads updated to
|
||||
reference it). This trades a small amount of operational flexibility for a real safety
|
||||
property: an accidental or malicious edit to a live Secret can't silently happen.
|
||||
|
||||
## A survey of further topics (labs available, treat as optional depth)
|
||||
|
||||
- **Audit policy** — configuring what the Kubernetes API server logs about requests made against
|
||||
it (who did what, when) — genuinely important for incident investigation, but a cluster-admin
|
||||
concern, not something most workload teams configure themselves.
|
||||
- **RuntimeClass** — lets specific Pods run under a different container runtime configuration
|
||||
(e.g. an extra-sandboxed runtime like gVisor or Kata Containers) for workloads that need
|
||||
stronger isolation than the default runtime provides — relevant mainly for genuinely
|
||||
untrusted or high-risk workloads, not typical applications.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read (shorter than other lessons — deliberately survey-depth).
|
||||
|
||||
**Framing for instructors:** Be explicit with students that this module is intentionally
|
||||
lighter than the earlier ones in this course — the goal is awareness ("this exists, here's
|
||||
roughly what it's for") rather than hands-on fluency, consistent with not every CKS-style topic
|
||||
belonging in the core learning path.
|
||||
|
||||
**Discussion question:** "Why would a platform team prefer enforcing security standards via
|
||||
Pod Security Admission at the Namespace level, instead of trusting every team to configure
|
||||
SecurityContext correctly on every Pod by hand?" (Answer: PSA provides a hard backstop —
|
||||
correctly-configured SecurityContext relies on every individual Pod author doing it right every
|
||||
time; PSA rejects violations automatically regardless of whether any individual author
|
||||
remembered to.)
|
||||
|
||||
**Optional extension:** ask which of `privileged`/`baseline`/`restricted` a team should start a
|
||||
brand-new, security-conscious project with, and why not jump straight to the strictest option
|
||||
on a cluster already running older, unaudited workloads (a good practical judgment question
|
||||
about migration risk, not just definitions).
|
||||
+86
@@ -0,0 +1,86 @@
|
||||
# Why Health Checks Exist & Probe Types
|
||||
|
||||
Kubernetes cannot see inside your application. It doesn't know if your process is deadlocked,
|
||||
stuck in an infinite loop, or quietly failing every request while the process itself stays
|
||||
alive. Without a way to ask "are you actually working?", the only signal Kubernetes has is
|
||||
whether the container's process is still running — and a hung process still counts as running.
|
||||
|
||||
**A health probe is how your application tells Kubernetes the truth.** It's a small check —
|
||||
an HTTP request, a TCP connection attempt, or a command run inside the container — that
|
||||
Kubernetes runs on a schedule and uses to make real decisions: restart this container, or stop
|
||||
sending it traffic.
|
||||
|
||||
## The three probe types
|
||||
|
||||
Kubernetes supports three distinct probes, and they answer three different questions. Mixing
|
||||
them up is the single most common probe mistake in production.
|
||||
|
||||
**Liveness probe — "Is this container still alive, or should it be restarted?"**
|
||||
If a liveness probe fails repeatedly, the kubelet kills the container and it gets restarted per
|
||||
its restart policy. Use this for detecting a genuinely stuck process — a deadlock, a crashed
|
||||
event loop. Don't use liveness to mean "not ready for traffic yet" — that's what readiness is
|
||||
for, and confusing the two causes the exact incident you're about to fix.
|
||||
|
||||
**Readiness probe — "Is this container ready to receive traffic right now?"**
|
||||
If a readiness probe fails, the Pod is removed from the Service's list of endpoints —
|
||||
Kubernetes stops routing traffic to it, but does **not** restart the container. This is the
|
||||
right tool for "still starting up," "warming a cache," or "temporarily overloaded, back off for
|
||||
a bit." A Pod can be Running and healthy but still correctly Not Ready.
|
||||
|
||||
**Startup probe — "Has this slow-starting container finished starting up yet?"**
|
||||
Some applications take a long time to become ready (loading a large dataset, running
|
||||
migrations). A startup probe disables liveness/readiness checks until it succeeds once, so a
|
||||
slow-but-healthy startup doesn't get killed by an impatient liveness probe. Once it succeeds,
|
||||
liveness and readiness take over as normal.
|
||||
|
||||
## How a probe is actually defined
|
||||
|
||||
Every probe has a **mechanism** (how the check runs) and a **schedule** (how often, how long to
|
||||
wait, how many failures before acting):
|
||||
|
||||
- `httpGet` — a GET request to a path and **port** on the container. A 2xx or 3xx response
|
||||
counts as success.
|
||||
- `tcpSocket` — a TCP connection attempt to a **port**. A successful connection counts as
|
||||
success — this doesn't check application-level health, just "is something listening."
|
||||
- `exec` — runs a command inside the container. Exit code `0` counts as success.
|
||||
|
||||
Timing fields (`initialDelaySeconds`, `periodSeconds`, `timeoutSeconds`, `failureThreshold`,
|
||||
`successThreshold`) control how patient Kubernetes is before it acts — but no amount of correct
|
||||
timing fixes a probe that's pointed at the wrong port or path in the first place.
|
||||
|
||||
## Common production mistakes
|
||||
|
||||
1. **Wrong port.** The probe's `httpGet.port` (or `tcpSocket.port`) doesn't match the port the
|
||||
application actually listens on. The container is completely healthy — Kubernetes is just
|
||||
asking the wrong door to open. This is the exact failure you'll diagnose in this module's lab.
|
||||
2. **Liveness probe too aggressive on a slow-starting app**, with no startup probe — the app
|
||||
gets killed for "not being ready yet," restarts, and never gets far enough to pass.
|
||||
3. **Readiness probe that always passes** (e.g. checking a static file instead of real
|
||||
dependencies) — traffic gets routed to a Pod that can't actually serve it.
|
||||
4. **No probes configured at all** — Kubernetes has no way to know the container is stuck, so a
|
||||
deadlocked process just sits there serving nothing, forever, looking "Running" the whole time.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 12-15 minutes for a live read-through with pauses for questions;
|
||||
10 minutes for a self-paced read.
|
||||
|
||||
**Demo plan:** Before moving to the guided exercise, it's worth live-editing one Deployment's
|
||||
`readinessProbe.httpGet.port` in front of the class to an obviously wrong value and running
|
||||
`kubectl get pods` / `kubectl describe pod` together, so students see the Ready column and the
|
||||
Events section change in real time before they do it themselves.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often assume a failing liveness probe and a failing readiness probe look the same in
|
||||
`kubectl get pods` — they don't (liveness failures show restart counts climbing; readiness
|
||||
failures show `0/1` Ready with no restarts). Worth calling out explicitly.
|
||||
- Students sometimes think `tcpSocket` checks are "less safe" than `httpGet` — reframe as
|
||||
"checks a different, shallower thing," not "worse."
|
||||
|
||||
**Discussion question:** "Why might a team deliberately choose a readiness probe that fails
|
||||
under high load, even though it takes healthy capacity out of rotation?" (Answer: shedding load
|
||||
from an overloaded Pod is often better than serving every request slowly or with errors.)
|
||||
|
||||
**Optional extension:** Ask students to sketch what would happen if liveness and readiness
|
||||
probes were swapped (readiness probe restarts on failure, liveness probe just removes from
|
||||
Service) — this clarifies the distinction by making the wrong version visibly worse.
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
# Demonstration: Probes in Action
|
||||
|
||||
Watch what a probe failure actually looks like from the outside, before you're the one
|
||||
diagnosing it.
|
||||
|
||||
Picture a Deployment called `web` running `nginx`, which listens on port 80. Someone adds a
|
||||
readiness probe but mistypes the port as `8080`:
|
||||
|
||||
```yaml
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /
|
||||
port: 8080
|
||||
initialDelaySeconds: 3
|
||||
periodSeconds: 5
|
||||
```
|
||||
|
||||
**What you'd see immediately after this rolls out:**
|
||||
|
||||
```
|
||||
$ kubectl get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
web-7d8f9c6b5d-abcde 0/1 Running 0 45s
|
||||
```
|
||||
|
||||
`STATUS` says `Running` — the container started fine, nginx is up. But `READY` says `0/1` — the
|
||||
readiness probe is failing, so this Pod has never been added to its Service's endpoints. No
|
||||
traffic reaches it. `RESTARTS` stays at `0` because this is a readiness failure, not a liveness
|
||||
failure — nothing is being killed, it's just being excluded.
|
||||
|
||||
**The trail that explains why**, from `kubectl describe pod`:
|
||||
|
||||
```
|
||||
Events:
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Normal Scheduled 50s default-scheduler Successfully assigned default/web-... to node-1
|
||||
Normal Pulled 49s kubelet Container image "nginx:1.25" already present on machine
|
||||
Normal Created 49s kubelet Created container web
|
||||
Normal Started 49s kubelet Started container web
|
||||
Warning Unhealthy 15s (x6 over 40s) kubelet Readiness probe failed: Get "http://10.244.0.7:8080/": dial tcp 10.244.0.7:8080: connect: connection refused
|
||||
```
|
||||
|
||||
That last line is the whole story: kubelet tried to connect to port `8080` and nothing was
|
||||
listening there — because nginx is listening on `80`. The container was never broken. The probe
|
||||
was asking the wrong question.
|
||||
|
||||
**The general diagnostic shape** for any probe issue: `kubectl get pods` tells you *that*
|
||||
something is wrong (Ready count, restart count). `kubectl describe pod` and its Events section
|
||||
tell you *what* is wrong, usually in plain English, including the exact port/path that was
|
||||
tried and the exact error. Reach for Events before logs — logs show you what the application
|
||||
did; Events show you what Kubernetes tried to do and how it failed.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Live-run this exact sequence against a throwaway Deployment (or the classroom
|
||||
projector) rather than only showing static output — `kubectl get pods`, then
|
||||
`kubectl describe pod <name>`, scrolling straight to Events. Ask the class to predict the
|
||||
`READY` column and `RESTARTS` count *before* you show it.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; ~5 minutes as a self-paced read.
|
||||
|
||||
**Common misconception to preempt:** students often reach for `kubectl logs` first when
|
||||
something isn't Ready. For a probe misconfiguration, logs are frequently silent or unhelpful —
|
||||
the application is healthy and has nothing to say. This is a good moment to reinforce
|
||||
"Events explain what Kubernetes tried; logs explain what the app did," and that reading Events
|
||||
first is usually faster for this class of failure.
|
||||
|
||||
**Optional extension:** show the same failure signature for a `tcpSocket` probe (a bare
|
||||
"connection refused" with no HTTP verb in the Events message) so students can tell the two
|
||||
mechanisms apart from Events alone.
|
||||
+56
@@ -0,0 +1,56 @@
|
||||
# Knowledge Check: Health Probes
|
||||
|
||||
Answer each question yourself before reading the answer that follows it. This isn't graded —
|
||||
it's a checkpoint to catch a misunderstanding before you're troubleshooting a live Pod in the
|
||||
next activity.
|
||||
|
||||
---
|
||||
|
||||
**1. A Pod shows `RESTARTS: 12` and `READY: 1/1`. What kind of probe failure, if any, is most
|
||||
consistent with this?**
|
||||
|
||||
*Answer:* A liveness probe that was failing and has now recovered (or is intermittently
|
||||
failing) — the climbing restart count is the signature of liveness failures, since a failed
|
||||
liveness check causes the kubelet to kill and restart the container. `READY: 1/1` means it's
|
||||
currently passing its readiness check right now.
|
||||
|
||||
---
|
||||
|
||||
**2. A Pod shows `RESTARTS: 0` and `READY: 0/1`, and has looked this way for several minutes.
|
||||
What kind of probe failure is most consistent with this?**
|
||||
|
||||
*Answer:* A readiness probe that is failing. The container is running fine (no restarts), but
|
||||
Kubernetes has decided it isn't ready to receive traffic and has excluded it from the Service's
|
||||
endpoints. This is exactly the symptom you'll investigate in this module's lab.
|
||||
|
||||
---
|
||||
|
||||
**3. Why does Kubernetes offer a separate startup probe instead of just making
|
||||
`initialDelaySeconds` on the liveness probe longer?**
|
||||
|
||||
*Answer:* `initialDelaySeconds` is a fixed, one-size-fits-all wait before checks even begin —
|
||||
set it too short and a normally-slow startup gets killed; set it too long and every fast
|
||||
startup pays that same delay before liveness/readiness protection is even active. A startup
|
||||
probe instead keeps checking until the application actually reports ready (bounded by its own
|
||||
failure threshold), so liveness/readiness only take over once startup is confirmed done —
|
||||
correct for both a consistently slow app and one whose startup time varies.
|
||||
|
||||
---
|
||||
|
||||
**4. True or false: a readiness probe failure will cause Kubernetes to restart the container.**
|
||||
|
||||
*Answer:* False. A readiness probe failure only removes the Pod from Service endpoints — it
|
||||
does not restart anything. Only a liveness (or startup, before it succeeds once) probe failure
|
||||
causes a restart.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** In a live classroom, ask these out loud and take verbal answers before
|
||||
revealing each one, rather than having students read this silently — the goal is to surface a
|
||||
wrong mental model while it's still cheap to correct.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
|
||||
**Likely misconception this catches:** question 2 specifically catches students who default to
|
||||
"something is broken, kubectl logs will show an error" — for a pure readiness-probe
|
||||
misconfiguration, logs are usually clean, which is disorienting the first time you hit it.
|
||||
+61
@@ -0,0 +1,61 @@
|
||||
# Guided Exercise: Configure a Readiness Probe
|
||||
|
||||
This is a paper-and-thinking exercise, not a Lab task — you won't run these commands against a
|
||||
real cluster here (that comes next). The goal is to practice reading and writing a probe
|
||||
definition correctly before you're diagnosing someone else's broken one.
|
||||
|
||||
## The scenario
|
||||
|
||||
You're deploying a small HTTP API that listens on port `3000` and exposes a health endpoint at
|
||||
`/healthz`. It takes a couple of seconds to finish connecting to its database on startup — if a
|
||||
request arrives before that finishes, it correctly returns a `503` at `/healthz` until it's
|
||||
ready.
|
||||
|
||||
## Your task
|
||||
|
||||
Work out, on paper (or in a scratch file), what you'd write for this container's readiness
|
||||
probe. Think through:
|
||||
|
||||
1. Which mechanism fits — `httpGet`, `tcpSocket`, or `exec`? (Hint: you have a real health
|
||||
endpoint that returns a meaningful status code — use it rather than a shallower check.)
|
||||
2. What `port` and `path` should the probe target?
|
||||
3. Roughly what `initialDelaySeconds` makes sense, given the database connection takes a couple
|
||||
of seconds?
|
||||
4. Should this be a `readinessProbe`, a `livenessProbe`, or both? Justify your answer using the
|
||||
distinction from the lesson — what actually goes wrong if you get this specific choice wrong
|
||||
here?
|
||||
|
||||
## A worked shape to check yourself against
|
||||
|
||||
Your answer should end up structurally similar to this (exact field names matter more than the
|
||||
precise numbers you chose):
|
||||
|
||||
```yaml
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 3000
|
||||
initialDelaySeconds: 3
|
||||
periodSeconds: 5
|
||||
```
|
||||
|
||||
If you reached for `livenessProbe` instead of `readinessProbe` for this specific scenario,
|
||||
re-read the lesson's distinction before continuing — "returns 503 while warming up" is
|
||||
precisely the readiness case, not the liveness case. A liveness probe here would restart a
|
||||
perfectly healthy, still-starting container over and over, which never lets it finish starting.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Have students write their probe block on paper or a shared doc before showing
|
||||
the worked shape — resist showing the answer early, since the value here is in students
|
||||
committing to an answer and comparing, not passively reading a correct one.
|
||||
|
||||
**Approximate timing:** 10-15 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Common mistake to watch for:** choosing `livenessProbe` because "the app isn't healthy yet"
|
||||
— this is the single most common real-world probe mistake and exactly what question 4 is
|
||||
designed to surface before the next activity's real diagnostic task.
|
||||
|
||||
**Optional extension:** ask what would need to change in this same YAML if the health endpoint
|
||||
took up to 60 seconds to become available on cold start (answer: a `startupProbe` with a
|
||||
generous `failureThreshold` × `periodSeconds`, so liveness/readiness don't engage too early).
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
# Launch Lab: Stuck Not-Ready: The Probe Port Mismatch
|
||||
|
||||
Time to apply everything from this module against a real, running incident — not a diagram, an
|
||||
actual Deployment in your own disposable Lab environment.
|
||||
|
||||
## What you're walking into
|
||||
|
||||
A deployment called `api-gateway` has been rolled out. The pods start, the container runs — but
|
||||
the deployment never reaches Ready, and traffic isn't being routed to it. Somewhere on-call, an
|
||||
alert is firing.
|
||||
|
||||
You now have everything you learned in this module to work with:
|
||||
|
||||
- **The lesson** told you what "stuck at 0/2 Ready with 0 restarts" actually means (a readiness
|
||||
problem, not a crash).
|
||||
- **The demonstration** showed you exactly where to look for the explanation — `kubectl get
|
||||
pods`, then `kubectl describe pod` and its Events section — and what a port-mismatch failure
|
||||
looks like in that output.
|
||||
- **The guided exercise** had you reason through what a correct probe definition looks like,
|
||||
which is exactly what you'll need to compare the broken one against.
|
||||
|
||||
## Your task in this Lab
|
||||
|
||||
Diagnose why the pods are failing their readiness check, correct the probe configuration, and
|
||||
restore the deployment to fully Ready. Start with pod status and events, the same way you just
|
||||
practiced — read before you patch.
|
||||
|
||||
This Lab has its own in-context hints if you get stuck, revealed one at a time — use the
|
||||
concepts from this module first; reach for a hint only if you're genuinely stuck, not as the
|
||||
first move.
|
||||
|
||||
When the Lab confirms your fix, come back to this module to close it out with a short
|
||||
reflection on what actually caused this.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Classroom flow:** if teaching this live, this is the natural point to have students launch
|
||||
their own Lab environment and work independently or in pairs, while you circulate. The
|
||||
Real-Time Classroom dashboard's help-request and progress views (see the instructor Curriculum
|
||||
view) show who's stuck without needing to walk around asking.
|
||||
|
||||
**Approximate timing:** 15-20 minutes for most students; some will resolve it in under 5 once
|
||||
the module's concepts have landed.
|
||||
|
||||
**Common mistake to watch for:** students jumping straight to `kubectl logs` here, which will
|
||||
be unhelpful (nginx has nothing to say about a probe pointed at the wrong port) — this is a good
|
||||
moment to reinforce the demonstration's point about reading Events first for this failure class.
|
||||
+61
@@ -0,0 +1,61 @@
|
||||
# Root-Cause Reflection & Real Ops Takeaway
|
||||
|
||||
You just fixed a real incident pattern, not a puzzle. Before moving on, take a few minutes to
|
||||
think through what actually happened and why it matters beyond this one Lab.
|
||||
|
||||
## What actually went wrong
|
||||
|
||||
The container was never broken. `nginx` started fine and was serving requests correctly the
|
||||
entire time. The only thing wrong was that the readiness probe was configured to check a port
|
||||
nginx wasn't listening on — so Kubernetes, working exactly as designed, correctly reported the
|
||||
Pod as Not Ready based on the information it had. The system didn't fail; it was given wrong
|
||||
information and acted rationally on it.
|
||||
|
||||
This is worth sitting with: most "Kubernetes is broken" incidents are actually "Kubernetes is
|
||||
doing exactly what its configuration tells it to do, and the configuration is wrong."
|
||||
|
||||
## Reflection questions
|
||||
|
||||
Think through your own answers to these — there's no single correct wording, but you should be
|
||||
able to explain each one out loud:
|
||||
|
||||
1. If you had jumped straight to `kubectl logs` instead of `kubectl describe pod`, how long do
|
||||
you think it would have taken you to find this? What would the logs have actually shown you?
|
||||
2. This Lab's port mismatch was introduced deliberately. What real-world change could introduce
|
||||
the exact same failure by accident? (Consider: someone changes the container's listening
|
||||
port in a new image version, but forgets the probe definition was hand-written separately.)
|
||||
3. Why does a rolling update stall — not fail outright, just hang — when new Pods can't pass
|
||||
their readiness probe? What is Kubernetes protecting you from by refusing to proceed?
|
||||
|
||||
## Real Ops takeaway: prevention, not just fixes
|
||||
|
||||
In a real production system, you'd want to prevent this class of incident from recurring, not
|
||||
just fix it once:
|
||||
|
||||
- **Keep the probe port derived from the same source as the container's actual port**, where
|
||||
your tooling allows it (a shared value in a Helm chart or Kustomize overlay), instead of two
|
||||
independently hand-typed numbers that can drift apart silently.
|
||||
- **Treat a stalled rollout as a first-class alert**, not just "the deploy is taking a while" —
|
||||
a Deployment sitting at partial readiness for an unusual length of time is exactly this
|
||||
failure mode, and catching it via rollout status is faster than waiting for a user-facing
|
||||
outage report.
|
||||
- **Practice reading Events before logs** for anything in the "Running but not Ready" state —
|
||||
you just did this under low stakes; the habit is what pays off during a real incident at 2am.
|
||||
|
||||
This module is complete. Return to the course to continue with the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** use the three reflection questions above as
|
||||
discussion prompts rather than a silent worksheet — question 2 in particular tends to produce
|
||||
good real-world examples from students who've touched CI/CD pipelines before.
|
||||
|
||||
**Approximate timing:** 10 minutes for a live discussion; 5-8 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to write the one-sentence postmortem summary they'd post
|
||||
in a team channel for this incident (real on-call teams write these) — a good answer names the
|
||||
root cause (probe port mismatch) and the fix (corrected port), not just "it's fixed now."
|
||||
|
||||
**Why this activity is a reflection, not another knowledge check:** the goal here is
|
||||
consolidation and transfer to real-world judgment, not testing recall — resist turning this
|
||||
into another right/wrong quiz.
|
||||
+92
@@ -0,0 +1,92 @@
|
||||
# The RootNode Troubleshooting Framework
|
||||
|
||||
## Why a framework, not just "get good at kubectl"
|
||||
|
||||
The gold-standard Health Probes module taught you a lot about ONE specific failure mode
|
||||
(readiness probe misconfiguration) and the specific commands to diagnose it. This module steps
|
||||
back and asks: what's the repeatable PROCESS that works for almost any Kubernetes failure, not
|
||||
just the ones you've already seen before?
|
||||
|
||||
Guessing under pressure — trying random fixes and seeing what sticks — is slow, often makes
|
||||
things worse, and doesn't build a transferable skill. A framework does.
|
||||
|
||||
## Observe → Gather Evidence → Form Hypothesis → Test → Repair → Validate → Prevent Recurrence
|
||||
|
||||
**Observe** — what's the actual, specific symptom? Not "it's broken," but "the Deployment shows
|
||||
0/3 Ready" or "requests are timing out" or "the Pod restarts every 30 seconds." Precision here
|
||||
saves time later.
|
||||
|
||||
**Gather Evidence** — before touching anything, collect what the system itself is telling you:
|
||||
`kubectl get pods`, `kubectl describe pod`, `kubectl logs`, `kubectl get events`. Resist the urge
|
||||
to start changing things before you've looked.
|
||||
|
||||
**Form Hypothesis** — based on the evidence, state a SPECIFIC guess about the cause. "The
|
||||
container is crashing because the command it's told to run doesn't exist" is testable. "Something's
|
||||
wrong with networking" is not specific enough to act on yet.
|
||||
|
||||
**Test** — check whether your hypothesis is actually true, ideally without yet making the fix
|
||||
(read the exact command being run, compare against what the image expects) — confirm before you
|
||||
commit to a repair.
|
||||
|
||||
**Repair** — make the minimal change that addresses the confirmed cause. Not a bigger change
|
||||
"while you're in there" — exactly the fix the evidence supports.
|
||||
|
||||
**Validate** — confirm the fix actually worked, using the same kind of evidence you gathered
|
||||
initially (not just "I made a change, it should be fine now").
|
||||
|
||||
**Prevent Recurrence** — the step most often skipped under time pressure. Could this class of
|
||||
mistake happen again? Is there a smaller, structural change (a default, a validation, a check)
|
||||
that would catch it earlier next time?
|
||||
|
||||
## Applying it: a CrashLoopBackOff
|
||||
|
||||
- **Observe:** `crash-app`'s Pods show `STATUS: CrashLoopBackOff`, restart count climbing.
|
||||
- **Gather Evidence:** `kubectl describe deployment crash-app` (check the configured command),
|
||||
`kubectl logs -l app=crash-app` (what did the container actually print before dying, if
|
||||
anything).
|
||||
- **Form Hypothesis:** "The command override is invalid/wrong for this image, so the container
|
||||
exits immediately every time it starts."
|
||||
- **Test:** compare the command actually configured against what the image's default/expected
|
||||
entrypoint should be.
|
||||
- **Repair:** fix the command to what the container image actually expects.
|
||||
- **Validate:** `kubectl get pods` shows `Running`, restart count stops climbing.
|
||||
- **Prevent Recurrence:** a code-review habit of never overriding a container's command without
|
||||
verifying it against the image's documentation first.
|
||||
|
||||
## Applying it: a broken Service selector
|
||||
|
||||
- **Observe:** traffic isn't reaching `my-svc`'s backend at all.
|
||||
- **Gather Evidence:** `kubectl get endpoints my-svc` (empty?), `kubectl get pods
|
||||
--show-labels` (what labels do the actual Pods have?).
|
||||
- **Form Hypothesis:** "The Service's selector doesn't match the Pods' actual labels."
|
||||
- **Test:** compare `kubectl describe service my-svc`'s selector against the Pods' real labels
|
||||
directly, side by side.
|
||||
- **Repair:** correct the selector to match.
|
||||
- **Validate:** `kubectl get endpoints my-svc` now shows the Pod IPs.
|
||||
- **Prevent Recurrence:** keeping label keys/values consistent (or generated from one source)
|
||||
between a Deployment's Pod template and any Service that targets it.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10-12 minutes live walking through both worked
|
||||
examples.
|
||||
|
||||
**Demo plan:** Use the demonstration activity right after this lesson to show the CrashLoopBackOff
|
||||
example live, narrating each framework step explicitly as you go rather than just running
|
||||
commands.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students under time pressure often skip straight to "Repair" without confirming a hypothesis
|
||||
first. Reinforce that a wrong guess implemented as a "fix" can make evidence harder to read
|
||||
afterward, not easier.
|
||||
- "Prevent Recurrence" is the step most likely to be treated as optional/skippable. Frame it as
|
||||
what separates a team that keeps hitting the same incident from one that doesn't.
|
||||
|
||||
**Discussion question:** "What's the cost of skipping 'Gather Evidence' and jumping straight to
|
||||
a guessed fix, even if the guess happens to be right?" (Answer: even a correct guess doesn't
|
||||
teach you whether your reasoning was right — you got lucky, and the same class of mistake could
|
||||
still recur since you never confirmed the actual cause.)
|
||||
|
||||
**Optional extension:** ask students to apply this exact framework, from memory, to the gold
|
||||
module's `rootnode-probe-port-mismatch` incident they already solved — a good retroactive
|
||||
exercise connecting a lab they already completed to the generalized framework.
|
||||
+91
@@ -0,0 +1,91 @@
|
||||
# Demonstration: Applying the Framework to a CrashLoopBackOff
|
||||
|
||||
Watch the full Observe → Evidence → Hypothesis → Test → Repair → Validate → Prevent sequence
|
||||
against a real symptom, narrated step by step.
|
||||
|
||||
## Observe
|
||||
|
||||
```
|
||||
$ kubectl get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
crash-app-6d9f8b7c5d-abcde 0/1 CrashLoopBackOff 5 4m
|
||||
```
|
||||
|
||||
Specific symptom: `CrashLoopBackOff`, restart count climbing (5 and counting) — the container
|
||||
is starting, exiting, and being restarted repeatedly, not just failing to start once.
|
||||
|
||||
## Gather Evidence
|
||||
|
||||
```
|
||||
$ kubectl logs crash-app-6d9f8b7c5d-abcde
|
||||
nginx: [emerg] invalid command "daemon-off": expected valid command
|
||||
```
|
||||
|
||||
```
|
||||
$ kubectl describe deployment crash-app
|
||||
...
|
||||
Command:
|
||||
nginx
|
||||
-g
|
||||
daemon-off;
|
||||
...
|
||||
```
|
||||
|
||||
The logs show nginx itself rejecting the command it was given. The Deployment's configured
|
||||
command shows exactly why: `daemon-off;` (a typo — missing space, should be `daemon off;`).
|
||||
|
||||
## Form Hypothesis
|
||||
|
||||
"The container's command override has a syntax error that nginx can't parse, so nginx exits
|
||||
immediately every time it starts — which is why it's crash-looping rather than just failing to
|
||||
start once."
|
||||
|
||||
## Test
|
||||
|
||||
Compare the broken command (`daemon-off;`) against nginx's actual expected syntax
|
||||
(`daemon off;`, two separate words) — confirmed: this is exactly the kind of typo that would
|
||||
produce exactly this error.
|
||||
|
||||
## Repair
|
||||
|
||||
```
|
||||
$ kubectl patch deployment crash-app --type='json' \
|
||||
-p='[{"op":"replace","path":"/spec/template/spec/containers/0/command","value":["nginx","-g","daemon off;"]}]'
|
||||
```
|
||||
|
||||
## Validate
|
||||
|
||||
```
|
||||
$ kubectl get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
crash-app-7f8c9d6b5d-fghij 1/1 Running 0 15s
|
||||
```
|
||||
|
||||
New Pod, `Running`, `RESTARTS: 0` — the fix is confirmed, not just assumed.
|
||||
|
||||
## Prevent Recurrence
|
||||
|
||||
A command override like this is a hand-typed string with no validation until the container
|
||||
actually tries to run it. A team that hit this once might decide: don't override a well-known
|
||||
image's default command unless there's a specific reason to, and if you must, test the exact
|
||||
command string locally (`docker run` or similar) before deploying it — catching a typo in
|
||||
seconds locally beats debugging a CrashLoopBackOff in the cluster.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Run this live if time allows, deliberately narrating which framework step
|
||||
you're on at each command — students retain the framework far better tied to real commands than
|
||||
as an abstract list.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; 5-7 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students sometimes skip straight from "Observe" to
|
||||
"Repair" once they spot the typo, without stating a hypothesis or testing it. Reinforce that
|
||||
even an "obvious" fix benefits from the explicit hypothesis step — it's what makes the eventual
|
||||
Validate step meaningful (you're confirming a SPECIFIC claim, not just "trying something").
|
||||
|
||||
**Optional extension:** ask what evidence you'd gather differently if the logs had been
|
||||
completely empty instead of showing a clear nginx error (a good bridge to cases where the
|
||||
container dies before it can log anything at all — `kubectl describe pod`'s Events section
|
||||
becomes the primary evidence source in that case, same as the gold module's probe-mismatch
|
||||
lesson).
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
# Knowledge Check: The Troubleshooting Framework
|
||||
|
||||
---
|
||||
|
||||
**1. You spot what looks like an obvious typo causing a failure and fix it immediately without
|
||||
writing down a hypothesis first. What's the actual cost of skipping that step, even if the fix
|
||||
works?**
|
||||
|
||||
*Answer:* You don't get a confirmed, testable claim about the cause — if the fix happens to
|
||||
"work" for an unrelated reason (a coincidental restart, a different underlying issue also
|
||||
resolving), you'd have no way to distinguish that from your fix actually being correct. The
|
||||
hypothesis step is what makes Validate meaningful.
|
||||
|
||||
---
|
||||
|
||||
**2. During the "Gather Evidence" step for a CrashLoopBackOff, which two commands give you the
|
||||
most direct evidence, and what does each one tell you that the other doesn't?**
|
||||
|
||||
*Answer:* `kubectl logs` (what the container itself printed, if anything, before exiting) and
|
||||
`kubectl describe pod`/`describe deployment` (the container's actual configuration, plus recent
|
||||
Events). Logs show application-level output; describe/Events show what Kubernetes itself
|
||||
observed and configured — a crash with no logs at all often means the failure happened before
|
||||
the application could log anything, which describe/Events can still reveal.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: "Prevent Recurrence" only applies to major incidents, not small
|
||||
one-off mistakes like a typo.**
|
||||
|
||||
*Answer:* False. A small mistake that recurs repeatedly across a team becomes a real, ongoing
|
||||
cost even if no single instance is severe — the framework applies the same reasoning regardless
|
||||
of incident size: could a small, structural change catch this earlier next time?
|
||||
|
||||
---
|
||||
|
||||
**4. A broken Service selector (endpoint-fix-task's scenario) and a CrashLoopBackOff
|
||||
(crashloop-fix's scenario) are very different failures. What's the SAME about how you'd
|
||||
approach both?**
|
||||
|
||||
*Answer:* The same seven-step framework — Observe the specific symptom, Gather Evidence before
|
||||
guessing, form a specific testable Hypothesis, Test it, make the minimal Repair the evidence
|
||||
supports, Validate with the same kind of evidence you started with, and consider how to Prevent
|
||||
Recurrence. The framework doesn't change; only the specific commands and evidence do.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 4 is the whole point of this module — make sure it's discussed even
|
||||
if answered correctly, since it's the transferability claim the entire framework rests on.
|
||||
|
||||
**Approximate timing:** 5-7 minutes.
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
# Guided Exercise: Applying the Framework to a Broken Service, on Paper First
|
||||
|
||||
Before the `endpoint-fix-task` Lab, walk through the framework yourself against a described
|
||||
symptom — no cluster needed yet.
|
||||
|
||||
## The scenario
|
||||
|
||||
A teammate says: "`my-svc` isn't routing any traffic to our Pods, and I don't know why — the
|
||||
Pods look healthy in `kubectl get pods`."
|
||||
|
||||
## Your task
|
||||
|
||||
Write out, step by step, what you would actually DO for each framework stage, before looking at
|
||||
the worked answer:
|
||||
|
||||
1. **Observe** — what's the precise, specific symptom here (not "it's broken")?
|
||||
2. **Gather Evidence** — name the exact `kubectl` commands you'd run, in order, and what each
|
||||
one is checking for.
|
||||
3. **Form Hypothesis** — based on what ROOT 07's Services lesson taught you, what's the most
|
||||
likely cause of "Pods healthy, but Service routes nowhere"?
|
||||
4. **Test** — how would you CONFIRM that hypothesis before changing anything?
|
||||
5. **Repair** — what's the minimal fix, assuming your hypothesis is correct?
|
||||
6. **Validate** — what would you check afterward to PROVE the fix worked, not just assume it?
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. Traffic sent to `my-svc` never reaches any Pod, despite the Pods themselves reporting
|
||||
`Running`/`Ready` — this rules out "the Pods are unhealthy" as the cause.
|
||||
2. `kubectl get endpoints my-svc` (does it show any Pod IPs at all?), then `kubectl describe
|
||||
service my-svc` (what selector is it actually using?), then `kubectl get pods --show-labels`
|
||||
(what labels do the real Pods actually carry?).
|
||||
3. A Service-to-Pod label selector mismatch — exactly the failure mode ROOT 07's demonstration
|
||||
walked through, where a Service's selector no longer matches the Pods' actual labels.
|
||||
4. Directly compare the Service's selector (from `describe service`) against the Pods' labels
|
||||
(from `get pods --show-labels`) side by side — confirmed if they genuinely don't match.
|
||||
5. Correct the Service's selector (or the Pods' labels, whichever is actually wrong) so they
|
||||
match again.
|
||||
6. `kubectl get endpoints my-svc` now shows the Pod IPs populated — not just "the YAML looks
|
||||
right now," but the actual Endpoints object reflecting real, matched Pods.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** This exercise deliberately reuses ROOT 07's own content — a good moment to
|
||||
point out explicitly that the framework's value is applying PRIOR knowledge systematically, not
|
||||
learning something entirely new each time.
|
||||
|
||||
**Approximate timing:** 10-12 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what would change about steps 2-4 if the symptom were instead
|
||||
"Pods healthy, but ONE specific caller can't reach the Service while others can" — a good bridge
|
||||
to considering NetworkPolicy (ROOT 09) as an alternative hypothesis once the simpler
|
||||
selector-mismatch explanation is ruled out.
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
# Reflection: A Framework You've Actually Used Is One You'll Trust Under Pressure
|
||||
|
||||
You just solved two real, different failures — a CrashLoopBackOff and a broken Service
|
||||
selector — using the same seven-step framework both times. Take a few minutes to consolidate
|
||||
this.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Which framework step did you personally find easiest to skip or rush through across both
|
||||
Labs? Why do you think that step specifically tempts shortcuts?
|
||||
2. Compare how you approached `crashloop-fix` versus `endpoint-fix-task`. What was IDENTICAL
|
||||
about your process, and what was different only because the evidence looked different?
|
||||
3. Now that you've practiced this framework on two failures you'd never seen exact instructions
|
||||
for, how confident do you feel applying it to a THIRD, completely unfamiliar Kubernetes
|
||||
failure? What would you still want to double-check?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**The framework's value shows up most on failures you've never seen before.** Anyone can
|
||||
memorize "if X symptom, do Y fix" for a handful of common problems — that knowledge doesn't
|
||||
transfer. Observe → Evidence → Hypothesis → Test → Repair → Validate → Prevent Recurrence
|
||||
transfers to failures nobody has specifically taught you, because it's a PROCESS, not a lookup
|
||||
table.
|
||||
|
||||
**"Prevent Recurrence" is what separates incident response from firefighting.** Both Labs in
|
||||
this module had a real, nameable prevention practice (verify command strings before deploying;
|
||||
keep label keys consistent between a Deployment and its Service). Teams that consistently ask
|
||||
this question after every fix accumulate fewer repeat incidents over time — not because they got
|
||||
lucky, but because they closed the actual gap each time.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 1 tends to produce honest, useful
|
||||
answers — "Prevent Recurrence" and "Form Hypothesis" (skipping straight to a guessed fix) are
|
||||
the two most commonly admitted shortcuts, both worth normalizing as a real temptation to resist
|
||||
rather than treating as a personal failing.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5-7 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to apply the framework, from memory and without re-reading
|
||||
this module, to a THIRD failure type they haven't practiced yet (e.g., a Pod stuck `Pending` —
|
||||
not built as a hands-on Lab in this module, just a thought exercise) to test genuine
|
||||
transferability.
|
||||
+88
@@ -0,0 +1,88 @@
|
||||
# DevOps as an Operating Model, Not a Job Title
|
||||
|
||||
## You've already been doing this — just not naming it yet
|
||||
|
||||
Every course before this one taught you a Kubernetes skill: declaring desired state, hardening
|
||||
a Pod, diagnosing a failure with a repeatable framework, running an incident with real
|
||||
discipline. This module doesn't teach a new hands-on skill — it names the OPERATING MODEL those
|
||||
skills are all part of, so you can recognize it and apply it beyond Kubernetes specifically.
|
||||
|
||||
## DevOps: a set of practices, not a team you join
|
||||
|
||||
"DevOps" is often misused as a job title ("I'm a DevOps engineer") or a team name ("throw it
|
||||
over to DevOps"). Properly understood, DevOps is a set of PRACTICES for how development and
|
||||
operations work together: shorter feedback loops, automation over manual process, and shared
|
||||
ownership of what happens after code ships — not a separate team that operations gets outsourced
|
||||
to.
|
||||
|
||||
Concretely, you already practiced core DevOps ideas without the label:
|
||||
|
||||
- **Declarative thinking** (ROOT 05) — describing desired state and letting a system enforce it,
|
||||
instead of manually executing steps, is the same principle behind Infrastructure as Code.
|
||||
- **Fast, safe rollback** (ROOT 06) — treating recovery as a rehearsed, ordinary action rather
|
||||
than a crisis is a direct DevOps practice (fast feedback, low-drama recovery).
|
||||
- **A repeatable troubleshooting framework** (ROOT 10) and **structured incident response**
|
||||
(ROOT 12) — both are operational discipline, the "Ops" half of DevOps, practiced for real.
|
||||
|
||||
## Platform Engineering: what DevOps looks like at scale
|
||||
|
||||
**Platform engineering** is the next layer: instead of every application team reinventing how
|
||||
to deploy safely, monitor, and secure their own workloads, a platform team builds SELF-SERVICE
|
||||
tooling and golden paths that let other engineers move fast without needing deep infrastructure
|
||||
expertise themselves. The Kubernetes cluster, the CI/CD pipeline, the observability stack — all
|
||||
of it becomes a PRODUCT a platform team builds for INTERNAL customers (other engineers), not
|
||||
just infrastructure that happens to exist.
|
||||
|
||||
A useful lens: platform engineering exists to **reduce cognitive load**. A developer who can
|
||||
deploy a well-configured, secure, observable service through a simple, guided path doesn't need
|
||||
to personally understand every Kubernetes concept from ROOT 05-10 to ship safely — the platform
|
||||
encodes that expertise once, for everyone.
|
||||
|
||||
## Where Kubernetes fits in the larger picture
|
||||
|
||||
Kubernetes is INFRASTRUCTURE, not the whole platform. A real platform engineering practice
|
||||
typically layers several things around it:
|
||||
|
||||
- **Declarative delivery** — describing what should be deployed and letting automation apply it
|
||||
(the GitOps model — future ROOT 11 content, requires a Git+deployment-automation environment
|
||||
this course doesn't yet provide).
|
||||
- **Continuous integration/delivery** — automatically testing and shipping code changes (CI/CD —
|
||||
future content, requires a CI runner environment).
|
||||
- **Infrastructure as Code** — declaring cloud/infrastructure resources the same way you declare
|
||||
Kubernetes objects (future content, requires a provisioning environment like Terraform).
|
||||
- **Observability** — knowing what's actually happening in production beyond "is the Pod
|
||||
Running" (future content, requires a metrics/logging/tracing stack).
|
||||
|
||||
None of these require abandoning what you learned in ROOT 05-10 — they build directly on the
|
||||
same mental models (desired vs. actual state, evidence-based diagnosis, least privilege) applied
|
||||
to a wider set of systems.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Lesson goal:** Give students a name and a framework for practices they've already exercised,
|
||||
and an honest map of what ROOT 11 will eventually cover once appropriate lab environments exist —
|
||||
without pretending those labs exist today.
|
||||
|
||||
**Approximate timing:** 20 minutes read; 15 minutes live with discussion.
|
||||
|
||||
**Preparation:** None beyond having read `docs/ROOT11_DEVOPS_PLATFORM_SCOPE.md` to speak
|
||||
accurately about what's planned versus what exists.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- "DevOps" as a job title/team is a very common prior belief — correct this directly and early,
|
||||
it's the central point of the lesson.
|
||||
- Platform engineering is sometimes conflated with "just Kubernetes." Emphasize the self-service/
|
||||
golden-path angle — the PRODUCT framing, not just the technology.
|
||||
|
||||
**Questions to ask students:** "Which ROOT_ course so far most directly practiced a DevOps
|
||||
principle, even though it was never labeled that way?" (Most will cite ROOT 06's rollback
|
||||
discipline or ROOT 12's incident framework — both are legitimate, good answers.)
|
||||
|
||||
**When to let a student struggle vs. provide a hint:** Not applicable — this is an orientation
|
||||
lesson with no hands-on task.
|
||||
|
||||
**Optional extension:** ask students to name one manual process at a company they've worked for
|
||||
(or heard about) that platform engineering-style self-service tooling could have eliminated.
|
||||
|
||||
**Expected takeaway:** DevOps and platform engineering are the organizing ideas behind almost
|
||||
everything ROOT_ has already taught — not a new, separate subject to learn from scratch.
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
# Knowledge Check: DevOps & Platform Thinking
|
||||
|
||||
---
|
||||
|
||||
**1. A colleague says "DevOps is the team that handles our servers." What's wrong with that
|
||||
statement?**
|
||||
|
||||
*Answer:* DevOps describes a set of PRACTICES (shared ownership, fast feedback loops,
|
||||
automation) for how development and operations work together — not a separate team that
|
||||
operations gets handed off to. Treating it as a team name misses the point that these practices
|
||||
are meant to be shared across engineering, not isolated in one group.
|
||||
|
||||
---
|
||||
|
||||
**2. Which ROOT_ course's central skill is most directly an example of "declarative thinking"
|
||||
applied outside Kubernetes objects specifically?**
|
||||
|
||||
*Answer:* ROOT 05's desired-state model is the direct root of declarative thinking generally —
|
||||
Infrastructure as Code applies the exact same principle (describe what should exist, let
|
||||
automation enforce it) to cloud infrastructure rather than Kubernetes objects specifically.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: platform engineering's main goal is building the most technically
|
||||
sophisticated internal tooling possible.**
|
||||
|
||||
*Answer:* False. Its main goal is reducing COGNITIVE LOAD for other engineers via self-service,
|
||||
golden-path tooling — sophistication that doesn't make another engineer's job easier isn't the
|
||||
point, even if it's technically impressive.
|
||||
|
||||
---
|
||||
|
||||
**4. Why can't this course build a hands-on GitOps or CI/CD Lab the same way ROOT 05-10 built
|
||||
Kubernetes Labs?**
|
||||
|
||||
*Answer:* Those require different infrastructure the current Lab platform doesn't provide (a
|
||||
Git repository + deployment-automation tool like ArgoCD for GitOps; a CI runner for CI/CD) — see
|
||||
`docs/ROOT11_DEVOPS_PLATFORM_SCOPE.md`. Building a fake version of either would teach an
|
||||
inaccurate mental model, which is worse than honestly deferring the hands-on practice.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 4 is worth stating plainly in a live classroom — students should
|
||||
understand this is a genuine platform limitation being handled honestly, not an oversight.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
# Reflection: Where You Already Practiced This
|
||||
|
||||
You've now named the operating model behind most of what ROOT_ taught. Take a few minutes to
|
||||
trace it back through your own experience in this program.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Pick one specific moment from ROOT 05-10 or ROOT 12 where you practiced a DevOps/platform
|
||||
engineering principle without it being labeled that way. Name the principle explicitly now.
|
||||
2. If you were designing a platform team's "golden path" for deploying a new service safely,
|
||||
which THREE things from ROOT_ (health probes, RBAC, resource limits, NetworkPolicy, etc.)
|
||||
would you bake in as defaults rather than leave to each team to remember?
|
||||
3. What's one thing about GitOps, CI/CD, Infrastructure as Code, or observability you're
|
||||
genuinely curious about, now that you have the Kubernetes foundation to eventually build
|
||||
hands-on skill in it?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**The skills you already have transfer further than "just Kubernetes."** Desired-state
|
||||
thinking, least privilege, evidence-based troubleshooting, and structured incident response
|
||||
aren't Kubernetes-specific facts — they're how experienced platform engineers reason about ANY
|
||||
complex system. Recognizing that transfer is what turns "I learned Kubernetes" into "I think
|
||||
like a platform engineer."
|
||||
|
||||
**Honest scope beats fake completeness.** This course didn't pretend to teach you GitOps or
|
||||
CI/CD hands-on today — it told you exactly what's missing and why, and pointed at what it would
|
||||
take to build it for real. That's the same standard the rest of ROOT_ held itself to
|
||||
throughout — a module that doesn't exist yet is more useful, honestly labeled, than one that
|
||||
exists in name only.
|
||||
|
||||
This module is complete.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 2 works well as a small-group exercise —
|
||||
have groups compare their "golden path defaults" lists and discuss disagreements.
|
||||
|
||||
**Approximate timing:** 10 minutes live discussion; 8 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to research (outside class) one real company's public
|
||||
engineering blog post about their platform engineering practice, and bring one concrete example
|
||||
to share next session.
|
||||
|
||||
**Expected takeaway:** ROOT_ built real, transferable engineering judgment — this module makes
|
||||
that transfer explicit rather than leaving students to discover it on their own.
|
||||
@@ -0,0 +1,88 @@
|
||||
# The RootNode Incident-Response Framework
|
||||
|
||||
## This module feels different on purpose
|
||||
|
||||
Everything before ROOT 12 asked you to complete an exercise. This module asks something
|
||||
different: **you are responsible for this environment.** The incidents in this module are
|
||||
modeled on real operational failures — not textbook exercises designed to teach one clean
|
||||
concept, but the kind of messy, "why is this actually broken" situations a real on-call engineer
|
||||
gets paged for.
|
||||
|
||||
## Detect → Triage → Diagnose → Mitigate → Recover → Verify → Learn
|
||||
|
||||
This is a lightweight incident-response framework — not enterprise incident management
|
||||
tooling, just a way of thinking that scales down to "one engineer, one broken thing" as well as
|
||||
it scales up to a real incident-response team.
|
||||
|
||||
**Detect** — something alerted you (a monitoring page, a user report, or in this Lab's case,
|
||||
the scenario telling you something is wrong). You know an incident exists before you know what
|
||||
it is.
|
||||
|
||||
**Triage** — how bad is this, right now, before you dive into root cause? Is anything actively
|
||||
getting worse? Is this affecting one thing or cascading into others? Triage decides how urgently
|
||||
you act, before you fully understand why.
|
||||
|
||||
**Diagnose** — this is where ROOT 10's troubleshooting framework (Observe, Gather Evidence, Form
|
||||
Hypothesis, Test) does its work — the specific mechanics of finding the actual cause.
|
||||
|
||||
**Mitigate** — the fastest safe action that stops things from getting worse, which is not always
|
||||
the same as the final fix. Sometimes mitigation and repair are the same step; sometimes
|
||||
mitigation buys time (e.g. temporarily reducing load) while a proper fix is prepared.
|
||||
|
||||
**Recover** — restore the system to its actual working state — not just "stopped getting worse"
|
||||
but genuinely operational again.
|
||||
|
||||
**Verify** — confirm recovery with real evidence, the same discipline as ROOT 10's Validate
|
||||
step. "I think it's fixed" is not the same as checking.
|
||||
|
||||
**Learn** — what would prevent this exact class of incident from happening again? This is where
|
||||
`rootnode-enforce-limits` picks up directly after `rootnode-runaway-monitor` — not a separate
|
||||
topic, but the deliberate "Learn" step of the SAME incident, made into its own scenario.
|
||||
|
||||
## What this module will NOT tell you
|
||||
|
||||
The specific fixes for each incident are not in this lesson, and instructor notes on the
|
||||
individual incident activities go deeper than what's shown here — this module gives you the
|
||||
FRAMEWORK for approaching an unfamiliar incident, not a cheat sheet for these specific four.
|
||||
That's deliberate: a framework you had to apply yourself is one you'll actually remember under
|
||||
real pressure.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Lesson goal:** Set the tone shift for this entire module — from "complete an exercise" to
|
||||
"you own this." This lesson is short by design; the real teaching happens through the incidents
|
||||
themselves.
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live framing discussion.
|
||||
|
||||
**Preparation:** Read each incident's own instructor notes (in the Lab activities below) before
|
||||
teaching this module live — several contain deeper solution guidance not repeated here,
|
||||
specifically so students still have to work for it.
|
||||
|
||||
**Demonstration plan:** Not applicable to this framing lesson — the four incidents themselves
|
||||
serve as the demonstrations.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students sometimes treat "Mitigate" and "Recover" as the same step. They can be, but aren't
|
||||
always — worth naming a real-world example (rolling back a bad deploy mitigates immediately;
|
||||
fully recovering might still require redeploying the CORRECT version afterward).
|
||||
- "Learn" is the step most likely to get skipped once a Lab is marked complete. Explicitly
|
||||
connect it to why `rootnode-enforce-limits` exists as its own scenario.
|
||||
|
||||
**Questions to ask students:** "Which step of this framework do you think real teams are worst
|
||||
at, under real pressure, and why?" (There's no single right answer — the goal is getting
|
||||
students to reason about incident response as a discipline, not just a checklist.)
|
||||
|
||||
**When to let a student struggle vs. provide a hint:** These incidents are intentionally less
|
||||
guided than earlier modules. Let a student sit with "I don't know where to start" for a few
|
||||
minutes before nudging — the discomfort of a genuinely unfamiliar failure is part of what this
|
||||
module is teaching. Redirect toward the Diagnose step's ROOT 10 framework if truly stuck, rather
|
||||
than pointing at the specific fix.
|
||||
|
||||
**Optional extension:** ask students to write a two-sentence incident summary for
|
||||
`rootnode-runaway-monitor` as if posting it to a real team channel — Detect/Diagnose/Recover in
|
||||
plain language, no jargon.
|
||||
|
||||
**Expected takeaway:** A framework applied under pressure beats improvisation, even for
|
||||
engineers who are technically skilled enough to eventually stumble into the right answer anyway
|
||||
— the framework is what makes the RIGHT answer arrive faster and more reliably.
|
||||
@@ -0,0 +1,47 @@
|
||||
# Post-Incident Reflection: What Real Ops Actually Feels Like
|
||||
|
||||
You just worked through four incidents that "own it, don't just complete it" framing described.
|
||||
Take a few minutes to reflect on the module as a whole, not just any single incident.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Which of the four incidents took the longest to even understand what was actually wrong,
|
||||
before you got to fixing anything? What made diagnosis slower there than the others?
|
||||
2. `rootnode-runaway-monitor` and `rootnode-enforce-limits` are the same underlying incident,
|
||||
split into "fix it now" and "prevent it forever." Which one did you find more satisfying to
|
||||
solve, and why do you think real teams sometimes skip the second one under time pressure?
|
||||
3. If you were handing these four incidents off to a teammate mid-shift, what's the one-sentence
|
||||
summary you'd give for each, so they could pick up without re-diagnosing from scratch?
|
||||
4. Compare how this module felt to the earlier, more guided modules in ROOT_. What specifically
|
||||
was different about the experience, not just the difficulty?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**"It's fixed" and "it won't happen again" are two different jobs, and skipping the second one
|
||||
is how the same incident recurs.** `rootnode-enforce-limits` exists as its own scenario
|
||||
specifically to make this point structurally, not just rhetorically — the immediate fix and the
|
||||
prevention work are genuinely separate efforts, and a real team's incident isn't actually closed
|
||||
until both are done.
|
||||
|
||||
**Being handed an unfamiliar, under-specified problem is the actual job.** Every other module in
|
||||
ROOT_ told you roughly what concept you were about to practice. This one didn't — and that gap
|
||||
is deliberate, because real production incidents don't come with a syllabus telling you which
|
||||
concept they're testing. The framework from this module's opening lesson is what you reach for
|
||||
instead of a hint system.
|
||||
|
||||
This module is complete. If a Capstone is part of your program, its readiness module is next.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 2 is the module's core point — spend
|
||||
real time on it rather than treating it as a quick check-the-box question.
|
||||
|
||||
**Approximate timing:** 10-12 minutes live discussion; 8 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to identify which incident from this module most closely
|
||||
resembles something they'd expect to encounter in an actual job, and why — grounds the module in
|
||||
something beyond the classroom.
|
||||
|
||||
**Expected takeaway:** Real operational work is less about knowing every possible failure mode
|
||||
in advance and more about having a repeatable process for approaching the ones you've never
|
||||
seen before.
|
||||
@@ -0,0 +1,72 @@
|
||||
# What the Capstone Measures & How the Draw Works
|
||||
|
||||
## What the RootNode Capstone is actually testing
|
||||
|
||||
The Capstone is not a test of memorized `kubectl` syntax. It's a 90-minute operational
|
||||
assessment weighted heavily toward RootNode's own Real Ops incident scenarios — it's checking
|
||||
whether you can DO the thing ROOT_ has been building toward: diagnose and repair real,
|
||||
under-specified problems using the frameworks and skills from every course before it, under a
|
||||
clock, without step-by-step guidance.
|
||||
|
||||
If you've completed ROOT 05 through ROOT 12 honestly (not just clicked through), the Capstone is
|
||||
testing exactly what you've already practiced — just without a lesson telling you which concept
|
||||
is about to be tested.
|
||||
|
||||
## How the scenario draw works
|
||||
|
||||
The Capstone draws a mix of:
|
||||
|
||||
- **Required scenarios** — every attempt includes RootNode's own Real Ops incidents (the same
|
||||
scenarios from ROOT 12, though a fresh attempt against them, not literally the same completed
|
||||
attempt carried over).
|
||||
- **Randomly drawn scenarios** — a smaller number of additional scenarios, randomly selected
|
||||
from broader pools spanning workloads, configuration, and troubleshooting.
|
||||
|
||||
This means two students taking the Capstone will very likely see a different exact mix of
|
||||
random scenarios, while both always face the same core Real Ops incidents. The exact scenario
|
||||
IDs and the random draw itself are never published in advance — see the next lesson for why
|
||||
that's intentional, not withheld information.
|
||||
|
||||
## What "unlocked" actually means
|
||||
|
||||
The Capstone doesn't unlock just because you decided you're ready — it unlocks when the
|
||||
platform confirms, server-side, that you've genuinely completed every prerequisite stage
|
||||
(Foundation, Workloads, Networking, Configuration, Troubleshooting, and Real Ops). This isn't a
|
||||
UI suggestion you can bypass; it's enforced the same way everywhere you'd try to start it. If
|
||||
you see a "locked" message, the platform is telling you something true about your actual
|
||||
progress, not gatekeeping arbitrarily.
|
||||
|
||||
## What completion means
|
||||
|
||||
Passing the Capstone requires resolving every scenario in your specific draw — this is recorded
|
||||
as a real, permanent achievement (`capstone_completed`) tied to your account, the same
|
||||
achievement mechanism that recorded every stage completion before it. It's the platform's own
|
||||
confirmation that you can operate a Kubernetes environment under exam conditions, not just that
|
||||
you clicked through a curriculum.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Lesson goal:** Set accurate expectations before a student attempts the real assessment —
|
||||
reduce anxiety from not knowing "what kind of thing" the Capstone tests, without revealing any
|
||||
specific content.
|
||||
|
||||
**Approximate timing:** 10 minutes read; 8-10 minutes live.
|
||||
|
||||
**Preparation:** None beyond having completed ROOT 12 yourself if teaching this live, so you can
|
||||
speak concretely about the Real Ops incidents' style without naming specifics from the current
|
||||
draw pool.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students sometimes think "random draw" means the Capstone is inconsistent/unfair between
|
||||
attempts. Clarify: the REQUIRED Real Ops scenarios are constant across every attempt; only a
|
||||
smaller supplementary set is randomized.
|
||||
- "It's locked, so something's broken" — reinforce that a locked Capstone is the platform
|
||||
correctly reporting incomplete prerequisites, not a bug.
|
||||
|
||||
**Questions to ask students:** "Which ROOT_ course do you feel least confident about right now?
|
||||
What would you do about that before attempting the Capstone?" — a good self-assessment prompt
|
||||
that ties directly into the next lesson's time-management content.
|
||||
|
||||
**Expected takeaway:** The Capstone measures real operational capability built across the whole
|
||||
ROOT_ sequence — approaching it as "one more scenario to guess your way through" misses the
|
||||
point of everything that led up to it.
|
||||
@@ -0,0 +1,66 @@
|
||||
# Time Management & Troubleshooting Strategy Under a Clock
|
||||
|
||||
## 90 minutes, several scenarios — budget before you start
|
||||
|
||||
Once the Capstone begins, the clock is running against the whole attempt, not per scenario.
|
||||
Spending 60 of your 90 minutes perfectly solving one incident, then rushing (or failing to
|
||||
attempt) the remaining ones, is a worse outcome than solving most scenarios well within a
|
||||
reasonable time each. Before diving into the first incident, take 60 seconds to look at how many
|
||||
scenarios you're facing and roughly divide your available time — treat that budget as a real
|
||||
constraint, not a suggestion.
|
||||
|
||||
## Apply the frameworks you already have — don't invent a new approach under pressure
|
||||
|
||||
You've already practiced two frameworks across ROOT_:
|
||||
|
||||
- **Observe → Gather Evidence → Form Hypothesis → Test → Repair → Validate → Prevent Recurrence**
|
||||
(ROOT 10) for the mechanics of diagnosing a specific technical failure.
|
||||
- **Detect → Triage → Diagnose → Mitigate → Recover → Verify → Learn** (ROOT 12) for the
|
||||
broader "you own this incident" framing.
|
||||
|
||||
Under time pressure, the temptation is to abandon structure and start guessing. Resist it — the
|
||||
frameworks exist precisely because they're FASTER than guessing once you've internalized them,
|
||||
not slower. Reach for the same habits: read `kubectl describe`/Events before assuming, state
|
||||
what you think is wrong before changing anything, verify after you fix.
|
||||
|
||||
## When to move on
|
||||
|
||||
If you're genuinely stuck on one scenario well past its budgeted time with no new evidence
|
||||
emerging, it is a legitimate strategic choice to move to a different scenario and return later
|
||||
if time allows — partial credit on more scenarios can beat a perfect score on fewer. This isn't
|
||||
giving up; it's the same triage instinct from the incident-response framework, applied to your
|
||||
own time as a resource.
|
||||
|
||||
## What's genuinely allowed
|
||||
|
||||
Everything you've used throughout ROOT_ Labs remains available: `kubectl` and its full command
|
||||
set, the Lab's own in-context hints (if the specific scenario offers them), and your own notes
|
||||
from earlier modules. There is no expectation of memorizing anything you'd normally look up —
|
||||
the Capstone tests operational judgment and diagnostic process, not recall of obscure flags.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Lesson goal:** Reduce a common source of avoidable Capstone underperformance — poor time
|
||||
allocation — separately from technical skill gaps.
|
||||
|
||||
**Approximate timing:** 10 minutes read; 8 minutes live.
|
||||
|
||||
**Demonstration plan:** If teaching this live, consider a quick show-of-hands: "who has ever
|
||||
spent way too long on one problem and run out of time for an easier one elsewhere?" — normalizes
|
||||
the behavior this lesson is trying to prevent.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students sometimes think asking for more time or pausing mid-scenario is against the rules.
|
||||
Clarify based on your actual classroom/platform policy rather than assuming — this varies by
|
||||
delivery mode and isn't something this lesson should guess at.
|
||||
|
||||
**Questions to ask students:** "If you had to guess right now, which TYPE of scenario (not a
|
||||
specific one) do you think would eat the most of your time, and why?" — surfaces self-awareness
|
||||
about weak areas before the real attempt.
|
||||
|
||||
**When to let a student struggle vs. provide a hint:** This lesson itself needs no struggle —
|
||||
it's orientation, not a challenge. Save "let them struggle" judgment for the actual Capstone
|
||||
attempt.
|
||||
|
||||
**Expected takeaway:** Time management under the Capstone's clock is a skill you can prepare for
|
||||
explicitly, separate from and in addition to the technical skills the rest of ROOT_ built.
|
||||
@@ -0,0 +1,58 @@
|
||||
# Readiness Check: Are You Actually Ready?
|
||||
|
||||
This isn't a graded quiz — it's a genuine self-assessment before you attempt something you can't
|
||||
easily "retry with different answers memorized." Answer each honestly.
|
||||
|
||||
---
|
||||
|
||||
**1. Can you explain, without looking it up, the difference between a liveness probe and a
|
||||
readiness probe, and diagnose which one is failing from `kubectl get pods` output alone?**
|
||||
|
||||
*If no:* revisit ROOT 10's gold-standard Health Probes module before attempting the Capstone —
|
||||
this exact distinction shows up in Real Ops-style incidents.
|
||||
|
||||
---
|
||||
|
||||
**2. Given a Service with zero Endpoints, do you know the first two commands you'd run to find
|
||||
out why, without being told which module this relates to?**
|
||||
|
||||
*If no:* revisit ROOT 07's Services & Service Discovery module.
|
||||
|
||||
---
|
||||
|
||||
**3. Can you apply the Observe → Evidence → Hypothesis → Test → Repair → Validate → Prevent
|
||||
framework to an unfamiliar failure you've never specifically seen before, not just to the exact
|
||||
examples from the lesson?**
|
||||
|
||||
*If no:* this is worth practicing deliberately — pick any Practice-tab scenario you haven't done
|
||||
yet and narrate the framework out loud to yourself while solving it.
|
||||
|
||||
---
|
||||
|
||||
**4. Have you completed the Real Ops incidents in ROOT 12 for real — diagnosed and repaired
|
||||
them yourself, not looked up the answer?**
|
||||
|
||||
*If no:* do this before attempting the Capstone. The required scenarios in every Capstone
|
||||
attempt are drawn from exactly this style of incident.
|
||||
|
||||
---
|
||||
|
||||
**5. Do you have a rough time-budget plan for a 90-minute, multi-scenario attempt?**
|
||||
|
||||
*If no:* re-read the time-management lesson in this module before you begin.
|
||||
|
||||
## What to do if you answered "no" to more than one
|
||||
|
||||
That's genuinely useful information, not a failure — it tells you specifically where to spend
|
||||
your remaining prep time. Go back to the relevant module, do the work for real, and come back
|
||||
to this check again before attempting the Capstone.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** In a live classroom, this works well as private/individual reflection rather
|
||||
than a group discussion — the value is honest self-assessment, not a public performance.
|
||||
|
||||
**Approximate timing:** 5-8 minutes.
|
||||
|
||||
**Expected takeaway:** Readiness is something a student can genuinely self-assess against
|
||||
concrete, specific criteria — not a vague feeling of confidence.
|
||||
@@ -0,0 +1,42 @@
|
||||
# Readiness Reflection: What Completion Means
|
||||
|
||||
Before you move toward the actual Capstone attempt, take a moment to reflect on the whole ROOT_
|
||||
journey up to this point.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Think back to ROOT 05's opening lesson on desired state. How differently would you explain
|
||||
that concept now, after everything since, compared to how you understood it the first time?
|
||||
2. Which single module across ROOT_ changed how you approach a problem the most — not which was
|
||||
most interesting, but which actually changed your behavior when something breaks?
|
||||
3. What does passing the Capstone mean to you, beyond a completion record on the platform?
|
||||
|
||||
## What completion actually represents
|
||||
|
||||
A `capstone_completed` achievement isn't a certificate claiming you've memorized Kubernetes
|
||||
trivia. It's the platform's record that you demonstrated, under real time pressure, without
|
||||
step-by-step guidance, that you can diagnose and repair real operational problems using
|
||||
frameworks you internalized rather than looked up in the moment. That's a materially different
|
||||
and more durable skill than passing a multiple-choice exam about Kubernetes objects.
|
||||
|
||||
## When you're ready
|
||||
|
||||
There's no "perfect" moment where every module feels fully mastered — that's normal. If the
|
||||
readiness check in the previous activity mostly came back "yes," and you've done the Real Ops
|
||||
incidents for real, you're ready to attempt it. The Capstone will unlock automatically once the
|
||||
platform confirms your prerequisite stages are genuinely complete — when you see it unlocked,
|
||||
that's the platform's own confirmation, not just this module's.
|
||||
|
||||
This module is complete. Good luck.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 2 tends to produce genuinely varied,
|
||||
interesting answers across a cohort — worth having students share out loud if time allows.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Preparation:** None — this is a closing/orientation activity, not a technical one.
|
||||
|
||||
**Expected takeaway:** Completing ROOT_'s Kubernetes sequence is a real, demonstrable skill
|
||||
change, not just curriculum progress — the Capstone is where that becomes visible and provable.
|
||||
@@ -0,0 +1,137 @@
|
||||
# Course Authoring Standard
|
||||
|
||||
Phase 6A §11 — the standard every future RootNode lesson/module must follow, so a future Claude
|
||||
Code session or a human instructor can author content consistently. The module at
|
||||
`content/root/root-10-kubernetes-troubleshooting-observability/m01-health-probes/` (backing
|
||||
`root-10-m01-health-probes-and-troubleshooting`) is the reference implementation — when in
|
||||
doubt, read it before writing new content.
|
||||
|
||||
## 1. Target learner
|
||||
|
||||
State explicitly, in the module's Postgres `description` (not buried in prose): who this is for
|
||||
and what they're assumed to already know. root_ targets ages ~15-18+ through adult learners
|
||||
(see `docs/PHASE6A_ROOT_COURSE_FOUNDATION.md` §1) — write at a level that respects that range;
|
||||
don't write down to it.
|
||||
|
||||
## 2. Prerequisites
|
||||
|
||||
Declare real prerequisite course/module ids in the `prerequisites` column, not prose alone (see
|
||||
`docs/COURSE_CONTENT_MODEL.md` §5 for how locking derives from this). If a module assumes a
|
||||
concept the platform doesn't yet teach anywhere, say so explicitly in the module description
|
||||
rather than silently assuming it.
|
||||
|
||||
## 3. Measurable objectives
|
||||
|
||||
3-5 objectives per module, stored in the `objectives` column, each a sentence that starts with a
|
||||
verb describing an observable capability ("Diagnose...", "Configure...", "Explain..."), not a
|
||||
topic list ("Probes"). A learner (or an instructor) should be able to look at an objective and
|
||||
know whether it was met.
|
||||
|
||||
## 4. Estimated time
|
||||
|
||||
Set `estimated_duration_minutes` per activity and let the module/course total derive from the
|
||||
sum — don't hand-guess a module-level number that doesn't match its activities. Time estimates
|
||||
are honest guesses, not commitments; revise them once real completion-time data exists (the
|
||||
mechanically-mapped modules' per-scenario 15/20/30-minute estimates in
|
||||
`022_course_content.sql` are explicitly flagged as a rough heuristic for exactly this reason).
|
||||
|
||||
## 5. Concept explanation (the `lesson` activity)
|
||||
|
||||
Explain the underlying concept BEFORE any lab is presented — this is the module's core
|
||||
pedagogical premise (see `docs/PHASE6A_ROOT_COURSE_FOUNDATION.md`'s "core product principle").
|
||||
Cover: why the concept/mechanism exists (the problem it solves), the mechanism itself, and
|
||||
common real-world mistakes. Use concrete, runnable examples (`kubectl` output, YAML snippets)
|
||||
over abstract description wherever possible.
|
||||
|
||||
## 6. Demonstration
|
||||
|
||||
Show the concept in action with a worked example BEFORE asking the learner to do it themselves.
|
||||
A demonstration activity should include what the learner would actually SEE (real-looking
|
||||
command output), not just a description of what would happen.
|
||||
|
||||
## 7. Guided practice (the `guided_exercise` activity)
|
||||
|
||||
Lower-stakes practice with a visible worked answer to check against (paper/scratch-file
|
||||
exercises are fine — see the gold module's readiness-probe exercise). The goal is committing to
|
||||
an answer and comparing, not passively reading a correct one first.
|
||||
|
||||
## 8. Independent practice → lab mapping
|
||||
|
||||
A `lab` activity references an EXISTING `scenario_id` — never invent duplicate lab content when
|
||||
an existing scenario already teaches the concept (check `docs/ROOT_CURRICULUM_INVENTORY.md`
|
||||
first). Write a short `content_path` intro (see the gold module's `05-lab-intro.md`) that
|
||||
connects what was just taught to the incident the student is about to solve, without handing out
|
||||
the scenario's own hints/solution — the Lab's own progressive hints stay the only place the
|
||||
actual answer path lives.
|
||||
|
||||
### Classifying near-duplicate or advanced scenarios (Phase 6B)
|
||||
|
||||
When two scenarios teach essentially the same concept (e.g. `pod-security-context` and
|
||||
`cks-pod-security-context`), don't wire both as required core labs — pick the better-fitting one
|
||||
as CORE and mark the other `is_required = false` as INDEPENDENT PRACTICE (see
|
||||
`docs/ROOT_CURRICULUM_COVERAGE_MATRIX.md`'s "duplicate concepts identified" for worked examples).
|
||||
Likewise, a genuinely harder or more niche scenario (CKS-exam-style, a less commonly needed
|
||||
skill) belongs as `is_required = false` GUIDED/ADVANCED practice, not force-fit into the
|
||||
required core path just because it exists — see `docs/COURSE_CONTENT_MODEL.md`'s optional-vs-
|
||||
required section for how this affects module completion.
|
||||
|
||||
## 9. Knowledge check
|
||||
|
||||
A handful of self-check questions with answers shown inline (not a graded exam — that's what the
|
||||
assessment engine is for). Use plain Markdown (`*Answer:*` after each question), not raw HTML
|
||||
`<details>` — `react-markdown` renders this repo's course content and does not have `rehype-raw`
|
||||
enabled, so raw HTML tags render as literal text, not a collapsible element.
|
||||
|
||||
## 10. Reflection
|
||||
|
||||
A reflection activity consolidates and connects to real-world judgment — it is not a second
|
||||
knowledge check. Ask "why does this matter beyond this one Lab," not "what did you just do."
|
||||
Include a "Real Ops takeaway": a concrete prevention/production practice a learner could apply
|
||||
on a real team.
|
||||
|
||||
## 11. Instructor notes
|
||||
|
||||
Every activity type CAN have an `## Instructor Notes` section (see
|
||||
`docs/COURSE_CONTENT_MODEL.md` §3); a `lesson`/`demonstration`/`guided_exercise`/`reflection`
|
||||
activity SHOULD have one. At minimum include: approximate timing, one likely misconception to
|
||||
watch for, and (where relevant) a discussion question or optional extension activity. Keep it
|
||||
actionable — a bullet list an instructor can glance at mid-class, not a second essay.
|
||||
|
||||
## 12. Accessibility/readability
|
||||
|
||||
Plain Markdown only (headings, lists, code fences, bold/italic, tables) — no content that
|
||||
depends on color alone to convey meaning, no ASCII art carrying information a screen reader
|
||||
would lose. Write short paragraphs; prefer lists over dense prose for anything enumerable
|
||||
(steps, mistakes, questions).
|
||||
|
||||
## 13. Terminology rules
|
||||
|
||||
Match the terminology the scenario JSON and existing Platform UI already use exactly (e.g.
|
||||
"readiness probe," not "health check probe" — check `scenarios/data/<id>.json`'s own
|
||||
`description`/`hints` for the canonical phrasing before introducing a lesson's own term for the
|
||||
same thing). Define a term the first time it's used in a module; don't assume a prerequisite
|
||||
module's terminology carried over without a one-line reminder.
|
||||
|
||||
## 14. Versioning
|
||||
|
||||
Bump the `version` integer on courses/course_modules/module_activities by hand on any semantic
|
||||
content edit — same convention `assessment_definitions`/`programs` already use. A version bump
|
||||
is informational only in Phase 6A (nothing snapshots against it the way an assessment session
|
||||
snapshots its definition version) — it exists so a future phase can detect "this changed since a
|
||||
student last saw it" without re-deriving that from `updated_at` alone.
|
||||
|
||||
## 15. Review/testing requirements
|
||||
|
||||
Before marking a module `status = 'ready'`:
|
||||
|
||||
- Every activity's `content_path` file must exist and load (verify via
|
||||
`platform/api/src/content.js`'s `loadContent`, or just hit the admin curriculum API and check
|
||||
`body` is non-empty).
|
||||
- A `lab` activity's `scenario_id` must be a real, existing scenario id.
|
||||
- An `assessment`/`capstone` activity's `assessment_definition_id` must be a real, active
|
||||
`assessment_definitions` row.
|
||||
- Walk the module once as a self-paced student end to end (lesson → ... → lab → reflection),
|
||||
and once through the instructor Curriculum view, before shipping — this is what
|
||||
`e2e-phase6a.js` and `infra/scripts/drills/test-phase6a-course-rehearsal.js` automate for the
|
||||
gold module; extend one of them (or add a new phase-scoped test) for future modules rather
|
||||
than skipping this step.
|
||||
@@ -0,0 +1,162 @@
|
||||
# Course Content Model
|
||||
|
||||
Phase 6A §3 — how RootNode represents actual teaching content (lessons, demonstrations,
|
||||
knowledge checks, guided exercises, reflections) around the existing scenario/assessment
|
||||
systems, without duplicating either. See `docs/ROOT_PROGRAM_ARCHITECTURE.md` for what this
|
||||
model is populated with, and `docs/COURSE_AUTHORING_STANDARD.md` for how to author new content
|
||||
against it.
|
||||
|
||||
## 1. Shape
|
||||
|
||||
```
|
||||
program (existing, 019_programs.sql — unchanged)
|
||||
→ course (new: courses)
|
||||
→ module (new: course_modules)
|
||||
→ activity (new: module_activities)
|
||||
```
|
||||
|
||||
`platform/api/migrations/022_course_content.sql` is the schema. Every table follows the same
|
||||
conventions already established by `programs`/`assessment_definitions`: a TEXT slug primary key,
|
||||
a plain incrementing `version` integer bumped by hand on a semantic edit, `status`/`visibility`
|
||||
CHECK constraints, `created_at`/`updated_at`.
|
||||
|
||||
## 2. Activity types and what they reference
|
||||
|
||||
| Type | References | Completion behavior |
|
||||
|---|---|---|
|
||||
| `lesson`, `demonstration`, `knowledge_check`, `guided_exercise`, `reflection` | a Markdown file (`content_path`) | `manual_marker` |
|
||||
| `lab` | an EXISTING `scenario_id` (scenarios/data/\*.json) | `auto_scenario` |
|
||||
| `assessment`, `capstone` | an EXISTING `assessment_definition_id` (`assessment_definitions`) | `auto_assessment` |
|
||||
|
||||
A `lab` activity never duplicates scenario content — it stores a bare `scenario_id` string, the
|
||||
same id the existing curriculum/attempt/progress system already uses. An `assessment`/`capstone`
|
||||
activity never duplicates an assessment definition — it stores the existing
|
||||
`assessment_definition_id` and a foreign key enforces it actually exists. Neither activity type
|
||||
is prevented from also having its own `content_path` (e.g. the gold module's lab activity has a
|
||||
short Markdown intro at `05-lab-intro.md`) — that's framing prose around the reference, not a
|
||||
copy of the referenced thing.
|
||||
|
||||
## 3. Where the prose lives
|
||||
|
||||
Long-form lesson content is git-tracked Markdown under `content/root/<course>/<module>/`, not
|
||||
Postgres. Reasoning, weighed explicitly per the authorization's instruction:
|
||||
|
||||
- **Reviewable like code.** A lesson edit is a normal diff/PR, not an opaque row update.
|
||||
- **No CMS needed.** The authorization explicitly ruled one out ("Do not build a complex CMS").
|
||||
A Postgres TEXT column holding Markdown would technically work but loses git history/diffing
|
||||
for the one thing (prose) that benefits from it most.
|
||||
- **Structural metadata still belongs in Postgres** (ordering, type, references, status,
|
||||
objectives, prerequisites) because it's queried, joined, and mutated (locking/completion) —
|
||||
exactly the kind of relational state Postgres is for, and exactly what the authorization asked
|
||||
for ("Postgres preferred for metadata/state").
|
||||
|
||||
### The Instructor Notes convention
|
||||
|
||||
One Markdown file per activity, not two. Everything before a `## Instructor Notes` heading is
|
||||
shared (self-paced-visible) content; that heading and everything after it is instructor-only.
|
||||
`platform/api/src/content.js` does the split (`splitContent()`) — a plain string search on that
|
||||
exact heading, no frontmatter parser, no template engine. The API layer (`course-progress.js`'s
|
||||
`shapeActivity()`) only ever attaches `instructor_notes` to the response when the caller is
|
||||
resolved (server-side, from session identity) to be an instructor — never from a request
|
||||
parameter. A self-paced student and an instructor read the identical file; they just see
|
||||
different slices of it.
|
||||
|
||||
## 4. Completion rules — one source of truth per behavior
|
||||
|
||||
`platform/api/src/course-progress.js`'s `isActivityComplete()` is the only place completion is
|
||||
decided, and it never invents new state for anything that's independently verifiable:
|
||||
|
||||
- **`auto_scenario`** — reads `progress[scenario_id].status === 'completed'`, the exact same
|
||||
`progress` table `curriculum-rules.js` already reads for stage completion (`platform/api/src/
|
||||
progress.js`'s `loadProgress`). A lab activity's completion is a real fact about whether the
|
||||
student actually solved the scenario, not a flag a course layer could get out of sync with.
|
||||
- **`auto_assessment`** — reads `achievements` (`achievement_type IN ('assessment_completed',
|
||||
'capstone_completed')` with `metadata->>'assessment_definition_id'` matching), the exact
|
||||
mechanism `achievements.js`'s `recordAssessmentAchievement` already writes on a real pass. No
|
||||
new achievement type, no new table.
|
||||
- **`manual_marker`** — the ONE new piece of state this phase adds: `activity_completions`
|
||||
(student_id, activity_id). Used ONLY for activity types with nothing else to derive completion
|
||||
from (a lesson has no "pass/fail," a reflection has no validator). `POST /api/student/
|
||||
activities/:id/complete` refuses (409 `not_manually_completable`) for any activity whose
|
||||
`completion_behavior` isn't `manual_marker` — a student can never fake a lab or assessment
|
||||
completion through this route.
|
||||
|
||||
Course and module completion are pure aggregates over their activities' completion (`total`/
|
||||
`completed`/`pct`/`complete` — see `totals()`), never separately stored.
|
||||
|
||||
### Optional vs. required activities (Phase 6B)
|
||||
|
||||
`module_activities.is_required` (defaults `true`) lets a module distinguish CORE teaching from
|
||||
GUIDED/INDEPENDENT-practice duplicates and ADVANCED/optional extras — see
|
||||
`docs/ROOT_CURRICULUM_COVERAGE_MATRIX.md`'s classification scheme, used throughout ROOT 09. A
|
||||
module's `complete` flag is true once every REQUIRED activity is done; an incomplete optional
|
||||
activity never blocks it, though `total`/`pct` still count every activity, required or not.
|
||||
Every Phase 6A activity has `is_required = true` (the column's default) — this is purely
|
||||
additive, nothing that was complete before Phase 6B becomes incomplete now.
|
||||
|
||||
## 5. Prerequisite/locking rule
|
||||
|
||||
A course or module lists `prerequisites` (an array of other course/module ids). It's locked if
|
||||
ANY prerequisite is not satisfied, where **satisfied** means: the prerequisite has zero total
|
||||
activities (a PLANNED entry — vacuously satisfied, since there's nothing to have completed) OR
|
||||
the prerequisite is fully complete. This is deliberate: without the "zero activities is vacuously
|
||||
satisfied" clause, an honestly-PLANNED course with no content yet would permanently lock every
|
||||
real course that lists it as a prerequisite, which would perversely punish honesty about what
|
||||
isn't built yet. See `course-progress.js`'s `prerequisitesSatisfied()`.
|
||||
|
||||
Locking is computed in two passes over each level (compute every entry's totals/prerequisite-
|
||||
check, THEN strip working state) rather than one — mutating and reading totals in the same pass
|
||||
made the result depend on Map iteration order, a real bug caught by `e2e-phase6a.js` during
|
||||
development (see that file's "found by testing" comment in `course-progress.js`).
|
||||
|
||||
## 6. Draft vs. published — separate from READY/PARTIAL/PLANNED
|
||||
|
||||
`status` (`ready`/`partial`/`planned`) describes authoring completeness and is shown to
|
||||
everyone, including students, honestly ("PLANNED — coming soon" is useful information, not a
|
||||
leak). `visibility` (`published`/`draft`) is a hard gate: a `draft` course/module/activity is
|
||||
excluded from every query a student-facing route runs (`buildCourseTree`'s `visClause`), full
|
||||
stop, regardless of its `status`. Use `draft` for genuinely unfinished authoring work you don't
|
||||
want a student to see at all yet; use `status = planned` for an honest "not built yet" that's
|
||||
fine to be visible.
|
||||
|
||||
## 6b. Content versioning (Phase 6B §21)
|
||||
|
||||
Follows the exact same convention `assessment_definitions`/`programs` already established — a
|
||||
plain incrementing `version` integer, bumped by hand, informational rather than enforced:
|
||||
|
||||
- **A typo-only edit** (fixing a spelling mistake, a broken link, rewording a sentence without
|
||||
changing its meaning) does NOT require a version bump — nothing about what a student is being
|
||||
taught or held to changed.
|
||||
- **A semantic edit** (changing a learning objective, swapping a lab's `scenario_id`, changing
|
||||
`completion_behavior`, adding/removing/reordering activities in a way that changes what
|
||||
"complete" means) SHOULD bump `version` on the affected row(s) — see
|
||||
`023_phase6b_curriculum.sql`'s own `version = version + 1` on every row it semantically
|
||||
changes (resequencing, prerequisite changes, `is_required` changes).
|
||||
- **What happens to existing completions**: nothing, automatically. Unlike an assessment
|
||||
session (which snapshots the definition version it was drawn against, because a session is a
|
||||
point-in-time timed attempt), a course/module/activity's completion is derived LIVE every time
|
||||
it's read (`course-progress.js`) from the CURRENT row plus the student's progress/achievements/
|
||||
activity_completions — there is no snapshot to invalidate. A `manual_marker` completion
|
||||
(`activity_completions`) is keyed by `activity_id`, which is stable across a version bump
|
||||
(the id doesn't change, only its content/metadata does) — so a student's prior "I did this"
|
||||
record stays valid even after a semantic edit, exactly like editing a scenario's own
|
||||
Markdown description doesn't retroactively un-complete it in `progress`.
|
||||
- **Scenario references across versions**: a `lab` activity's `scenario_id` is just a string
|
||||
reference into `scenarios/data/*.json` — if that scenario file itself changes, the activity
|
||||
automatically reflects the new content next time it's loaded (same as every other consumer of
|
||||
`curriculum.js`'s cache). If an activity's `scenario_id` itself is changed to point at a
|
||||
DIFFERENT scenario, that's a semantic edit (bump the version) and — depending on intent — may
|
||||
warrant a new `activity_id` instead of mutating the existing one, so a student's prior
|
||||
completion of the OLD scenario isn't silently reinterpreted as completion of the new one.
|
||||
- **What this deliberately does NOT build**: no content-revision history table, no diffing, no
|
||||
"students who saw v1 vs v2" cohort tracking. If a future phase needs that level of rigor
|
||||
(e.g. for compliance/audit reasons), it's a real, separate feature — not something to bolt
|
||||
onto a plain integer column.
|
||||
|
||||
## 7. Why this doesn't touch Phase 1-5 systems
|
||||
|
||||
No column was added to `scenarios/*.json`, `assessment_definitions`, `achievements`,
|
||||
`progress`, or `scenario_attempts`. No existing API route's behavior changed (the `/api/
|
||||
progress/record`, `/api/assessments/*`, `/api/capstone` routes are byte-for-byte unchanged).
|
||||
The course layer is purely additive: new tables, new routes, new frontend views, reading
|
||||
existing tables through their existing shapes.
|
||||
@@ -88,6 +88,12 @@ To add/reorder a stage: edit `rootnode-path.json` directly, referencing existing
|
||||
`scenarios/data/*.json` ids. No backend code change needed — it's read fresh via
|
||||
`POST /api/cache/reload` or on server restart.
|
||||
|
||||
As of Phase 6A, `root_` also has a course/module/activity layer ON TOP of this (lessons,
|
||||
demonstrations, guided exercises around the same scenario ids) — see §17 and
|
||||
`docs/COURSE_CONTENT_MODEL.md`. `rootnode-path.json` itself is unmodified and still the thing
|
||||
`curriculum-rules.js` computes stage lock/completion from; the course layer is a parallel,
|
||||
additive structure that references the same scenario ids, not a replacement.
|
||||
|
||||
## 5. Student identity — what's built, what isn't
|
||||
|
||||
**Current state (Phase 5A — see `docs/PHASE5A_STUDENT_IDENTITY_REMOTE_CLASSROOM.md` for the
|
||||
@@ -357,3 +363,132 @@ Full design: `docs/PHASE5A_STUDENT_IDENTITY_REMOTE_CLASSROOM.md`, plus
|
||||
students and confirming zero orphaned data) — both against a live `docker-compose.dev.yml`
|
||||
stack, not mocks. The full pre-existing regression suite was re-run given how much shared code
|
||||
this phase touched (`server.js`, `admin-routes.js`, `sessions.js`, `rbac.js`).
|
||||
|
||||
## 15. Self-Paced Learning & Enrollment Foundation (Phase 5B)
|
||||
|
||||
Full design: `docs/PHASE5B_SELF_PACED_ENROLLMENT.md`, plus
|
||||
`docs/RUNBOOK_SELF_PACED_ENROLLMENT.md` and `docs/WEBSITE_ENROLLMENT_INTEGRATION.md`.
|
||||
|
||||
- **What it is:** `programs` (a first-class learning product, distinct from curriculum JSON —
|
||||
`curriculum_path` is just what it points at) and `enrollments` (a student's academic access to
|
||||
a program, independent of cohort membership — a self-paced enrollment has no cohort at all).
|
||||
Server-enforced entitlement (`enrollment.deriveEnrollmentAccess`, mirroring `classroom.js`'s
|
||||
derived-not-stored `derivePresence` pattern from Phase 5A) gates a new
|
||||
`POST /api/student/programs/:id/launch` in front of the exact same disposable-Lab sequence
|
||||
`/api/environment/launch` already used (extracted into `environment-view.js` so both share one
|
||||
implementation) — pending/future/expired/cancelled enrollments cannot launch, server-side, not
|
||||
just hidden in the UI. Completion is entirely server-derived from the existing capstone
|
||||
achievement mechanism (Phase 3B, unmodified) and idempotent. A new, separate
|
||||
`POST /api/integrations/enrollments` boundary (its own pre-shared-secret auth, no session
|
||||
cookie involved — same trust-boundary pattern as Gateway↔Provisioner's
|
||||
`PROVISIONER_SHARED_SECRET`) is what the *separate* public website will eventually call;
|
||||
idempotent via a DB unique constraint on the caller's own event id, proven safe under real
|
||||
concurrent duplicate requests, not just sequential retries.
|
||||
- **A real bug found and fixed:** `POST /api/progress/record` predated the `wrap()`
|
||||
async-error-handling convention — an unexpected internal error there (a malformed
|
||||
`environment_id`) crashed the entire Platform API process, not just that one request. Fixed
|
||||
(wrapped + input-validated) and regression-tested.
|
||||
- **Tested, not just designed:** `platform/test/e2e-phase5b.js` (32 checks) and
|
||||
`infra/scripts/drills/test-phase5b-selfpaced-rehearsal.js` (15 checks) — both against a live
|
||||
stack, including real Lab destroy/reaper-reconciliation/recreate persistence proofs, a full
|
||||
real capstone completion, integration-endpoint concurrency, and a real Platform API restart.
|
||||
Migration safety proven explicitly against a fresh Postgres seeded with realistic pre-5B data
|
||||
(see that doc §14).
|
||||
|
||||
## 16. Enrollment Operations & Communications (Phase 5C)
|
||||
|
||||
Full design: `docs/PHASE5C_ENROLLMENT_COMMUNICATIONS.md`, plus
|
||||
`docs/RUNBOOK_GOOGLE_WORKSPACE_EMAIL.md` and `docs/RUNBOOK_STUDENT_ACTIVATION.md`.
|
||||
|
||||
- **What it is:** a durable, provider-neutral notification pipeline (`notification_events`, a
|
||||
`FOR UPDATE SKIP LOCKED` worker polling on a plain `setInterval` — same pattern as the
|
||||
existing session/login-attempt sweepers, no new infrastructure) that finally sends real email
|
||||
for activation, enrollment confirmation, password reset, and password-changed — closing the
|
||||
gap Phase 5A explicitly left open. Google Workspace adapter: Gmail API via a service account
|
||||
with domain-wide delegation (no human password, no SMTP basic auth — see that doc §2 for why),
|
||||
behind the same provider interface a deterministic dev provider implements, so no test or dev
|
||||
run ever reaches a real inbox. Reuses Phase 5A's activation/reset token mechanism entirely — no
|
||||
competing invitation-token system.
|
||||
- **A real bug found and fixed:** `request-password-reset` would mint a reset token for a
|
||||
never-activated student, which doesn't make sense (there's no password yet to reset) and would
|
||||
have sent a confusing email. Fixed to fall through to the same anti-enumeration response as a
|
||||
nonexistent account.
|
||||
- **Tested, not just designed:** `platform/test/e2e-phase5c.js` (28 checks: activation vs.
|
||||
confirmation routing, duplicate/concurrent webhook idempotency, temporary/permanent/exhausted
|
||||
retry classification, resend/regenerate token invalidation, production token-secrecy gating
|
||||
via a real container recreate with `COOKIE_SECURE=1`, minor/contact-email resolution, RBAC
|
||||
scoping, metrics/audit hygiene, a real restart) and
|
||||
`infra/scripts/drills/test-phase5c-enrollment-activation.js` (10 checks: one continuous story
|
||||
from website enrollment through a real rendered/extracted activation email, a real Lab launch
|
||||
and scenario completion, a second enrollment's confirmation email, and a full password-reset
|
||||
round trip) — both against a live stack, using only the dev provider (no real network calls).
|
||||
Live Google Workspace delivery has since been validated for real, end to end, via
|
||||
`academy@rootnodeacademy.com` — see `docs/PHASE5C_ENROLLMENT_COMMUNICATIONS.md` §14.
|
||||
|
||||
## 17. ROOT_ Course Architecture + Content Delivery Foundation (Phase 6A)
|
||||
|
||||
Full design: `docs/PHASE6A_ROOT_COURSE_FOUNDATION.md`, plus `docs/ROOT_CURRICULUM_INVENTORY.md`,
|
||||
`docs/ROOT_PROGRAM_ARCHITECTURE.md`, `docs/COURSE_CONTENT_MODEL.md`, and
|
||||
`docs/COURSE_AUTHORING_STANDARD.md`.
|
||||
|
||||
- **What it is:** a course → module → activity content model (`022_course_content.sql`) layered
|
||||
over the existing scenario/assessment/achievement systems for `root_` only, so a lesson can be
|
||||
taught BEFORE a lab is presented instead of the platform only ever showing a bare scenario
|
||||
list. A `lab` activity references an existing `scenario_id`; an `assessment`/`capstone`
|
||||
activity references an existing `assessment_definition_id` — neither duplicates content.
|
||||
Long-form lesson prose lives in git-tracked Markdown (`content/root/`), split into
|
||||
shared/instructor-only sections by a `## Instructor Notes` heading convention
|
||||
(`content.js`). One `buildCourseTree()` (`course-progress.js`) serves both the self-paced
|
||||
student route and the instructor curriculum view — no forked curriculum for the two delivery
|
||||
modes.
|
||||
- **Completion, never a second source of truth:** a lab's completion reads the existing
|
||||
`progress` table; an assessment/capstone activity's completion reads the existing
|
||||
`achievements` table; only lesson-only activities (nothing else to derive completion from) get
|
||||
the one genuinely new table, `activity_completions`.
|
||||
- **A real bug found and fixed during testing:** computing a module/course's prerequisite lock
|
||||
and then stripping its working totals in the SAME map iteration made the result depend on Map
|
||||
iteration order — a dependent entry could see its prerequisite's totals already deleted. Fixed
|
||||
by splitting into two explicit passes (`course-progress.js`).
|
||||
- **One gold-standard module fully authored:** ROOT 10's Kubernetes Health Probes &
|
||||
Troubleshooting (lesson → demonstration → knowledge check → guided exercise → real Lab
|
||||
`rootnode-probe-port-mismatch` → reflection), the template `docs/COURSE_AUTHORING_STANDARD.md`
|
||||
documents for all future modules. Every other course is honestly `partial` (real labs, no
|
||||
lessons yet) or `planned` (no content exists) — nothing was fabricated to look finished.
|
||||
- **Tested, not just designed:** `platform/test/e2e-phase6a.js` (21 checks: hierarchy/ordering,
|
||||
all three completion behaviors, prerequisite locking, resume, cross-student isolation, RBAC
|
||||
for self-paced no-cohort students, draft-content hiding, version stability) and
|
||||
`infra/scripts/drills/test-phase6a-course-rehearsal.js` (12 checks, real k3s: a student reads
|
||||
the gold module's real lesson content, launches a real Lab, the Lab's own `setup_commands`
|
||||
stand up the broken deployment, a real `kubectl patch` fixes it, and the module completes and
|
||||
unlocks the next one) — both against a live stack.
|
||||
|
||||
## 18. ROOT_ Curriculum Authoring & Coverage Expansion (Phase 6B)
|
||||
|
||||
Full design: `docs/PHASE6B_ROOT_CURRICULUM_EXPANSION.md`, plus
|
||||
`docs/ROOT_CURRICULUM_COVERAGE_MATRIX.md` and `docs/ROOT_CAPSTONE_READINESS.md`.
|
||||
|
||||
- **What it is:** 13 new teaching modules across ROOT 05-10/12/Capstone
|
||||
(`023_phase6b_curriculum.sql`), built entirely from the 98-scenario inventory Phase 6A already
|
||||
catalogued — no new scenarios needed. ROOT 08/09 moved from PLANNED to PARTIAL; scenario
|
||||
coverage in structured curriculum went from 44/98 to 88/98 (89.8%), verified by direct query,
|
||||
not estimated. 3 modules (plus the pre-existing gold standard) reach the full READY bar; every
|
||||
other new module is honestly PARTIAL, missing a demonstration or guided exercise, documented
|
||||
per-module rather than glossed over.
|
||||
- **`is_required`** (new `module_activities` column, defaults `true`): distinguishes CORE
|
||||
teaching from GUIDED/INDEPENDENT-practice duplicates and ADVANCED/optional extras — a module
|
||||
is complete once every REQUIRED activity is done; an incomplete optional one never blocks it.
|
||||
Every Phase 6A activity is unaffected (defaults to required).
|
||||
- **The capstone server-side unlock fix:** `POST /api/assessments/:id/start` never checked
|
||||
`unlock_rule` before this phase — a direct API call could start the capstone before its
|
||||
prerequisite stages were complete, even though the UI never exposed that path. Fixed by
|
||||
binding to the same `computeCurriculumState()` gate `GET /api/capstone` already used. See
|
||||
`docs/ROOT_CAPSTONE_READINESS.md` §4 for the full story, including a pre-existing gap this
|
||||
fix exposed in `e2e-phase5b.js`'s own capstone test (fixed alongside).
|
||||
- **Tested, not just designed:** `platform/test/e2e-phase6b.js` (24 checks: new-module
|
||||
hierarchy, prerequisite chains through the newly-inserted front-door modules, the
|
||||
optional-vs-required completion rule, the capstone bypass fix in both directions, persistence/
|
||||
isolation/RBAC/draft-hiding proven against the new content) and
|
||||
`infra/scripts/drills/test-phase6b-root-course-rehearsal.js` (11 checks, real k3s: a student
|
||||
walks a brand-new module — ROOT 07 Services & Service Discovery — end to end including a real
|
||||
`kubectl expose` against a live cluster, plus an instructor narrative proving Curriculum view,
|
||||
Prepare Labs, and classroom tooling all still work against the expanded curriculum).
|
||||
|
||||
@@ -507,3 +507,28 @@ any of it — the Provisioner, the `environments` state machine, the Lab Host Ro
|
||||
Lab lifecycle, and the Platform/Lab-Host VM boundary are all untouched. The classroom dashboard
|
||||
reads Lab status the exact same way every other admin view does (`provisioner.listForStudent`,
|
||||
the same state enum) rather than inventing a second one.
|
||||
|
||||
## Phase 5B: still reused, not modified — a gate in front, not a second Lab
|
||||
|
||||
Phase 5B (`docs/PHASE5B_SELF_PACED_ENROLLMENT.md`) added self-paced programs/enrollment on top
|
||||
of the exact same disposable Lab this document describes — again without changing the
|
||||
Provisioner, `environments`, the reaper, or the Lab Host Router at all. A self-paced launch
|
||||
(`POST /api/student/programs/:id/launch`) is an entitlement check followed by calling the
|
||||
identical launch sequence `/api/environment/launch` always has — verified for real by killing a
|
||||
Lab container out-of-band and restarting the Provisioner to trigger its actual
|
||||
`reconcileOnStartup` reconciliation path (the same technique Phase 4.1's
|
||||
`test-lab-host-loss.js` established), then confirming a self-paced student's academic progress
|
||||
is completely unaffected and a fresh Lab boots normally.
|
||||
|
||||
## Phase 6A: reused, not modified — a teaching layer in front, still one Lab
|
||||
|
||||
Phase 6A (`docs/PHASE6A_ROOT_COURSE_FOUNDATION.md`) added a course/module/activity content model
|
||||
in front of the exact same disposable Lab this document describes — no change to the
|
||||
Provisioner, `environments`, the reaper, the Lab Host Router, or the Lab image itself
|
||||
(`Dockerfile`/`entrypoint.sh`/`backend/server.js` untouched). A course's `lab` activity is a
|
||||
`scenario_id` reference; "Launch Lab" from the course view calls the identical
|
||||
`/api/student/programs/:id/launch` sequence Phase 5B already established — not a third launch
|
||||
path. Verified for real, not assumed: `infra/scripts/drills/test-phase6a-course-rehearsal.js`
|
||||
launches a real Lab, calls the Lab's own `POST /api/scenarios/:id/setup` through the Gateway's
|
||||
existing `/lab/:token/*` proxy exactly as the browser UI does, and runs a real `kubectl patch`
|
||||
against the live k3s cluster inside that container before recording completion.
|
||||
|
||||
@@ -0,0 +1,195 @@
|
||||
# Phase 5B — Self-Paced Learning & Enrollment Foundation
|
||||
|
||||
**Status: PHASE 5B COMPLETE. SELF-PACED LEARNING FOUNDATION COMPLETE. WEBSITE ENROLLMENT BOUNDARY READY.**
|
||||
|
||||
Phase 5A gave students real identity and instructors a remote classroom. Phase 5B's job was
|
||||
the last piece needed for an authenticated student to be enrolled in a RootNode Academy program
|
||||
and complete it entirely on their own — plus the clean, provider-neutral boundary a *separate*
|
||||
public website/commerce system will call to create those enrollments later. This phase does
|
||||
**not** build that website or any payment processing — see §12.
|
||||
|
||||
## 1. The RootNode program ladder
|
||||
|
||||
`init_` (ages ~5-10, robotics/block coding), `compile_` (ages ~11-14, ESP32/MicroPython/IoT),
|
||||
`root_` (ages ~15-18+, Linux/containers/Kubernetes/DevOps). The existing KubeKosh-derived
|
||||
curriculum belongs to `root_`. This phase does not write init_/compile_ curriculum content — it
|
||||
makes the platform capable of holding those programs later without another schema change (the
|
||||
`programs.family` enum already accepts `init`/`compile`/`root`).
|
||||
|
||||
## 2. Programs — a first-class object, not curriculum JSON
|
||||
|
||||
`programs` (migration `019_programs.sql`) is the commercial/academic unit a student enrolls in.
|
||||
It is deliberately **not** the curriculum file itself — `programs.curriculum_path` just points at
|
||||
one (today, always `rootnode-path`, the only curriculum that exists). `program_id` is the slug
|
||||
primary key, matching `assessment_definitions`/`cohorts`' existing convention rather than a
|
||||
separate id+slug pair. `family` (`init`/`compile`/`root`) and `delivery_mode`
|
||||
(`self_paced`/`cohort`/`hybrid`) are both real CHECK-constrained enums, not free text — small,
|
||||
stable vocabularies, matching how this codebase already treats `assessment_definitions.type` and
|
||||
`hint_policy`. The phase seeds exactly one program: `root-kubernetes-foundations`, wrapping the
|
||||
existing RootNode/Kubernetes curriculum and the existing `rootnode-capstone` as its completion
|
||||
criterion — no new curriculum content invented.
|
||||
|
||||
`programs.version` bumps automatically whenever an edit changes a field an enrollment snapshots
|
||||
(`curriculum_path`, `delivery_mode`) — same versioning principle `assessment_definitions`
|
||||
already established, so editing a program later never silently changes what an already-enrolled
|
||||
student is being held to.
|
||||
|
||||
## 3. Enrollments — real academic access, separate from cohorts
|
||||
|
||||
`enrollments` (migration `020_enrollments.sql`) represents a student's access to a program:
|
||||
`pending → active → completed`, or `pending`/`active → cancelled`. All transitions are
|
||||
server-side only (`enrollment.js`) — nothing accepts a client-supplied status. `expired` is
|
||||
**derived**, not stored: `deriveEnrollmentAccess(enrollment, now)` is a pure function (matching
|
||||
Phase 5A's `classroom.js` `derivePresence` pattern) that computes it from `access_ends_at` at
|
||||
read time, so there's no background sweep and nothing to go stale.
|
||||
|
||||
A self-paced enrollment has **no cohort at all** — `cohort_enrollments` (Phase 1) is completely
|
||||
untouched. A student can hold a cohort enrollment, a self-paced enrollment, and multiple
|
||||
historical enrollments simultaneously without any of those concepts colliding; re-enrollment
|
||||
after completion/cancellation always creates a new row rather than reviving a historical one (a
|
||||
partial unique index enforces at most one `pending`/`active` enrollment per student+program at a
|
||||
time — DB-level, not app-level "check then insert").
|
||||
|
||||
## 4. Curriculum assignment — the schema Phase 4.1 flagged, now wired
|
||||
|
||||
Phase 4.1's `curriculum_assignments` table (schema-ready since Phase 2B, never read by
|
||||
anything) is **not** reused as-is — it models cohort/student curriculum assignment
|
||||
independent of any commercial concept, which is a different problem than "this enrollment in
|
||||
this program uses this curriculum." Instead, `enrollments.curriculum_path` is itself the
|
||||
snapshot: set from `programs.curriculum_path` at enrollment time, so a later edit to the
|
||||
program never changes what an existing enrollment means. `curriculum_assignments` remains
|
||||
exactly as Phase 4.1 left it — still available for a future direct per-student assignment need
|
||||
outside the program/enrollment model, not duplicated or removed.
|
||||
|
||||
The resolution chain is real: `enrollment.student_id → enrollment.program_id →
|
||||
enrollment.curriculum_path → curriculum.loadCurriculumPath() → stages → scenarios`
|
||||
(`program-status.js`'s `buildProgramStatus`). Only one curriculum path exists today, so this
|
||||
always resolves to `rootnode-path` — the extension point for a second path is
|
||||
`enrollments.curriculum_path` itself, already snapshotted per-enrollment.
|
||||
|
||||
## 5. Self-paced dashboard and resume — reused, not reinvented
|
||||
|
||||
`GET /api/student/programs` returns every enrollment a student holds (never a public catalog —
|
||||
no public course browsing exists), each annotated with real access state, progress, and a
|
||||
deterministic next activity. The heavy lifting is **entirely existing Phase 3B machinery**:
|
||||
`curriculum-rules.js`'s `computeCurriculumState` already returns `current_stage` and each
|
||||
stage's `next_scenario` — `program-status.js` just wraps that per-enrollment. "Continue
|
||||
Learning" resume is not a guess based on the last page rendered; it's the same rule-driven
|
||||
current-stage/next-scenario computation the My Learning tab has used since Phase 3B, now scoped
|
||||
per program. Destroying a Lab never touches this — `progress`/`scenario_attempts` live in
|
||||
Postgres, completely independent of any container's lifecycle (verified for real, not just
|
||||
asserted — see §9).
|
||||
|
||||
## 6. Entitlement — enforced server-side, not by hiding a button
|
||||
|
||||
`POST /api/student/programs/:enrollmentId/launch` is the only self-paced entry point into a
|
||||
Lab. It resolves the enrollment (404 if it isn't the caller's own), calls
|
||||
`deriveEnrollmentAccess`, and 403s with a `reason` (`pending_activation` / `not_yet_started` /
|
||||
`access_expired` / `cancelled`) if `can_launch` is false — **before** ever touching the
|
||||
Provisioner. A pending, future-dated, expired, or cancelled enrollment cannot provision a Lab no
|
||||
matter what the browser sends. The existing `POST /api/environment/launch` (cohort/pilot path)
|
||||
is completely unchanged — this is a new, additive gate in front of the exact same disposable-Lab
|
||||
sequence (`environment-view.js`, extracted from `server.js` so both routes share one
|
||||
implementation), not a second kind of Lab.
|
||||
|
||||
## 7. Completion — server-derived and idempotent
|
||||
|
||||
`enrollment.maybeCompleteEnrollment` checks, for every `active` enrollment whose program has a
|
||||
`completion_assessment_id`, whether a matching `capstone_completed` achievement already exists
|
||||
(the achievement `recordAssessmentAchievement` already writes when a capstone session finalizes
|
||||
— Phase 3B, unmodified). If so, it transitions the enrollment to `completed` — an `UPDATE ...
|
||||
WHERE status = 'active'`, so calling it again once already completed is a guaranteed no-op, not
|
||||
a duplicate transition or an error. This runs opportunistically on every program-status read
|
||||
(`program-status.js`), which means completion is detected on the next read after the qualifying
|
||||
event, not pushed instantly — acceptable latency for a status field, and it means there is no
|
||||
"mark complete" button anywhere in the frontend to accidentally trust.
|
||||
|
||||
## 8. Pacing and availability
|
||||
|
||||
`starts_at`/`access_ends_at` are the entire pacing model this phase builds — no drip scheduling,
|
||||
no per-lesson unlock dates. `starts_at` in the future: the enrollment is visible
|
||||
(`effective_status: 'active'`) but `can_launch: false`, reason `not_yet_started`. `access_ends_at`
|
||||
in the past: `effective_status: 'expired'`, prior progress/achievements remain fully visible,
|
||||
`can_launch: false`. Both are enforced in `deriveEnrollmentAccess`, called from the same
|
||||
server-side gate every launch goes through — never a frontend-only check.
|
||||
|
||||
## 9. Real Lab-persistence proof
|
||||
|
||||
`platform/test/e2e-phase5b.js` and `infra/scripts/drills/test-phase5b-selfpaced-rehearsal.js`
|
||||
both drive this against a real stack: launch a Lab, complete a scenario, destroy the environment
|
||||
via the admin API, relaunch, confirm identical progress and resume target; separately, kill the
|
||||
Lab container directly (bypassing the Provisioner, exactly the idle reaper's failure mode from
|
||||
the Provisioner's point of view) and restart the Provisioner to run its real
|
||||
`reconcileOnStartup` reconciliation (the same technique `infra/scripts/drills/
|
||||
test-lab-host-loss.js` established in Phase 4.1) before relaunching — progress is identical
|
||||
both times.
|
||||
|
||||
## 10. Instructor/admin management
|
||||
|
||||
`/admin` → Programs: list/create/edit/activate-deactivate (admin/owner only, matching
|
||||
`assessment_definitions`' authoring convention — every role can view), per-program enrollment
|
||||
list and manual-enroll form, per-enrollment activate/cancel. The existing student detail view
|
||||
gained an "Enrolled programs" card (status, delivery mode, progress, last activity).
|
||||
|
||||
**RBAC** (`rbac.js`'s new `requireEnrollmentAccess`): owner/admin have global management, as
|
||||
everywhere else. A plain instructor may view/manage an enrollment if they created it
|
||||
(`enrollments.created_by`) **or** the student shares one of their cohorts — never blanket
|
||||
visibility into every self-paced student, since a self-paced enrollment can have no cohort
|
||||
relationship at all. Enrollment *creation* stays admin/owner only, matching the existing
|
||||
roster-creation precedent (`POST /api/admin/students` is the same restriction).
|
||||
|
||||
## 11. The website/commerce integration boundary
|
||||
|
||||
`POST /api/integrations/enrollments` — completely separate from student/instructor session
|
||||
auth (no cookie is ever read), authenticated by a pre-shared secret header
|
||||
(`X-RootNode-Integration-Key`), the exact same trust-boundary pattern this codebase already
|
||||
uses for Gateway↔Provisioner (`X-Provisioner-Key`/`PROVISIONER_SHARED_SECRET`), not a new
|
||||
scheme. Full contract for the separate website team: `docs/WEBSITE_ENROLLMENT_INTEGRATION.md`.
|
||||
|
||||
Idempotent by construction: a unique index on `enrollments.external_event_id`, not an
|
||||
app-level "check then insert" — proven under real concurrency (three simultaneous identical
|
||||
requests produce exactly one enrollment, one `created: true`, two `created: false`) in both the
|
||||
permanent suite and manually during development. Student matching is create-or-match by email
|
||||
**or** login_identifier (§13's minor/no-email requirement) — reuses Phase 5A's exact
|
||||
`students-auth.js` credential-issuing path, not a second account-creation mechanism, and this is
|
||||
a trusted server-to-server caller (the website's own verified-payment backend), not a public
|
||||
signup form.
|
||||
|
||||
## 12. What was deliberately not built (§22)
|
||||
|
||||
Stripe/payment processing, checkout UI, the public website itself, public self-registration, a
|
||||
Parent Portal or full guardian accounts, email delivery infrastructure, certificates, complex
|
||||
drip scheduling, and full init_/compile_ curriculum content. Each is a plausible "while we're in
|
||||
here" addition; each was left alone on purpose. Nothing built this phase blocks any of them —
|
||||
`enrollments.external_reference`/`metadata` and the create-or-match student resolution are the
|
||||
deliberate extension points for a future guardian-purchases-for-minor flow, without inventing a
|
||||
fake guardian record now.
|
||||
|
||||
## 13. A real bug found and fixed
|
||||
|
||||
`POST /api/progress/record` predated the `wrap()` async-error-handling convention (it's a
|
||||
Phase 1/2A route, never updated). An unexpected internal error there — including Postgres
|
||||
rejecting a malformed `environment_id` — became an unhandled promise rejection that **crashed
|
||||
the entire Platform API process**, not just that one request. Found by this phase's own test
|
||||
suite. Fixed: the handler is now wrapped like every route added since Phase 3B, plus explicit
|
||||
UUID-shape validation so a bad `environment_id` is a clean 400 and never reaches Postgres as a
|
||||
type error at all. Regression-tested in `e2e-phase5b.js`.
|
||||
|
||||
## 14. Testing
|
||||
|
||||
- `platform/test/e2e-phase5b.js` (32 checks): program seed, entitlement gating
|
||||
(pending/future/expired/cancelled), cross-student isolation, real Lab launch/destroy/reaper/
|
||||
recreate persistence, server-derived idempotent completion, the full integration boundary
|
||||
(auth, idempotency, concurrency, unknown/inactive program rejection, minor/no-email path, no
|
||||
card data), RBAC scoping, Phase 5A regression smoke, and a real Platform API restart.
|
||||
- `infra/scripts/drills/test-phase5b-selfpaced-rehearsal.js` (15 checks): one continuous
|
||||
realistic story — activation, a simulated website enrollment event, real Lab work, Lab
|
||||
destroy, a new session later, Continue Learning, a fresh Lab, full capstone completion,
|
||||
program moves to Completed, and a second uninvolved student proven unable to reach any of it.
|
||||
- Both against a live `docker-compose.dev.yml` stack, real HTTP, real k3s Lab boots.
|
||||
- Migration safety proven explicitly: a fresh Postgres taken through Phase 1-5A's migrations
|
||||
only, seeded with realistic pre-5B data (credentialed student, cohort membership, progress,
|
||||
scenario attempts, achievements, classroom history), then 5B's migrations applied — every
|
||||
prior record intact and correctly associated with the same `student_id`; a new enrollment
|
||||
correctly links to that same pre-existing student; running the migration set a second time
|
||||
applies nothing.
|
||||
@@ -0,0 +1,248 @@
|
||||
# Phase 5C — Enrollment Operations & Communications
|
||||
|
||||
**Status: PHASE 5C COMPLETE. ENROLLMENT COMMUNICATIONS COMPLETE. STUDENT ACTIVATION PIPELINE READY.**
|
||||
**LIVE GOOGLE WORKSPACE DELIVERY VALIDATED** — real activation and password-reset emails
|
||||
delivered end to end via `[email protected]` (a Workspace alias), DKIM/SPF passing,
|
||||
inbox delivery confirmed. See §14.
|
||||
|
||||
Phase 5B built the enrollment/entitlement model and the website integration boundary. What was
|
||||
still missing for a real student: nobody ever told them. This phase closes that gap — real
|
||||
email delivery for activation, enrollment confirmation, and password reset — plus finally
|
||||
completing the production password-reset delivery Phase 5A explicitly left unfinished.
|
||||
|
||||
## 1. Architecture: provider-neutral by design
|
||||
|
||||
```
|
||||
Enrollment/Auth code (integration-routes.js, admin-routes.js, student-auth-routes.js)
|
||||
↓ (mint token if needed, then) notifications.enqueue()
|
||||
notification_events row, committed in the SAME request — no network I/O yet
|
||||
↓
|
||||
notification-worker.js (a setInterval poller, same pattern as sessions.js's sweeper)
|
||||
↓
|
||||
email/provider.js — EMAIL_PROVIDER env var picks the implementation
|
||||
↓
|
||||
email/dev-provider.js (default) OR email/gws-provider.js
|
||||
(never a real network call) (Gmail API, service account)
|
||||
```
|
||||
|
||||
No enrollment/auth code anywhere imports a concrete provider. Swapping Google Workspace for a
|
||||
different provider later means writing one new file under `platform/api/src/email/` and
|
||||
changing `EMAIL_PROVIDER` — nothing else changes.
|
||||
|
||||
## 2. Google Workspace delivery mechanism: Gmail API + service account domain-wide delegation
|
||||
|
||||
**Selected mechanism:** the Gmail API's `users.messages.send`, authenticated as a Google Cloud
|
||||
service account using Workspace domain-wide delegation to impersonate one dedicated sending
|
||||
mailbox.
|
||||
|
||||
**Why, not SMTP or a human login:**
|
||||
- **No human password, ever.** Basic SMTP auth (or an "app password") means storing credentials
|
||||
that log in as an actual person's Workspace account — anyone with that value could read that
|
||||
mailbox's mail, change its settings, or use it to send as that person. A service account has
|
||||
no login of its own and no inbox; domain-wide delegation grants it exactly one narrow OAuth
|
||||
scope (`gmail.send`) against one specific mailbox, nothing more.
|
||||
- **Least privilege.** `gmail.send` cannot read mail, cannot manage the mailbox, cannot act as
|
||||
any address other than the one configured. Compare to SMTP relay, which typically wants either
|
||||
a static allow-listed sending IP (fragile for infrastructure that isn't pinned yet) or a
|
||||
Workspace login.
|
||||
- **This is Google's own documented pattern** for "an application needs to send mail as a
|
||||
specific Workspace address" — see `docs/RUNBOOK_GOOGLE_WORKSPACE_EMAIL.md` for the exact
|
||||
setup steps and links.
|
||||
|
||||
**No `googleapis`/`google-auth-library` dependency.** The JWT-bearer OAuth flow (RS256-sign a
|
||||
short JWT, exchange it for an access token, POST one REST call) is ~120 lines using only Node's
|
||||
built-in `crypto` and `fetch` — consistent with this codebase's established "hand-roll before
|
||||
adding a dependency" convention (CSRF tokens, Prometheus exposition, and CSV parsing all do the
|
||||
same). See `platform/api/src/email/gws-provider.js`.
|
||||
|
||||
## 3. RootNode email identity
|
||||
|
||||
`EMAIL_FROM_NAME` / `EMAIL_FROM_ADDRESS` / `EMAIL_REPLY_TO` — all environment-configured, no
|
||||
default mailbox address baked into the code (`EMAIL_FROM_NAME` defaults to the display string
|
||||
"RootNode Academy"; `EMAIL_FROM_ADDRESS` has no default at all — sending fails closed with a
|
||||
clean `gws_not_configured` error until it's set, rather than guessing a real mailbox that
|
||||
doesn't exist in this repository's configuration). `EMAIL_FROM_ADDRESS` doubles as the Gmail API
|
||||
adapter's delegated-user (`sub`) claim — it must be a real mailbox in the Workspace domain,
|
||||
authorized for delegation (see the runbook).
|
||||
|
||||
DNS/domain requirements before any of this can send real mail in production — SPF, DKIM, DMARC,
|
||||
and Workspace's own sender-authorization step — are **entirely external to this repository**
|
||||
and are documented, not performed, in `docs/RUNBOOK_GOOGLE_WORKSPACE_EMAIL.md` (marked
|
||||
`REQUIRES GOOGLE WORKSPACE ADMIN ACTION`).
|
||||
|
||||
## 4. Notification model
|
||||
|
||||
`notification_events` (migration `021_notification_events.sql`): `student_id` (required — every
|
||||
notification this phase sends is about a specific student), `enrollment_id` (nullable —
|
||||
password-reset/changed notifications have none), `notification_type`
|
||||
(`activation`/`enrollment_confirmation`/`password_reset`/`password_changed`), `recipient_email`
|
||||
(a snapshot, not a live join — see §10), `status`
|
||||
(`queued`→`sending`→`sent`/`failed`, or `cancelled`, unused this phase but reserved),
|
||||
`attempt_count`/`next_attempt_at` for bounded exponential backoff, `last_error_code` (a
|
||||
classified string, never a raw stack trace), and `metadata` JSONB.
|
||||
|
||||
**Privacy note on `recipient_email`:** duplicates PII already stored elsewhere (`students.email`
|
||||
or an enrollment's `contact_email`) purely so the worker and any later delivery audit don't need
|
||||
to reconstruct "what address would this have gone to" after the student record potentially
|
||||
changes. Same sensitivity class as data already in this database — never a new exposure, and
|
||||
never returned unmasked through any admin API (`notifications.js`'s `maskEmail`/`toAdminView`
|
||||
are the only shapes any route returns).
|
||||
|
||||
## 5. The transient-plaintext-token design (read this before touching token handling)
|
||||
|
||||
Phase 5A's invariant — only a HASH of an activation/reset token is ever persisted long-term
|
||||
(`students.credential_token_hash`) — is unchanged and non-negotiable. But an activation/reset
|
||||
*email* needs the real token to build a working link, and the whole point of this phase is that
|
||||
the actual send happens later, asynchronously, in the worker — not in the request that
|
||||
originally minted the token.
|
||||
|
||||
The resolution: the triggering request mints the token exactly as Phase 5A already does
|
||||
(synchronously, in-request), and hands the plaintext to `notifications.enqueue()`, which stores
|
||||
it in that one row's `metadata.token` **only until the row reaches a terminal state**
|
||||
(`markSent`/`markFailedOrRetry` both scrub it — see their comments in `notifications.js`). This
|
||||
is a deliberate, narrow, documented exception for the row that exists specifically to deliver
|
||||
the plaintext once — not a second credential store, and never returned by any admin API
|
||||
regardless of whether it's still present (`toAdminView` never includes `metadata` at all).
|
||||
|
||||
Activation notifications from a **new enrollment** are the one case where no token is minted
|
||||
ahead of time and handed in — `notifyForEnrollment()` mints it itself, at enqueue time, because
|
||||
there's no other synchronous caller already holding one. Everywhere else (admin create/convert/
|
||||
reset-password, self-service forgot-password, admin resend), the existing Phase 5A minting call
|
||||
site is reused unchanged and its plaintext is simply also passed to `enqueue()`.
|
||||
|
||||
## 6. Activation flow
|
||||
|
||||
Reuses Phase 5A's activation token mechanism entirely — no competing invitation-token system was
|
||||
built. `issueCredentialToken` already unconditionally invalidates any previous outstanding token
|
||||
on mint (Phase 5A), which means **"resend" and "regenerate" are mechanically the same
|
||||
operation**: there is never a moment with two simultaneously valid activation links, so a second
|
||||
button that did something different would have nothing different to do. `POST /api/admin/
|
||||
students/:id/resend-activation` is the one action that covers both.
|
||||
|
||||
```
|
||||
Enrollment event/admin enrollment → student matched/created
|
||||
→ credentials already active? → enrollment_confirmation (§7)
|
||||
→ else → issueCredentialToken('activation') → notifications.enqueue()
|
||||
→ worker delivers via the configured provider
|
||||
→ student opens the link → existing Set Password flow (Phase 5A, unchanged)
|
||||
→ token consumed atomically with setting the password
|
||||
→ account active, student can log in
|
||||
```
|
||||
|
||||
## 7. Enrollment confirmation flow
|
||||
|
||||
Fires whenever a **new enrollment** is created (integration event or admin manual enroll) for a
|
||||
student who **already has credentials** (`students.password_hash IS NOT NULL`). No token, no
|
||||
link beyond "go sign in" — `notifyForEnrollment()` (shared by both creation paths) is the single
|
||||
decision point, so a website-triggered and an admin-triggered enrollment behave identically.
|
||||
|
||||
## 8. Password reset flow — now actually delivered
|
||||
|
||||
Phase 5A's `POST /api/student/request-password-reset` already minted a token and, only outside
|
||||
production, echoed it back for local/test use. Phase 5C adds exactly one thing: the same
|
||||
minted token is now also handed to `notifications.enqueue()`, so a real email goes out. Every
|
||||
other behavior is unchanged: the response is the same generic anti-enumeration message
|
||||
regardless of whether the account exists, is inactive, or was never activated (this phase
|
||||
additionally excludes never-activated accounts, a real fix — see §13); production never returns
|
||||
the token, dev/test still can. `POST /api/student/reset-password` (consuming the token) now also
|
||||
enqueues a `password_changed` security-notice email once the new password is set.
|
||||
|
||||
## 9. Password-changed / security notice
|
||||
|
||||
Fires after both `POST /api/student/change-password` (self-service) and `POST /api/student/
|
||||
reset-password` (token consumption) succeed — no token, no link, purely "this happened, if it
|
||||
wasn't you here's what to do."
|
||||
|
||||
## 10. Minor/no-email communication design
|
||||
|
||||
Phase 5A/5B's minor-safe model (a student may have `login_identifier` with no `email` at all) is
|
||||
preserved exactly. `enrollments.metadata.contact_email` (an existing JSONB column, not a new
|
||||
migration) holds an OPTIONAL "who to notify about THIS enrollment" address — a guardian's email
|
||||
collected at checkout, for example. **It is never written to `students.email`** — that field
|
||||
stays reserved for the student's own identity/login credential; a guardian is not a student.
|
||||
`enrollment.js`'s `enrollmentContactEmail()` is the one accessor with that one meaning; recipient
|
||||
resolution (`notifications.js`'s `resolveRecipient`) tries the student's own email first, falls
|
||||
back to the enrollment's contact email only for enrollment-scoped notification types, and
|
||||
correctly has no fallback at all for password-reset/changed (which require the student to
|
||||
already be able to log in themselves). This is deliberately transitional — not a guardian
|
||||
account, not a linked identity, no relationship table — and does not block a future real
|
||||
Guardian Portal from being built on top of it later.
|
||||
|
||||
## 11. Website → enrollment → communication
|
||||
|
||||
`POST /api/integrations/enrollments` (Phase 5B, unchanged) now also decides and enqueues the
|
||||
right notification, but **only when its own enrollment-creation call reports `created: true`**
|
||||
(a genuinely new enrollment, not an idempotent replay). This is the entire duplicate-webhook
|
||||
safety net for notifications: since Phase 5B's DB-level idempotency already guarantees at most
|
||||
one caller ever sees `created: true` for a given event/race, at most one notification is ever
|
||||
queued, with zero new dedup mechanism needed.
|
||||
|
||||
## 12. Retry/failure handling
|
||||
|
||||
`markFailedOrRetry` classifies every provider error as `temporary` or `permanent` (both
|
||||
providers' `send()` throw an `Error` with `.errorClass`/`.errorCode` set). Permanent failures
|
||||
(bad recipient, missing/invalid Google config) go straight to `failed` on the first attempt —
|
||||
retrying can't fix them. Temporary failures (network/provider hiccups, rate limiting) requeue
|
||||
with bounded exponential backoff (30s base, doubling, capped at 1 hour) until `MAX_ATTEMPTS` (5)
|
||||
is exhausted, then `failed` too. Never an infinite loop; every attempt is persisted
|
||||
(`attempt_count`, `last_error_code`).
|
||||
|
||||
## 13. Real bugs found and fixed
|
||||
|
||||
- `request-password-reset` would previously mint a `reset` token for a student who had **never
|
||||
been activated** (`password_hash IS NULL`) — a "reset" for an account with no password to
|
||||
reset doesn't make semantic sense, and would have sent a confusing "reset your password" email
|
||||
to someone who needs activation, not reset. Fixed: that branch now also checks
|
||||
`student.password_hash` and falls through to the same generic anti-enumeration response,
|
||||
exactly as if the account didn't exist.
|
||||
- **HTML injection in email templates.** `displayName` (which can come straight from a website
|
||||
integration event's `student.display_name` — genuinely external, caller-controlled text, not
|
||||
admin-typed) was interpolated directly into the HTML email body with no escaping. `templates.js`
|
||||
now escapes every user-controlled value before it reaches HTML (`escapeHtml`); the plain-text
|
||||
bodies are correctly left unescaped, since there's no markup there to break. Both fixes are
|
||||
regression-tested in `e2e-phase5c.js`.
|
||||
|
||||
## 14. Testing
|
||||
|
||||
- `platform/test/e2e-phase5c.js` (28 checks): activation vs enrollment-confirmation decision,
|
||||
duplicate/concurrent webhook idempotency, temporary/permanent/exhausted retry classification,
|
||||
resend/regenerate token invalidation, activation/reset single-use and expiration, production
|
||||
token-secrecy gating (a real container recreate with `COOKIE_SECURE=1`), minor/contact-email
|
||||
resolution, RBAC scoping, metrics/audit hygiene, and a real Platform API restart.
|
||||
- `infra/scripts/drills/test-phase5c-enrollment-activation.js` (10 checks): one continuous
|
||||
story — website enrollment → activation email rendered and its URL extracted exactly like a
|
||||
real inbox click → password set → real Lab launched → real scenario completed → a second
|
||||
enrollment gets confirmation not activation → full password-reset round trip with session
|
||||
revocation → the minor/contact-email path.
|
||||
- Both against a live `docker-compose.dev.yml` stack, using the deterministic dev email provider
|
||||
exclusively — **no test in this phase ever makes a real network call to Google or anyone
|
||||
else.**
|
||||
- Migration safety proven explicitly against a fresh Postgres taken through every prior phase's
|
||||
migrations, seeded with realistic Phase 5B data (credentialed student, minor/no-email student,
|
||||
cohort, enrollment with a contact-email, progress, attempts, achievements, classroom history),
|
||||
then this phase's migration applied — every prior record intact; a new notification correctly
|
||||
links to the pre-existing student/enrollment; re-running the migration set is a no-op.
|
||||
|
||||
**Live Google Workspace delivery has been validated for real**, both in isolation
|
||||
(`infra/scripts/gws-email-test.js`, one message, DKIM/SPF passing, delivered) and through the
|
||||
full genuine flow: a real student created, enrolled, receiving a real activation email, setting
|
||||
their password from the real link, logging in, seeing their enrolled program, requesting and
|
||||
receiving a real password-reset email, and completing the reset — every step against the live
|
||||
Platform API, `EMAIL_PROVIDER=gws`, nothing simulated.
|
||||
|
||||
**One real Google Workspace limitation found and handled:** newly-authorized domain-wide
|
||||
delegation does not take effect instantly across all of Google's auth infrastructure — the
|
||||
identical token request succeeded, then failed with `unauthorized_client`, then succeeded again,
|
||||
over the span of about a minute, before settling. This is Google-side propagation delay, not a
|
||||
configuration error, and it self-resolved. `gws-provider.js` now classifies `unauthorized_client`
|
||||
as a *temporary* failure (retried with the existing bounded backoff) rather than permanent —
|
||||
this is exactly the failure mode that backoff exists for, and it means a real deployment doesn't
|
||||
need a human to notice and manually retry during that propagation window. See
|
||||
`docs/RUNBOOK_GOOGLE_WORKSPACE_EMAIL.md`'s troubleshooting section.
|
||||
|
||||
**A second real-world detail found during setup, not a bug:** the sending address
|
||||
(`[email protected]`) is a Workspace *alias* of an existing account rather than its
|
||||
own paid seat — a deliberate cost-saving choice. Domain-wide delegation authorizes a real
|
||||
account, not an arbitrary alias string, so `GOOGLE_DELEGATED_USER` (the alias's real owner) was
|
||||
added as a setting distinct from `EMAIL_FROM_ADDRESS` (the visible sending address) — see §2 and
|
||||
the runbook §5.
|
||||
@@ -0,0 +1,138 @@
|
||||
# Phase 6A — ROOT_ Course Architecture + Content Delivery Foundation
|
||||
|
||||
**Status: PHASE 6A COMPLETE. COURSE/MODULE/ACTIVITY CONTENT MODEL LIVE. GOLD-STANDARD MODULE
|
||||
READY, VALIDATED AGAINST A REAL LAB.**
|
||||
|
||||
Phases 1-5 built the RootNode Academy platform: identity, cohorts, curriculum, the assessment
|
||||
engine, remote classrooms, self-paced enrollment, and enrollment communications. All of it sat
|
||||
on top of `scenarios/rootnode-path.json` — a flat list of scenario ids per stage, with no notion
|
||||
of a lesson, a demonstration, or "learn this before you're asked to solve it." Phase 6A builds
|
||||
that layer, for the `root_` track only, without touching any of the systems underneath it.
|
||||
|
||||
## 1. The canonical youth ladder (context, not built here)
|
||||
|
||||
RootNode Academy's product spans three tracks by age/stage:
|
||||
|
||||
- **`init_`** (~5-10) — robotics / computational thinking / block coding. Not built.
|
||||
- **`compile_`** (~11-14) — electronics / microcontrollers / ESP32 / programming. Not built.
|
||||
- **`root_`** (~15-18+, college, adult) — Linux → systems → containers → Kubernetes → DevOps/
|
||||
platform engineering. **This phase.**
|
||||
|
||||
`programs.family` (Phase 5B) already models this three-way split; only `root` has real content.
|
||||
Phase 6A does not invent `init_`/`compile_` curriculum — see §15 of the authorization and
|
||||
`docs/ROOT_CURRICULUM_INVENTORY.md` §7 for the evidence that no such content exists yet.
|
||||
|
||||
## 2. Core product principle
|
||||
|
||||
KubeKosh/the RootNode Lab is the LABORATORY, not the whole course. A RootNode course can contain
|
||||
lessons, demonstrations, examples, knowledge checks, guided exercises, labs (existing
|
||||
scenarios), assessments, reflections, and capstone work — a student learns a concept BEFORE
|
||||
being asked to solve the corresponding scenario. See `docs/COURSE_AUTHORING_STANDARD.md`.
|
||||
|
||||
## 3. Google Workspace status (recorded per this phase's instruction, not new work)
|
||||
|
||||
The external blocker at the end of Phase 5C — no real Google Workspace mailbox to test against —
|
||||
was resolved by the owner during this session before Phase 6A began: `[email protected]`
|
||||
now exists and live delivery was validated end to end (isolated send test + a full real
|
||||
activation/reset lifecycle). This is already recorded in `docs/PHASE5C_ENROLLMENT_COMMUNICATIONS.md`
|
||||
(status header, §14) and `docs/RUNBOOK_GOOGLE_WORKSPACE_EMAIL.md` (§11's propagation-delay note),
|
||||
committed at `4c0a950` and tagged `v0.5.3-enrollment-comms` — this phase did not need to redo
|
||||
that work, only confirm it was recorded, which it was.
|
||||
|
||||
## 4. Architecture summary
|
||||
|
||||
```
|
||||
program (existing) ── courses ── course_modules ── module_activities
|
||||
├─ lab → scenario_id (existing)
|
||||
└─ assessment/capstone → assessment_definition_id (existing)
|
||||
```
|
||||
|
||||
Full design: `docs/COURSE_CONTENT_MODEL.md`. Full course/module map:
|
||||
`docs/ROOT_PROGRAM_ARCHITECTURE.md`. Full existing-content inventory this was built from:
|
||||
`docs/ROOT_CURRICULUM_INVENTORY.md`.
|
||||
|
||||
Delivery modes: the same content model serves both instructor-led (Phase 5A classrooms) and
|
||||
self-paced (Phase 5B enrollment) students — one `buildCourseTree()` function
|
||||
(`platform/api/src/course-progress.js`), one `content.js` Markdown loader, consumed by both
|
||||
`course-routes.js` (student) and `admin-course-routes.js` (instructor), the latter additionally
|
||||
exposing instructor notes via the same file's `## Instructor Notes` convention. No forked
|
||||
curriculum.
|
||||
|
||||
## 5. What was built
|
||||
|
||||
**Migration** — `platform/api/migrations/022_course_content.sql`: `courses`, `course_modules`,
|
||||
`module_activities`, `activity_completions`, plus seed data for 13 courses, 8 modules, and 51
|
||||
activities (44 mechanically mapped from existing `rootnode-path.json` stages, 6 gold-module
|
||||
activities, 1 capstone activity) — see that file's own extensive header comments.
|
||||
|
||||
**Content** — `content/root/root-10-kubernetes-troubleshooting-observability/m01-health-probes/`:
|
||||
6 Markdown files (lesson, demonstration, knowledge check, guided exercise, lab intro,
|
||||
reflection), each following the shared/instructor-notes split convention.
|
||||
|
||||
**Backend** — `content.js` (Markdown loader), `course-progress.js` (completion derivation +
|
||||
prerequisite locking, shared by both surfaces), `course-routes.js` (student API),
|
||||
`admin-course-routes.js` (instructor API), wired into `server.js`. `platform/api/Dockerfile`
|
||||
now also copies `content/`.
|
||||
|
||||
**Frontend** — `CourseView.jsx` (student: My Programs → ROOT_ → Course → Module → Activity →
|
||||
Launch Lab → back → continue), wired into `ProgramsTab.jsx`'s existing "Continue Learning" flow
|
||||
via a new "View Courses" button (the original direct-to-Lab button is unchanged).
|
||||
`CurriculumView.jsx` (instructor: program structure, per-module teaching detail, instructor
|
||||
notes, per-student progress lookup), wired into `AdminPortal.jsx` as a new "Curriculum" tab.
|
||||
|
||||
**Tests** — `platform/test/e2e-phase6a.js` (21 assertions: hierarchy, ordering, all three
|
||||
completion behaviors, locking, resume, cross-student isolation, RBAC, draft-hiding, version
|
||||
stability) and `infra/scripts/drills/test-phase6a-course-rehearsal.js` +
|
||||
`run-phase6a-course-rehearsal.sh` (a real-k3s dress rehearsal of the gold module end to end,
|
||||
including the Lab's own `setup_commands` and a real `kubectl patch` against the live cluster).
|
||||
|
||||
## 6. The gold-standard module
|
||||
|
||||
`root-10-m01-health-probes-and-troubleshooting`, under ROOT 10 (Kubernetes Troubleshooting &
|
||||
Observability): lesson (probe concepts) → demonstration (reading `kubectl describe pod` Events
|
||||
for a probe failure) → knowledge check → guided exercise (write a readiness probe on paper) →
|
||||
Launch Lab (`rootnode-probe-port-mismatch`, RootNode-authored) → reflection (root cause + Real
|
||||
Ops prevention takeaway). Instructor notes on 5 of its 6 activities (demo plan, misconceptions,
|
||||
timing, discussion questions, extensions). This is the template `docs/COURSE_AUTHORING_STANDARD.md`
|
||||
documents for all future module authoring.
|
||||
|
||||
## 7. Honest content states
|
||||
|
||||
No course is `ready`. 5 courses (01-04, 11) are `planned` with zero modules — genuinely no
|
||||
content exists for them. 7 courses (05-10, 12, capstone) are `partial` — real, working labs
|
||||
mapped in, one fully `ready` module (the gold standard), everything else awaiting lesson
|
||||
authoring in Phase 6B. Nothing was fabricated to make the UI look more complete than it is.
|
||||
|
||||
## 8. Test results
|
||||
|
||||
- `e2e-phase6a.js`: **21 passed, 0 failed.**
|
||||
- `test-phase6a-course-rehearsal.js` (real k3s Lab): **12 passed, 0 failed**, including a real
|
||||
`kubectl patch` against a live cluster inside the student's actual disposable Lab container.
|
||||
- Full regression (Phase 2A/2B/3A/3B/5A/5B/5C) — see the final report for the run-by-run result;
|
||||
no Phase 6A change touched any file those suites exercise other than additive route
|
||||
registration in `server.js`.
|
||||
|
||||
## 9. What Phase 6A deliberately did not build
|
||||
|
||||
Per the authorization's explicit scope line: `init_`/`compile_` curriculum, a Parent Portal,
|
||||
Stripe/payments, public registration, an AI tutor, chat/video/voice, proctoring, a mobile app,
|
||||
certificates, a multi-host scheduler, production deployment changes, a CMS, or dozens of
|
||||
placeholder lessons. Lesson authoring for courses 06-10/12/capstone beyond the gold standard is
|
||||
explicitly Phase 6B+ scope.
|
||||
|
||||
## 10. Known limitations / recommended Phase 6B scope
|
||||
|
||||
- ROOT 08/09's real, existing scenario content (storage/scheduling, and a 22-scenario CKS
|
||||
security bank) is inventoried but not yet wired into modules — the natural next content slice.
|
||||
- Course-level lesson authoring for ROOT 05-07/10 (m02)/12/capstone: labs exist, teaching
|
||||
content doesn't yet — apply `docs/COURSE_AUTHORING_STANDARD.md` per module.
|
||||
- `POST /api/assessments/:definitionId/start` does not itself re-check the capstone's
|
||||
`unlock_rule` server-side (a pre-existing Phase 3B gap, not introduced by this phase — the
|
||||
student-facing `/api/capstone` view does compute and display lock state correctly, and the
|
||||
UI never exposes a "start" action while locked, but the raw route would technically accept a
|
||||
premature start if called directly). Not fixed here — it's outside the course/content-model
|
||||
scope this phase was authorized for; worth a small, isolated fix in a future phase.
|
||||
- Course-to-course prerequisite locking is currently sequential/manually declared, not derived
|
||||
from the assessment engine's own `unlock_rule` JSON — the two happen to agree today by
|
||||
construction (see `docs/ROOT_PROGRAM_ARCHITECTURE.md`'s "Real Ops and Capstone placement");
|
||||
a future phase could unify them if course content grows complex enough to warrant it.
|
||||
@@ -0,0 +1,155 @@
|
||||
# Phase 6B — ROOT_ Curriculum Authoring & Coverage Expansion
|
||||
|
||||
**Status: PHASE 6B COMPLETE. 13 NEW TEACHING MODULES AUTHORED ACROSS ROOT 05-10/12/CAPSTONE.
|
||||
CAPSTONE SERVER-SIDE UNLOCK ENFORCEMENT FIXED. VALIDATED AGAINST A REAL LAB.**
|
||||
|
||||
Phase 6A built the course/module/activity content model and one gold-standard module. Phase 6B
|
||||
is the primary content-authoring phase: real, substantial teaching content across the
|
||||
Kubernetes/operations portion of `root_`, built strictly from the existing 98-scenario
|
||||
inventory — no new scenarios were needed or created (see §7).
|
||||
|
||||
## 1. What was authored
|
||||
|
||||
13 new modules across 7 courses, all evidence-mapped in `docs/ROOT_CURRICULUM_COVERAGE_MATRIX.md`:
|
||||
|
||||
| Course | New module(s) | Depth |
|
||||
|---|---|---|
|
||||
| ROOT 05 | Kubernetes Mental Model & Core Objects | **READY** — full gold-standard pattern |
|
||||
| ROOT 06 | Deployments, Scaling & Updates | **READY** — full gold-standard pattern |
|
||||
| ROOT 06 | Jobs, CronJobs & DaemonSets | PARTIAL — lesson + knowledge check + labs |
|
||||
| ROOT 07 | Services & Service Discovery | **READY** — full gold-standard pattern |
|
||||
| ROOT 07 | Ingress & NetworkPolicy Foundations | PARTIAL — lesson + knowledge check + labs |
|
||||
| ROOT 08 | Storage Fundamentals | PARTIAL — lesson + knowledge check + labs |
|
||||
| ROOT 08 | Scheduling & Resource Governance | PARTIAL — lesson + knowledge check + labs |
|
||||
| ROOT 09 | Security Model & SecurityContext | **READY** — full gold-standard pattern + guided/independent/advanced practice |
|
||||
| ROOT 09 | ServiceAccounts & RBAC | PARTIAL — lesson + knowledge check + labs |
|
||||
| ROOT 09 | Network & Supply Chain Security | PARTIAL — lesson + knowledge check + labs |
|
||||
| ROOT 09 | Cluster Hardening & Runtime Security | PARTIAL — survey-depth lesson + labs |
|
||||
| ROOT 10 | The RootNode Troubleshooting Framework | PARTIAL — lesson + demonstration + knowledge check + labs |
|
||||
| ROOT 12 | Incident-response framing added to the existing Real Ops module | PARTIAL — framework lesson + reflection wrapping the existing incidents |
|
||||
| ROOT Capstone | Capstone Readiness | **READY** — orientation module, see `docs/ROOT_CAPSTONE_READINESS.md` |
|
||||
|
||||
3 modules reach **READY** by the honest bar in `docs/COURSE_AUTHORING_STANDARD.md` (introduction,
|
||||
objectives, concept lesson, demonstration, knowledge check, guided practice, lab, reflection,
|
||||
instructor notes, estimated duration, all present) — plus the pre-existing gold-standard Health
|
||||
Probes module, unchanged. Every other new module is honestly **PARTIAL**: real lesson content
|
||||
and real labs, missing one or more of demonstration/guided-exercise (documented per-module in the
|
||||
coverage matrix, not glossed over).
|
||||
|
||||
ROOT 08 and ROOT 09 moved from **PLANNED** to **PARTIAL** at the course level — real content now
|
||||
exists where Phase 6A left them structurally empty.
|
||||
|
||||
## 2. The is_required extension (Phase 6B, not Phase 6A)
|
||||
|
||||
`module_activities.is_required` (new column, defaults `true` — every Phase 6A activity is
|
||||
unaffected) lets a module distinguish CORE activities from GUIDED/INDEPENDENT-practice
|
||||
duplicates and ADVANCED/optional extras (ROOT 09's CKS-style content, mostly). A module is
|
||||
"complete" once every REQUIRED activity is done — an incomplete optional activity never blocks
|
||||
it, though it still counts toward the displayed percent. See `docs/COURSE_CONTENT_MODEL.md`'s
|
||||
updated completion-rules section.
|
||||
|
||||
## 3. The capstone server-side unlock fix
|
||||
|
||||
Full detail in `docs/ROOT_CAPSTONE_READINESS.md` §4. Summary: `POST /api/assessments/
|
||||
:definitionId/start` previously never checked `unlock_rule` — a direct API call could start the
|
||||
capstone before its prerequisite stages were genuinely complete, even though the UI never
|
||||
exposed a way to trigger this. Fixed by binding to the same `computeCurriculumState()` gate
|
||||
`GET /api/capstone` already used. Found while fixing this: `e2e-phase5b.js`'s own capstone test
|
||||
had been passing only because of this exact gap (it never completed the `real-ops` stage before
|
||||
attempting to start) — fixed alongside the platform code, not routed around.
|
||||
|
||||
## 4. Content quality bar applied honestly
|
||||
|
||||
No module was marked READY merely for having activity rows, headings, or scenario references.
|
||||
The ROOT 08/09 storage/scheduling/security modules are real, substantive lessons with real
|
||||
knowledge checks — but without a demonstration or guided exercise, they're PARTIAL, not READY,
|
||||
by this project's own stated standard. See the coverage matrix for the specific gap in each.
|
||||
|
||||
## 5. Scenario coverage before/after
|
||||
|
||||
Verified by direct query against `module_activities.scenario_id`, not estimated.
|
||||
|
||||
Before Phase 6B: 44 of 98 scenarios wired into structured curriculum (all via Phase 6A's
|
||||
mechanical mapping of `rootnode-path.json`'s existing stages).
|
||||
|
||||
After Phase 6B: **88 of 98 scenarios (89.8%) now in structured curriculum** — 44 newly wired
|
||||
this phase: `deployment-rollback` (added to ROOT 06 m02 as hands-on rollback practice
|
||||
alongside its lesson), `job-task`, `cronjob-task` (ROOT 06 m03); `nodeport-task`,
|
||||
`dns-resolution-mcq` (ROOT 07 m01), `ingress-task`, `network-policy` (ROOT 07 m04);
|
||||
`emptydir-pod`, `pv-pvc-mount`, `pvc-dynamic-task`, `storageclass-mcq` (ROOT 08 m01),
|
||||
`node-label-selector`, `node-taint-toleration`, `pod-affinity-mcq`, `priority-class-task`,
|
||||
`pod-disruption-budget-task`, `resource-quota-ns`, `limitrange-task` (ROOT 08 m02);
|
||||
`cks-non-root-enforce`, `cks-drop-capabilities`, `cks-readonly-filesystem`,
|
||||
`cks-pod-security-context`, `cks-seccomp-profile`, `cks-apparmor-profile`,
|
||||
`pod-security-context` (ROOT 09 m01); `serviceaccount-pod`, `cks-automount-token`, `rbac-role`,
|
||||
`cks-rbac-clusterrole`, `cks-rbac-least-privilege`, `rbac-clusterrole` (ROOT 09 m02);
|
||||
`cks-network-policy`, `cks-egress-namespace`, `cks-tls-ingress`, `cks-image-pull-secret`,
|
||||
`networkpolicy-egress`, `cks-network-policy-metadata` (ROOT 09 m03); `cks-psa-namespace`,
|
||||
`cks-immutable-secret`, `cks-runtime-class`, `cks-audit-policy` (ROOT 09 m04); `crashloop-fix`,
|
||||
`endpoint-fix-task` (ROOT 10 m03), `container-logging-mcq` (ROOT 10 m02, existing bank).
|
||||
|
||||
**10 scenarios remain unwired**, each with a specific, documented reason rather than an
|
||||
oversight: `cluster-upgrade-mcq`, `etcd-backup-mcq` (ROOT 11/DevOps territory, explicitly out of
|
||||
Phase 6B's authoring scope); 7 CKS-style MCQs classified as ASSESSMENT BANK material
|
||||
(`cks-mcq-api-server`, `cks-mcq-audit-policy`, `cks-mcq-kubelet-auth`, `cks-mcq-psp-replacement`,
|
||||
`cks-mcq-runtime-security`, `cks-mcq-sandboxing`, `cks-mcq-image-footprint`) — recall-style
|
||||
questions better suited to a future structured exam pool than hands-on core teaching, flagged
|
||||
for that future use rather than silently dropped; and `readiness-probe-task` — a real, unused
|
||||
candidate for independent practice in ROOT 06 or ROOT 10 that this phase simply didn't get to,
|
||||
honestly left for Phase 6C rather than force-mapped at the last minute. Quality of mapping, not
|
||||
percentage, drove every placement decision — see the coverage matrix's "duplicate concepts
|
||||
identified" section for how near-duplicate scenarios were handled deliberately rather than both
|
||||
force-fit into "core."
|
||||
|
||||
## 6. Knowledge-check and guided-exercise approach
|
||||
|
||||
Every new knowledge check uses scenario-interpretation and reasoning questions ("A Deployment
|
||||
has three replicas but the Service has zero endpoints — which part of the system should you
|
||||
inspect first, and why?"), never "what command was shown two paragraphs ago." Every guided
|
||||
exercise bridges lesson to lab without revealing the lab's answer — modeled directly on the
|
||||
existing gold module's own guided exercise. No new knowledge-check engine capability was needed;
|
||||
the existing Markdown + manual-marker model was sufficient throughout.
|
||||
|
||||
## 7. Why no new scenarios were created
|
||||
|
||||
Every learning objective this phase authored had a suitable existing scenario — the 54-scenario
|
||||
unused bank Phase 6A inventoried was exactly the content Phase 6B needed. No objective was found
|
||||
that genuinely required new hands-on lab content with no existing scenario able to serve it, so
|
||||
no new `rootnode-*` scenario was created this phase (see the authorization's explicit
|
||||
justification bar in its §17 — none of the three required conditions were met for any specific
|
||||
gap identified during authoring).
|
||||
|
||||
## 8. What Phase 6B deliberately did not build
|
||||
|
||||
Per the authorization's explicit scope line: `init_`/`compile_` curriculum, deep authoring of
|
||||
ROOT 01-04 or ROOT 11 (both remain structurally PLANNED, as Phase 6A left them — genuinely no
|
||||
scenario content exists for either), a Parent Portal, payments, public registration, an AI
|
||||
tutor, chat/video/voice, proctoring, a mobile app, certificates, a multi-host scheduler,
|
||||
production deployment changes, an LMS/CMS, a WYSIWYG authoring system, or complex drip
|
||||
scheduling. The Provisioner, Gateway, Lab Router, student/instructor authentication, enrollment
|
||||
architecture, and notification architecture were not redesigned — the one fix made
|
||||
(capstone unlock) was the smallest correct change to the assessment engine's existing route,
|
||||
not a redesign of the engine itself.
|
||||
|
||||
## 9. Test results
|
||||
|
||||
- `e2e-phase6b.js`: **24 passed, 0 failed.**
|
||||
- `test-phase6b-root-course-rehearsal.js` (real k3s Lab, real `kubectl expose`, real classroom
|
||||
tooling): **11 passed, 0 failed.**
|
||||
- Full regression (2A/2B/3A/3B/5A/5B/5C/6A), each run in isolation per the convention Phase 6A
|
||||
established — see the final report for the run-by-run result and the two pre-existing test
|
||||
gaps found and fixed along the way (in `e2e-phase5b.js` and `e2e-phase6a.js`, both caused by
|
||||
this phase's own intentional, documented behavior changes, not by a defect).
|
||||
|
||||
## 10. Known limitations / recommended Phase 6C scope
|
||||
|
||||
- ROOT 06/07/08/09/10's PARTIAL modules are missing a demonstration and/or guided exercise —
|
||||
the natural next authoring pass, module by module, using the 3 READY modules as templates.
|
||||
- ROOT 09's 7 ASSESSMENT-BANK-classified MCQs are inventoried but not yet wired into an actual
|
||||
assessment definition.
|
||||
- ROOT 11 (DevOps & Platform Engineering) remains entirely PLANNED — no scenario content exists
|
||||
anywhere in this repository for CI/CD, IaC, or GitOps concepts; a future phase would need to
|
||||
either author new `rootnode-*` scenarios for it or explicitly scope it out of `root_`.
|
||||
- The Capstone Readiness module's completion is intentionally independent of the real capstone's
|
||||
unlock gate (see `docs/ROOT_CAPSTONE_READINESS.md` §3) — worth revisiting only if a future
|
||||
phase has a concrete reason to couple them, not as a default assumption.
|
||||
@@ -0,0 +1,150 @@
|
||||
# Phase 6C — Curriculum Completion + Assessment Bank + ROOT 11 Scope
|
||||
|
||||
Phase 6C's mission: upgrade PARTIAL modules toward the gold-module standard, build a real
|
||||
assessment bank from previously-unwired content, resolve ROOT 11's scope, and produce an
|
||||
explicit, honest pilot-ready determination. This document records what was actually done and
|
||||
why, using the existing content/assessment model throughout — no new application code was
|
||||
required.
|
||||
|
||||
## Baseline verification
|
||||
|
||||
Confirmed at start: branch `main`, clean tree, HEAD = `origin/main` =
|
||||
`2db136875f30809afa9c82f36e96605802bf7ee2`, descending from the Phase 6B milestone
|
||||
(`git merge-base --is-ancestor`, confirmed). Phase 6B's own final report claimed 17 PARTIAL
|
||||
modules; a direct database query showed the real figure was 15 out of 21 total modules — this
|
||||
correction is recorded in `docs/ROOT_CURRICULUM_COVERAGE_MATRIX.md`.
|
||||
|
||||
## Readiness standard
|
||||
|
||||
`docs/ROOT_PILOT_READINESS_STANDARD.md` defines READY as sufficiency, not a fixed activity
|
||||
count: enough instruction/scaffolding that a student can reasonably perform the practical work,
|
||||
checked against an 8-point standard (concept instruction, demonstrated reasoning, self-check,
|
||||
guided bridge, operational-context closing, instructor notes, self-paced sufficiency, resolved
|
||||
references). Modules were checked against this bar individually, not mechanically padded to six
|
||||
activities each.
|
||||
|
||||
## Modules upgraded (15 PARTIAL → 6 PARTIAL, 16 READY)
|
||||
|
||||
**Full gold-pattern upgrades** (lesson/demonstration/knowledge-check/guided-exercise/lab(s)/
|
||||
reflection), each with new content authored and existing lab activities preserved and only
|
||||
resequenced:
|
||||
- `root-06-m03-jobs-cronjobs-daemonsets`
|
||||
- `root-07-m04-ingress-networkpolicy-foundations`
|
||||
- `root-07-m02-configuration-and-secrets` (converted in place from a bare 5-lab bank — the last
|
||||
networking/config module with no front-door teaching)
|
||||
- `root-09-m02-serviceaccounts-and-rbac`
|
||||
- `root-09-m03-network-and-supplychain-security`
|
||||
- `root-10-m03-troubleshooting-framework` (only guided-exercise + reflection were missing)
|
||||
|
||||
**Bookend-only upgrade** (lesson + reflection wrapped around an existing, otherwise-complete
|
||||
practice bank, rather than a full six-part redesign — because the gap was framing, not
|
||||
per-task teaching):
|
||||
- `root-05-m01-pods-and-kubectl-basics` — 20 independent kubectl-fluency tasks, each already
|
||||
carrying its own explanation in its scenario description. Added a short lesson explaining why
|
||||
the module is structured as breadth practice, and a reflection asking students to name their
|
||||
own remaining gaps.
|
||||
|
||||
**Documented exceptions** (status promoted to READY with no new activities, because adding the
|
||||
"missing" activity type would have actively undermined the module's intentional design):
|
||||
- `root-12-m01-real-ops-incidents` — adding a knowledge-check or guided-exercise would
|
||||
contradict its deliberate "you are responsible for this environment" unguided design.
|
||||
- `root-capstone-m01-operate-and-recover` — its single `capstone` activity IS the terminal
|
||||
exam; the readiness/prep teaching already lives in the preceding, already-READY
|
||||
`root-capstone-m02-capstone-readiness`.
|
||||
- `root-09-m04-cluster-hardening-and-runtime` — Phase 6B already documented this as
|
||||
intentionally survey-depth ("not core, not daily-driver"); it now has a lesson, hands-on
|
||||
labs, and a real assessment checkpoint (below), which is sufficient for its scope without
|
||||
forcing a demonstration/guided-exercise pattern that would misrepresent it as core content.
|
||||
|
||||
**Intentionally deferred, not upgraded this phase:**
|
||||
- `root-08-m01-storage-fundamentals`, `root-08-m02-scheduling-resource-governance` — ROOT 08
|
||||
does not gate the capstone (the `rootnode-path.json` unlock stages are foundation, workloads,
|
||||
networking, configuration, troubleshooting, real-ops; storage/scheduling was never one of
|
||||
them). Given realistic effort constraints, these were prioritized below the
|
||||
capstone-critical path and below the security-assessment mandate. Left PARTIAL, classified
|
||||
DEFERRED/optional-supplementary in the pilot manifest.
|
||||
- `root-06-m01-deployments-and-workloads`, `root-07-m01-services-and-networking`,
|
||||
`root-10-m02-general-troubleshooting` — dense hands-on practice banks that sit alongside an
|
||||
already-fully-taught sibling module in the same course (root-06-m02, root-07-m03/m04,
|
||||
root-10-m01/m03). Rather than author three more full teaching layers under time constraints,
|
||||
classified as instructor-led-only for the first pilot (an instructor supplies the missing
|
||||
framing live); fully self-paced delivery of exactly these three is deferred. See
|
||||
`docs/ROOT_PILOT_CURRICULUM.md`.
|
||||
|
||||
## Content authored this phase
|
||||
|
||||
20 new Markdown content files (demonstrations, guided exercises, knowledge checks, lessons,
|
||||
reflections) across ROOT 05, 06, 07, 09, 10, and 11 — each following the established
|
||||
`## Instructor Notes` convention (instructor-only content below that heading, stripped from
|
||||
student-visible payload by `content.js`'s existing `splitContent()`). Every demonstration
|
||||
follows OBSERVE → HYPOTHESIS → INSPECT → CHANGE → VERIFY; every guided exercise bridges lesson
|
||||
to lab without revealing the lab's exact answer; every knowledge check tests reasoning over
|
||||
recall; every reflection closes with a named, transferable engineering principle.
|
||||
|
||||
## Assessment bank (Section 9)
|
||||
|
||||
New `assessment_definitions` row `rootnode-security-assessment` (type `exam`, 60 minutes,
|
||||
`hint_policy: disabled`), using the existing Phase 3B assessment engine unmodified. Pool: 3
|
||||
required hands-on CORE security scenarios (`cks-pod-security-context`, `cks-rbac-least-privilege`,
|
||||
`cks-network-policy`) plus 4 random-draw groups covering SecurityContext hardening depth,
|
||||
RBAC/ServiceAccount depth, network/supply-chain depth, and the 7 previously-unwired CKS MCQs as
|
||||
one pool — 9 scenarios drawn per attempt, server-authoritative (verified live: a real student
|
||||
session draw returned 9 scenario IDs matching the pool spec, no client input accepted). Wired
|
||||
into `root-09-m04-cluster-hardening-and-runtime` as an `assessment`-type activity — the natural
|
||||
home for a security checkpoint, not a new module.
|
||||
|
||||
## Remaining unwired scenarios (Section 13)
|
||||
|
||||
Recomputed after the security-assessment wiring consumed the 7 previously-unwired CKS MCQs:
|
||||
**2 scenarios remain unwired** — `cluster-upgrade-mcq` and `etcd-backup-mcq`, both Cluster
|
||||
Administration/CKA-track content (kubeadm upgrade order, etcd backup/restore). No current ROOT
|
||||
module teaches cluster administration at that level; classified **future-curriculum**, not
|
||||
neglect. One scenario that WAS unwired, `readiness-probe-task` (a constructive "author a
|
||||
readiness probe" Lab, distinct from the existing diagnostic `rootnode-probe-port-mismatch`
|
||||
Lab), was wired as an **optional practice** activity in `root-10-m01-health-probes-and-troubleshooting`
|
||||
using the existing `is_required` model from Phase 6B — no second practice system built.
|
||||
|
||||
## ROOT 11 (Section 11-12)
|
||||
|
||||
`docs/ROOT11_DEVOPS_PLATFORM_SCOPE.md` evaluates all 7 proposed modules individually. Only one
|
||||
was authored (`root-11-m01-devops-platform-thinking`, folding in the closely-related "Platform
|
||||
Engineering & DX" idea) — a lesson, knowledge check, and reflection, explicitly telling
|
||||
students which future topics (GitOps, CI/CD, IaC, observability) require infrastructure this
|
||||
platform doesn't yet provide. No fake labs were built. ROOT 11 moved from `planned` to
|
||||
`partial` at the course level — an intentional, documented scope, not an unexplained
|
||||
placeholder.
|
||||
|
||||
## New application code
|
||||
|
||||
None. Every genuine gap this phase was satisfiable with existing tables (`course_modules`,
|
||||
`module_activities`, `assessment_definitions`) and the existing content-delivery pipeline. One
|
||||
real bug was found and fixed during authoring (see below) but required no schema or API change.
|
||||
|
||||
## Bug found and fixed
|
||||
|
||||
An early draft of migration 024 set `root-11-m01-devops-platform-thinking`'s
|
||||
`course_modules.prerequisites` to a COURSE id (`root-10-kubernetes-troubleshooting-observability`)
|
||||
instead of a MODULE id — `course_modules.prerequisites` and `courses.prerequisites` are
|
||||
different namespaces. Caught by actually running `infra/scripts/validate-course-content.js`
|
||||
against the rebuilt database (`FAIL — module root-11-m01-devops-platform-thinking lists a
|
||||
prerequisite that doesn't exist`). Verified via direct query that the course-level gate was
|
||||
already correctly set from Phase 6A's migration 022. Fixed by emptying the module-level field
|
||||
(the course-level gate already fully enforces the dependency) and documenting why in the
|
||||
migration's SQL comments.
|
||||
|
||||
## Content validation (Section 24)
|
||||
|
||||
`infra/scripts/validate-course-content.js` (new, permanent) checks: no duplicate course/module/
|
||||
activity ids; no orphan modules/activities; every scenario_id and assessment_definition_id
|
||||
resolves to a real row/file; every prerequisite id resolves and contains no cycles; no sequence
|
||||
collisions; instructor notes never leak into student-visible content (verified by actually
|
||||
invoking `content.js`'s real `loadContent()`); every content_path resolves to a real file. Final
|
||||
run: **22 courses/modules loaded, 163 activities, 98 scenario files, 0 errors.**
|
||||
|
||||
## Pilot readiness determination
|
||||
|
||||
See `docs/ROOT_PILOT_CURRICULUM.md` for the full manifest. Every module in the pilot's required
|
||||
completion path (ROOT 05-07, 09-10, 12, Capstone) meets the READY bar for instructor-led
|
||||
delivery; three specific practice-bank modules are instructor-led-only this pilot; ROOT 08 and
|
||||
ROOT 11 modules 2-7 are explicitly outside the required path. The first pilot is scoped as
|
||||
instructor-led, and within that scope, the curriculum is ready.
|
||||
@@ -67,5 +67,40 @@ can prove in advance.
|
||||
- [ ] `docs/RUNBOOK_STUDENT_ACCOUNT_LIFECYCLE.md` and `docs/RUNBOOK_REMOTE_CLASSROOM.md`'s steps
|
||||
still make sense walked against the real admin portal
|
||||
|
||||
## Self-paced enrollment & website integration (Phase 5B — see docs/PHASE5B_SELF_PACED_ENROLLMENT.md)
|
||||
- [ ] `INTEGRATION_SHARED_SECRET` is set to a real, unique-to-production secret (not the dev
|
||||
value) and is stored only in the Platform API's own server-side environment — never
|
||||
committed, never in any browser-reachable config
|
||||
- [ ] The website team's production integration key was delivered out of band (not plaintext
|
||||
email/chat) and is confirmed working with one real, low-stakes test enrollment before go-live
|
||||
- [ ] A deliberate `POST /api/integrations/enrollments` call missing/with a wrong key returns a
|
||||
clean `401` against the real production endpoint, not a 500 or an information leak
|
||||
- [ ] `GET /metrics`'s `rootnode_enrollments_total`/`rootnode_program_completions_total` labels
|
||||
contain only status/delivery_mode/program_id values — confirm no student_id, email, or
|
||||
external_reference ever appears in a real production scrape
|
||||
- [ ] `docs/RUNBOOK_SELF_PACED_ENROLLMENT.md`'s steps still make sense walked against the real
|
||||
admin portal, and `docs/WEBSITE_ENROLLMENT_INTEGRATION.md` is what the website team
|
||||
actually integrated against (update it if reality drifted during that integration)
|
||||
|
||||
## Enrollment communications / Google Workspace email (Phase 5C — see docs/PHASE5C_ENROLLMENT_COMMUNICATIONS.md)
|
||||
- [ ] Every step in `docs/RUNBOOK_GOOGLE_WORKSPACE_EMAIL.md` marked REQUIRES GOOGLE WORKSPACE
|
||||
ADMIN ACTION has actually been completed (sending mailbox, service account, domain-wide
|
||||
delegation scoped to `gmail.send` only, SPF/DKIM/DMARC)
|
||||
- [ ] `EMAIL_PROVIDER=gws` set in the real production environment — confirm it is NOT left at
|
||||
the dev-provider default, which would silently mean no real student ever receives email
|
||||
- [ ] `GOOGLE_SERVICE_ACCOUNT_JSON`, `EMAIL_FROM_ADDRESS`, `EMAIL_FROM_NAME`, `EMAIL_REPLY_TO`,
|
||||
and `PUBLIC_APP_ORIGIN` are all set from the real production secret store, not left at any
|
||||
dev/example value
|
||||
- [ ] `infra/scripts/gws-email-test.js` (opt-in, `RUN_GWS_EMAIL_TEST=true`) run once against a
|
||||
real `TEST_EMAIL_RECIPIENT` and confirmed delivered — the one and only "does this actually
|
||||
work" proof this checklist can't fake
|
||||
- [ ] `COOKIE_SECURE=1` in the real production environment — confirms the dev-preview
|
||||
notification route (`GET /api/admin/notifications/:id/dev-preview`) is unreachable and no
|
||||
activation/reset token is ever returned by any API response
|
||||
- [ ] `GET /metrics`'s notification gauges/counters contain no student_id/email/token/
|
||||
notification_id/external_event_id in any label value on a real production scrape
|
||||
- [ ] `docs/RUNBOOK_STUDENT_ACTIVATION.md`'s steps still make sense walked against the real
|
||||
admin portal, including a real (not dev-provider) activation email actually arriving
|
||||
|
||||
Only once every box above is checked against the real server does the deployment graduate from
|
||||
"infrastructure package built" to "actually in production."
|
||||
|
||||
@@ -45,11 +45,12 @@ this flow specifically — see the Phase 4.1 report for the full run results.
|
||||
|
||||
## What's genuinely missing for a real self-paced product (Phase 5, not built here)
|
||||
|
||||
1. **No self-registration / public enrollment.** A student row is created only by an instructor
|
||||
(`POST /api/admin/students`, admin/owner RBAC-gated). There is no public "sign up" surface.
|
||||
This is almost certainly the right call to leave alone for now — it's explicitly a Phase 5+
|
||||
product/business decision (course commerce, is it free, who's allowed to enroll), not an
|
||||
infrastructure gap.
|
||||
1. **No self-registration / public enrollment surface.** Still true, still deliberate — Phase 5B
|
||||
built the boundary a *separate* public website will call
|
||||
(`POST /api/integrations/enrollments`, see `docs/PHASE5B_SELF_PACED_ENROLLMENT.md` §11 and
|
||||
`docs/WEBSITE_ENROLLMENT_INTEGRATION.md`) rather than a public sign-up form here. A student
|
||||
row can now be created automatically by a trusted server-to-server enrollment event, but there
|
||||
is still no browser-facing "sign up" page anywhere in this codebase — that's the website's job.
|
||||
|
||||
2. ~~**No real student authentication.**~~ **Closed in Phase 5A.** Students now have real
|
||||
credentials (email+password or login-identifier+password, `password_hash`/`scrypt` reusing
|
||||
@@ -59,25 +60,48 @@ this flow specifically — see the Phase 4.1 report for the full run results.
|
||||
/api/login` still works for not-yet-converted pilot rows only; it's rejected (409) the moment
|
||||
an account has real credentials or a pending activation token.
|
||||
|
||||
3. **`curriculum_assignments.student_id` is schema-ready but unwired.** The table has supported
|
||||
a direct per-student curriculum assignment (independent of any cohort) since Phase 2B
|
||||
(`CONSTRAINT curriculum_assignment_target CHECK (cohort_id IS NOT NULL OR student_id IS NOT
|
||||
NULL)`) — but nothing in the application ever reads or writes that column today; there's only
|
||||
one curriculum path, so nothing has needed to differentiate yet. Not a gap that blocks
|
||||
self-paced delivery now (every student already sees the one path with no gate), but the
|
||||
moment a second curriculum path exists, this is where per-individual assignment lands — the
|
||||
schema was designed for exactly this and doesn't need a migration to support it.
|
||||
3. ~~**`curriculum_assignments.student_id` is schema-ready but unwired.**~~ **Addressed
|
||||
differently in Phase 5B.** `curriculum_assignments` itself remains unwired, unchanged — but
|
||||
the real problem it was meant to eventually solve (which curriculum a given student sees) is
|
||||
now solved at the enrollment layer instead: `enrollments.curriculum_path` snapshots
|
||||
`programs.curriculum_path` per enrollment. `curriculum_assignments` is still available for a
|
||||
future direct per-student assignment need outside the program/enrollment model — it wasn't
|
||||
removed, just not the mechanism Phase 5B ended up using. See
|
||||
`docs/PHASE5B_SELF_PACED_ENROLLMENT.md` §4.
|
||||
|
||||
4. **No delivery-mode concept anywhere.** In-person, remote, and self-paced sessions aren't
|
||||
distinguished in any data model today, and nothing currently needs them to be — capacity
|
||||
planning, the reaper, and academic records all behave identically regardless of how a student
|
||||
is attending. Worth being aware this is unmodeled, not worth building a taxonomy for before a
|
||||
real product reason exists (YAGNI).
|
||||
4. ~~**No delivery-mode concept anywhere.**~~ **Closed in Phase 5B.**
|
||||
`programs.delivery_mode`/`enrollments.delivery_mode` (`self_paced`/`cohort`/`hybrid`) is now
|
||||
a real, snapshotted field. Capacity planning, the reaper, and Lab lifecycle mechanics still
|
||||
behave identically regardless of delivery mode — this only affects how enrollment/entitlement
|
||||
is modeled, not infrastructure.
|
||||
|
||||
5. **No self-paced enrollment/entitlement model.** ~~Closed in Phase 5B~~ — see
|
||||
`docs/PHASE5B_SELF_PACED_ENROLLMENT.md` for `programs`/`enrollments`, server-enforced
|
||||
entitlement (pending/future/expired/cancelled), server-derived completion, and the resulting
|
||||
self-paced student dashboard.
|
||||
|
||||
6. ~~**No enrollment communication.**~~ **Closed in Phase 5C.** An enrolled student now actually
|
||||
gets told — real activation/enrollment-confirmation/password-reset email, delivered
|
||||
asynchronously through a provider-neutral notification worker (Google Workspace via the
|
||||
Gmail API in production, a deterministic dev provider everywhere else). See
|
||||
`docs/PHASE5C_ENROLLMENT_COMMUNICATIONS.md`.
|
||||
|
||||
7. ~~**No notion of a lesson, only a flat scenario list per curriculum stage.**~~ **Closed in
|
||||
Phase 6A**, for `root_` only. A self-paced enrollment now resolves to a real course/module/
|
||||
activity tree (lessons, demonstrations, knowledge checks, guided exercises, labs,
|
||||
reflections, assessments/capstone) instead of just a bare scenario id list — the same tree an
|
||||
instructor-led classroom student sees, consumed through the identical `buildCourseTree()`.
|
||||
See `docs/COURSE_CONTENT_MODEL.md` and `docs/PHASE6A_ROOT_COURSE_FOUNDATION.md`. **Populated
|
||||
with real teaching content across most of the Kubernetes/operations courses in Phase 6B** —
|
||||
see `docs/PHASE6B_ROOT_CURRICULUM_EXPANSION.md`. The architecture itself (this closed gap)
|
||||
didn't change; what a self-paced student actually finds inside it did.
|
||||
|
||||
## What Phase 4.1 deliberately did NOT do here
|
||||
|
||||
No signup flow, no student password/email auth implementation, no per-student curriculum
|
||||
assignment route, no payments, no public course catalog, no held/premium scenario content. All
|
||||
explicit non-goals per this phase's authorization. Student authentication (item 2, above) was
|
||||
closed in Phase 5A; the rest remain open, unmodeled by design (YAGNI) until a real product
|
||||
reason exists.
|
||||
explicit non-goals per this phase's authorization. Student authentication (Phase 5A), the
|
||||
self-paced enrollment/entitlement model (Phase 5B), and enrollment/account-lifecycle
|
||||
communication (Phase 5C) have all since been built; a public self-registration *page*, real
|
||||
payment processing, and a Parent/Guardian portal remain deliberately out of scope — see
|
||||
`docs/PHASE5B_SELF_PACED_ENROLLMENT.md` §12 and `docs/PHASE5C_ENROLLMENT_COMMUNICATIONS.md`.
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
# ROOT 11 — DevOps & Platform Engineering: Scope Decision
|
||||
|
||||
Phase 6C's mandate for ROOT 11 was to stop it being an undefined `planned` placeholder by
|
||||
making an explicit, evidence-based scope decision — not to fake seven hands-on labs the current
|
||||
platform cannot honestly deliver. This document is that decision.
|
||||
|
||||
## Purpose and learning outcomes
|
||||
|
||||
ROOT 11 exists to give a student who has just finished ROOT 05-10 (Kubernetes hands-on) and
|
||||
ROOT 12 (incident response) the vocabulary and mental model to see their skills as part of a
|
||||
larger DevOps/platform engineering practice, and an honest map of what comes next once the
|
||||
platform can support it hands-on. It is explicitly NOT meant to teach GitOps, CI/CD, or
|
||||
Infrastructure as Code as hands-on skills in this phase.
|
||||
|
||||
## The seven proposed modules, evaluated against current architecture
|
||||
|
||||
| # | Proposed module | Classification | Why |
|
||||
|---|---|---|---|
|
||||
| 1 | Sysadmin → Platform Thinking | **READY TO AUTHOR NOW** | Pure orientation/reframing content. No new infrastructure needed — it's built entirely on ROOT_'s existing curriculum, referenced explicitly. Authored this phase as `root-11-m01-devops-platform-thinking`. |
|
||||
| 2 | GitOps & Declarative Delivery | **REQUIRES FUTURE LAB ENVIRONMENT** | Needs a Git repository the student can push to and a deployment-automation controller (ArgoCD or Flux) watching it. The Lab platform provisions an isolated k3s VM per student today; it does not provision or wire a Git remote or a GitOps controller. Faking this with a Kubernetes-only scenario would teach "apply a manifest," not GitOps — a materially wrong mental model. |
|
||||
| 3 | CI/CD Fundamentals | **REQUIRES FUTURE LAB ENVIRONMENT** | Needs a CI runner (even a minimal one) that can check out code, run a pipeline, and produce an artifact or trigger a deploy. No such runner exists in the Lab image or Platform API today. |
|
||||
| 4 | Infrastructure as Code | **REQUIRES FUTURE LAB ENVIRONMENT** | Needs a provisioning target and a tool like Terraform/OpenTofu applying against it. Kubernetes objects are already declarative, but IaC's actual lesson is about provisioning infrastructure *below* Kubernetes — cloud resources, networking, the cluster itself — which this platform doesn't expose to a student sandbox. |
|
||||
| 5 | Observability & SLO Thinking | **PARTIAL-LESSON-ONLY** | The conceptual half (what SLIs/SLOs are, error budgets, why "is the Pod Running" isn't observability) can be taught honestly without a metrics stack. A genuinely hands-on version needs Prometheus/Grafana (or equivalent) wired into the Lab VM, which doesn't exist today. Recommend authoring the lesson-only half in a future sub-phase; do not build the metrics stack this phase. |
|
||||
| 6 | Platform Engineering & DX | **READY TO AUTHOR NOW** (folded into module 1) | Conceptually adjacent enough to module 1 that a separate module would duplicate content. Its core ideas (self-service, golden paths, reducing cognitive load) are already covered in `root-11-m01`. |
|
||||
| 7 | Reliability / Automation / Operational Ownership | **PARTIAL-LESSON-ONLY** | Largely already covered hands-on by ROOT 12 (Real Ops incidents) and ROOT 10 (troubleshooting framework). A short lesson connecting those two courses' skills explicitly to "reliability engineering" as a named discipline is legitimate future content; a new hands-on lab would duplicate ROOT 12 rather than teach something new. |
|
||||
|
||||
## What was actually authored this phase
|
||||
|
||||
Only module 1 (folded with module 6's ideas) — `root-11-m01-devops-platform-thinking` — a
|
||||
lesson, knowledge check, and reflection. No Lab activity exists for ROOT 11 and none was
|
||||
faked. The lesson explicitly tells students which future topics (GitOps, CI/CD, IaC,
|
||||
observability) require infrastructure this course doesn't yet provide, rather than staying
|
||||
silent about the gap.
|
||||
|
||||
## What current infrastructure supports vs. what future modules need
|
||||
|
||||
**Supports today:** Kubernetes/k3s hands-on labs via the existing Lab VM provisioning
|
||||
(Platform API → Provisioner → Lab Router), the existing assessment engine, and the existing
|
||||
content/course model — sufficient for orientation and conceptual lesson content referencing
|
||||
Kubernetes skills already taught.
|
||||
|
||||
**Would need building for modules 2-4, and the hands-on half of 5 and 7 (future phases,
|
||||
NOT this phase):**
|
||||
- A Git hosting target reachable from a student's Lab VM, plus a GitOps controller
|
||||
(ArgoCD/Flux) pre-installed or installable in that VM.
|
||||
- A minimal CI runner (self-hosted GitHub Actions runner, Drone, or similar) wired to a
|
||||
student-scoped repository.
|
||||
- A provisioning target IaC tooling can act against — even a scoped-down one (e.g., Terraform
|
||||
against a mock/local provider) — plus the tooling itself in the Lab image.
|
||||
- A metrics/logging stack (Prometheus + Grafana at minimum) installed per-Lab-VM or shared with
|
||||
per-student scoping, for the hands-on half of Observability.
|
||||
|
||||
None of this was built in Phase 6C, per its explicit "do NOT build" list (Section 26).
|
||||
|
||||
## Pilot inclusion recommendation
|
||||
|
||||
Include `root-11-m01-devops-platform-thinking` in the first pilot as a **closing orientation
|
||||
module** — no hands-on requirement, explicitly scoped as conceptual. Do not present ROOT 11 as
|
||||
a complete course in pilot marketing or instructor materials; present it as "orientation now,
|
||||
hands-on modules 2-7 in a future phase once the required lab environments exist." This keeps
|
||||
the pilot honest about what ROOT 11 currently is versus what it will become.
|
||||
|
||||
See [`docs/ROOT_PILOT_CURRICULUM.md`](ROOT_PILOT_CURRICULUM.md) for how this fits the overall
|
||||
pilot manifest.
|
||||
@@ -0,0 +1,113 @@
|
||||
# ROOT Capstone Readiness
|
||||
|
||||
Phase 6B §9/§10 — the instructional runway that prepares a student for the existing
|
||||
`rootnode-capstone` assessment, and the server-side fix that makes its unlock rule actually
|
||||
enforced. This document does not redesign the capstone engine itself (`assessments.js`,
|
||||
`013_assessments.sql`) — see `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` for that.
|
||||
|
||||
## 1. The Capstone Readiness module
|
||||
|
||||
`root-capstone-m02-capstone-readiness` (sequenced BEFORE the real capstone module,
|
||||
`root-capstone-m01-operate-and-recover`, within the `root-capstone` course) — 4 activities:
|
||||
|
||||
1. **Lesson — What the Capstone Measures & How the Draw Works.** Explains the required-vs-random
|
||||
scenario draw, what "unlocked" means, and what completion records. Never publishes the exact
|
||||
scenario IDs in the current pool or any answer.
|
||||
2. **Lesson — Time Management & Troubleshooting Strategy Under a Clock.** Budgeting the 90
|
||||
minutes, reusing the ROOT 10/ROOT 12 frameworks under pressure, and when to move on from a
|
||||
stuck scenario.
|
||||
3. **Knowledge check — Readiness Check: Are You Actually Ready?** A genuine self-assessment
|
||||
(not a graded quiz) against concrete criteria, pointing back to specific ROOT_ modules for
|
||||
any "no" answer.
|
||||
4. **Reflection — What Completion Means.** Connects passing the Capstone to a durable,
|
||||
demonstrated skill rather than a completion checkbox.
|
||||
|
||||
This module derives its own completion the same way every other lesson-only module does
|
||||
(`activity_completions`, `manual_marker`) — it is intentionally NOT a prerequisite the real
|
||||
capstone assessment engine checks before allowing a start (see §3 below for why these two gates
|
||||
are deliberately independent).
|
||||
|
||||
## 2. What was NOT built
|
||||
|
||||
No answers, no exact scenario IDs from the current pool, no capstone-specific hint content
|
||||
beyond what the Lab's own progressive hints already provide. No new assessment engine, no
|
||||
second capstone, no readiness "score." This is orientation content, not a shadow exam.
|
||||
|
||||
## 3. Why readiness-module completion and the real unlock gate are independent
|
||||
|
||||
The real capstone start (`POST /api/assessments/rootnode-capstone/start`) is gated by the
|
||||
curriculum path's own `unlock_rule` — all 6 content stages of `rootnode-path.json` complete
|
||||
(Foundation, Workloads, Networking, Configuration, Troubleshooting, Real Ops) — computed by
|
||||
`computeCurriculumState()` (`curriculum-rules.js`), the exact same function `GET /api/capstone`
|
||||
already used to display lock state. Whether a student has viewed or completed the Capstone
|
||||
Readiness module has NO bearing on this gate.
|
||||
|
||||
This is deliberate, not an oversight: hard-locking the real assessment behind a CONTENT-MODEL
|
||||
module (course_modules/module_activities) would mean the security-critical enforcement boundary
|
||||
lives in two different systems that could drift out of sync — the curriculum-path unlock_rule
|
||||
(Postgres-backed, achievement-derived, already proven) and the course-content prerequisite chain
|
||||
(newer, additive, built for teaching sequencing rather than security enforcement). Keeping them
|
||||
independent means: the readiness module is free to be completed early, late, or skipped by a
|
||||
student who's already prepared some other way, while the real gate remains exactly as strict as
|
||||
`docs/PHASE3B_LEARNING_AND_ASSESSMENT.md` originally specified.
|
||||
|
||||
## 4. The server-side unlock enforcement fix (Phase 6B §10)
|
||||
|
||||
**The gap** (documented in `docs/PHASE6A_ROOT_COURSE_FOUNDATION.md` §10): `POST /api/assessments/
|
||||
:definitionId/start` (`learning-routes.js`) checked only that the assessment definition existed
|
||||
and was `status = 'active'` — it never checked `unlock_rule` at all. `GET /api/capstone`
|
||||
correctly computed and displayed lock state, and the student UI never exposed a "start" action
|
||||
while locked, but a direct POST to the start route bypassed the check entirely. Student UI state
|
||||
was the only thing standing between a student and an early capstone attempt.
|
||||
|
||||
**The fix**: the start route now resolves whether the requested definition is the curriculum
|
||||
path's OWN configured capstone (`path.stages.find(s => s.type === 'capstone' && s
|
||||
.capstone_assessment_id === definition.assessment_definition_id)`), and if so, computes the
|
||||
exact same `computeCurriculumState()` gate `/api/capstone` already relies on. A `locked` gate
|
||||
now returns a clean `403 { error: 'locked', message, blocking_stage_ids }` — the same
|
||||
`blocking_stage_ids` shape `/api/capstone` already exposes, so a client integrating against this
|
||||
route gets consistent, already-documented error shape rather than a new one. No internal state
|
||||
beyond the stage ids that are actually incomplete is ever returned.
|
||||
|
||||
**What is NOT gated**: any assessment definition the curriculum path does NOT name as its
|
||||
capstone (e.g. `rootnode-quick-assessment`) is completely unaffected — the check only ever
|
||||
triggers for the one definition the path itself points at via `capstone_assessment_id`.
|
||||
|
||||
**What remains valid**: no existing `assessment_sessions` row, no achievement, no enrollment
|
||||
completion record was touched by this fix — it only affects a NEW start attempt going forward.
|
||||
A student who already legitimately completed the capstone before this fix keeps that record
|
||||
exactly as it was.
|
||||
|
||||
**Regression coverage**: `platform/test/e2e-phase6b.js` proves both directions — a direct start
|
||||
attempt before prerequisites are met is rejected (`403 locked`), and a legitimately unlocked
|
||||
attempt (all 6 stages genuinely complete) still succeeds and completes normally. A pre-existing
|
||||
gap was also found and fixed in `e2e-phase5b.js`'s own capstone-completion test, which had never
|
||||
included the `real-ops` stage in its completion loop — it was passing only because the very
|
||||
bug this phase fixes let an incomplete curriculum still start the capstone. See that test file's
|
||||
own comment for detail.
|
||||
|
||||
## 5. Phase 6C re-review against the expanded curriculum
|
||||
|
||||
Re-checked every item this doc depends on against Phase 6C's changes and found no gap requiring
|
||||
a fix:
|
||||
|
||||
- **The unlock gate's 6 stages are unchanged**: Foundation, Workloads, Networking,
|
||||
Configuration, Troubleshooting, Real Ops (`rootnode-path.json`). Phase 6C added and upgraded
|
||||
modules within those stages' courses (ROOT 05-07, 10) plus ROOT 09 (Security) and the new
|
||||
ROOT 11 orientation — **ROOT 08 (Storage & Scheduling) and ROOT 09 (Security) do not appear
|
||||
among the 6 gating stages and still do not gate the capstone.** This is documented
|
||||
architecture, not an oversight — confirmed by direct test
|
||||
(`platform/test/e2e-phase6c.js`, "the capstone unlock gate is unaffected by ROOT 09 (security)
|
||||
being incomplete") rather than assumed.
|
||||
- **`root-capstone-m01-operate-and-recover`** was promoted from `partial` to `ready` this phase
|
||||
as a documented exception (no activity added — its single `capstone` activity IS the exam;
|
||||
see `docs/PHASE6C_CURRICULUM_COMPLETION.md`). This changes only its content-model status
|
||||
label, not the real unlock gate described in §3-4 above, which remains entirely independent.
|
||||
- **Random draw pools, timing, hint policy, and completion rule** for `rootnode-capstone` itself
|
||||
were not touched by Phase 6C — no scenario content this phase added or changed feeds that
|
||||
pool.
|
||||
- **The Capstone Readiness module (§1)** required no changes — its 4 activities and their
|
||||
relationship to the (still unchanged) real unlock gate remain accurate as described above.
|
||||
|
||||
No redesign was made or needed; Phase 6C's curriculum expansion did not require reopening the
|
||||
capstone engine.
|
||||
@@ -0,0 +1,157 @@
|
||||
# ROOT_ Curriculum Coverage Matrix
|
||||
|
||||
Phase 6B §1 — maps every scenario Phase 6B puts to structured use, with an intentional reason
|
||||
for each placement (not coverage-percentage padding). Built from
|
||||
`docs/ROOT_CURRICULUM_INVENTORY.md`'s evidence. Statuses: **READY** (meets
|
||||
`docs/ROOT_PILOT_READINESS_STANDARD.md`'s bar), **PARTIAL** (real lesson + labs, missing one or
|
||||
more of demonstration/guided-exercise/reflection), **PLANNED** (structure only, no content),
|
||||
**GAP** (no suitable scenario exists at all).
|
||||
|
||||
## Phase 6C verification (superseding Phase 6B's own self-reported counts)
|
||||
|
||||
Baseline (start of Phase 6C), run directly against the database and filesystem
|
||||
(`infra/scripts/validate-course-content.js`, 0 errors) rather than trusted from Phase 6B's
|
||||
narrative report, which had miscounted: **21 modules total — 6 READY, 15 PARTIAL** (Phase 6B's
|
||||
final report claimed 17 PARTIAL; the real count, re-verified here, was 15).
|
||||
|
||||
**After Phase 6C's upgrades: 22 modules total — 16 READY, 6 PARTIAL.** (22, not 21: Phase 6C
|
||||
added `root-11-m01-devops-platform-thinking`, ROOT 11's first real module — see
|
||||
`docs/ROOT11_DEVOPS_PLATFORM_SCOPE.md`.) 75 activities now carry content_path (up from 73),
|
||||
163 activities total (up from 140), and unwired scenarios dropped from 10 (Phase 6B) to 2
|
||||
(`cluster-upgrade-mcq`, `etcd-backup-mcq` — both Cluster Administration/CKA-track content with
|
||||
no current ROOT module to host them; see `docs/ROOT_PILOT_CURRICULUM.md`'s "not in pilot"
|
||||
section). Full before/after detail and rationale per module is in
|
||||
`docs/PHASE6C_CURRICULUM_COMPLETION.md`.
|
||||
|
||||
**READY (16):** `root-05-m00-kubernetes-mental-model` (Phase 6A gold standard, unchanged),
|
||||
`root-05-m01-pods-and-kubectl-basics` (Phase 6C: added bookending lesson + reflection around
|
||||
its 20-task kubectl-fluency bank), `root-06-m02-deployments-scaling-updates` (unchanged),
|
||||
`root-06-m03-jobs-cronjobs-daemonsets` (Phase 6C: added demonstration, guided exercise,
|
||||
reflection — full gold pattern), `root-07-m02-configuration-and-secrets` (Phase 6C: added
|
||||
lesson, demonstration, knowledge check, guided exercise, reflection — was a bare 5-lab bank),
|
||||
`root-07-m03-services-and-service-discovery` (unchanged), `root-07-m04-ingress-networkpolicy-foundations`
|
||||
(Phase 6C: full gold pattern added), `root-09-m01-security-model-and-securitycontext`
|
||||
(unchanged), `root-09-m02-serviceaccounts-and-rbac` (Phase 6C: full gold pattern added),
|
||||
`root-09-m03-network-and-supplychain-security` (Phase 6C: full gold pattern added),
|
||||
`root-09-m04-cluster-hardening-and-runtime` (Phase 6C: documented exception — survey depth by
|
||||
design, now anchored by a real security assessment), `root-10-m01-health-probes-and-troubleshooting`
|
||||
(Phase 6A gold standard; Phase 6C added one optional practice lab), `root-10-m03-troubleshooting-framework`
|
||||
(Phase 6C: added guided exercise + reflection), `root-12-m01-real-ops-incidents` (Phase 6C:
|
||||
documented exception — deliberately unguided by design, status promoted with no new activities),
|
||||
`root-capstone-m01-operate-and-recover` (Phase 6C: documented exception — the capstone exam IS
|
||||
the module), `root-capstone-m02-capstone-readiness` (unchanged).
|
||||
|
||||
**PARTIAL (6, all with a documented pilot disposition in `docs/ROOT_PILOT_CURRICULUM.md`):**
|
||||
`root-06-m01-deployments-and-workloads`, `root-07-m01-services-and-networking`,
|
||||
`root-10-m02-general-troubleshooting` (all three: dense practice banks, sibling module in the
|
||||
same course already carries the taught concept — instructor-led only for the first pilot,
|
||||
self-paced deferred), `root-08-m01-storage-fundamentals`, `root-08-m02-scheduling-resource-governance`
|
||||
(ROOT 08 does not gate the capstone; deferred as optional supplementary content this phase),
|
||||
`root-11-m01-devops-platform-thinking` (honest, lesson-only by design — no hands-on Lab exists
|
||||
for ROOT 11 yet).
|
||||
|
||||
## Duplicate concepts identified (handled explicitly, not silently mapped twice)
|
||||
|
||||
- `pod-security-context` (generic) vs. `cks-pod-security-context` (CKS-flavored) — same concept.
|
||||
`cks-pod-security-context` is the CORE guided lab (ROOT 09 m01); `pod-security-context` is
|
||||
classified INDEPENDENT PRACTICE (a second, optional rep).
|
||||
- `rbac-clusterrole` (generic) vs. `cks-rbac-clusterrole` (CKS-flavored) — same concept.
|
||||
`cks-rbac-clusterrole` is CORE (ROOT 09 m02); `rbac-clusterrole` is INDEPENDENT PRACTICE.
|
||||
- `network-policy` (generic, foundational) vs. `cks-network-policy` (CKS default-deny framing) vs.
|
||||
`networkpolicy-egress` (generic egress) — three related but distinct angles, deliberately split
|
||||
across two courses: `network-policy` teaches the ARCHITECTURAL concept in ROOT 07 (what a
|
||||
NetworkPolicy is/does); `cks-network-policy`/`cks-egress-namespace`/`networkpolicy-egress`
|
||||
teach it as a SECURITY CONTROL in ROOT 09 (default-deny posture, least-privilege network
|
||||
access) — not a duplicate once the teaching angle is made explicit, so both are CORE in their
|
||||
respective course, not one marked redundant.
|
||||
|
||||
## ROOT 05 — Kubernetes Foundations
|
||||
|
||||
| Module | Objective | Lesson | Demo | KC | Guided | Scenario(s) | Assessment | Real Ops | Status |
|
||||
|---|---|---|---|---|---|---|---|---|---|
|
||||
| m02 Kubernetes Mental Model & Core Objects (NEW) | Explain what Kubernetes solves, the control-plane/worker/desired-state model, and navigate Pods/Namespaces/labels with kubectl | Yes | Yes | Yes | Yes | pod-basics-mcq, namespaces-basics, labels-selectors-mcq, kubectl-essentials-mcq (existing, re-sequenced as this module's guided path) | — | "declarative config drifts back to desired state after manual edits" | **READY** |
|
||||
| m01 Pods & kubectl Basics (existing) | Mechanical kubectl fluency across the Foundation stage's 20 scenarios | No | No | No | No | all 20 Foundation scenarios | — | — | PARTIAL (unchanged from Phase 6A — m02 above is the new front door; m01's labs remain a practice bank behind it) |
|
||||
|
||||
`m02` is sequenced BEFORE `m01` (prerequisite: none; `m01`'s prerequisite becomes `m02`) so a
|
||||
student meets the mental model before the bare-lab bank, per this phase's core principle.
|
||||
|
||||
## ROOT 06 — Kubernetes Application Workloads
|
||||
|
||||
| Module | Objective | Lesson | Demo | KC | Guided | Scenario(s) | Real Ops | Status |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| m02 Deployments, Scaling & Updates (NEW) | Explain why Deployments/ReplicaSets exist, perform a rolling update, and reason about rollback | Yes | Yes | Yes | Yes | deploy-nginx, scale-deployment, rolling-update-task, deployment-rollback (newly wired — Workloads & Scheduling bucket, belongs here not ROOT 08) | "a bad rollout with no rollback plan turns a deploy into an outage" | **READY** |
|
||||
| m03 Jobs, CronJobs & DaemonSets (NEW) | Distinguish one-shot Jobs, scheduled CronJobs, and per-node DaemonSets from long-running Deployments | Yes | No | Yes | No | job-task, cronjob-task (newly wired), jobs-cronjobs-mcq, daemonset-mcq | "a cleanup Job that never completes silently exhausts cluster resources" | PARTIAL (lesson+KC authored; no separate demo/guided — labs carry the practice) |
|
||||
| m01 Deployments & Workloads (existing) | Bare-lab bank, Workloads stage's original 8 scenarios | No | No | No | No | (unchanged) | — | PARTIAL (unchanged) |
|
||||
|
||||
## ROOT 07 — Kubernetes Networking & Configuration
|
||||
|
||||
| Module | Objective | Lesson | Demo | KC | Guided | Scenario(s) | Real Ops | Status |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| m03 Services & Service Discovery (NEW) | Explain the Service/Endpoints/DNS request path and choose the right Service type | Yes | Yes | Yes | Yes | services-mcq, expose-service-basics, nodeport-task, dns-resolution-mcq (newly wired) | "a Service with zero endpoints looks identical to network failure from the outside" | **READY** |
|
||||
| m04 Ingress & NetworkPolicy Foundations (NEW) | Explain how Ingress routes external traffic and what a NetworkPolicy architecturally does | Yes | No | Yes | No | ingress-task, network-policy (newly wired) | "traffic reachability should be intentional, not incidental" | PARTIAL |
|
||||
| m02 Configuration & Secrets (existing) | Bare-lab bank | No | No | No | No | (unchanged) | — | PARTIAL (unchanged) |
|
||||
| m01 Services & Networking (existing) | Bare-lab bank, folded conceptually under m03 above | No | No | No | No | (unchanged) | — | PARTIAL (unchanged) |
|
||||
|
||||
## ROOT 08 — Kubernetes Storage & Scheduling (newly wired this phase — was PLANNED)
|
||||
|
||||
| Module | Objective | Lesson | Demo | KC | Guided | Scenario(s) | Real Ops | Status |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| m01 Storage Fundamentals (NEW) | Explain ephemeral vs. persistent storage and how PV/PVC/StorageClass fit together | Yes | No | Yes | No | emptydir-pod, pv-pvc-mount, pvc-dynamic-task, storageclass-mcq | "a Pod restart on ephemeral storage silently loses data no one told the team about" | PARTIAL |
|
||||
| m02 Scheduling & Resource Governance (NEW) | Explain how the scheduler places Pods and how quotas/limits/priority shape that placement | Yes | No | Yes | No | node-label-selector, node-taint-toleration, pod-affinity-mcq, priority-class-task, pod-disruption-budget-task, resource-quota-ns, limitrange-task | "an eviction during a voluntary disruption without a PDB can take down every replica at once" | PARTIAL |
|
||||
|
||||
`cluster-upgrade-mcq` and `etcd-backup-mcq` (Cluster Administration bucket) are left
|
||||
**PRACTICE-ONLY** — they're genuine cluster-operator content but conceptually belong with ROOT 11
|
||||
(DevOps & Platform Engineering, out of Phase 6B's authoring scope per the authorization), not
|
||||
storage/scheduling. Mapping them here just to raise a coverage number would violate this
|
||||
document's own "quality over percentage" rule.
|
||||
|
||||
## ROOT 09 — Kubernetes Security (newly wired this phase — was PLANNED)
|
||||
|
||||
Not built as 22 disconnected CKS tasks. Four modules, foundational → advanced:
|
||||
|
||||
| Module | Focus | Classification of its scenarios | Status |
|
||||
|---|---|---|---|
|
||||
| m01 Security Model & SecurityContext (NEW) | Defense in depth, least privilege, non-root/capabilities/read-only-fs | CORE: cks-pod-security-context, cks-non-root-enforce, cks-drop-capabilities, cks-readonly-filesystem. GUIDED: cks-seccomp-profile. ADVANCED: cks-apparmor-profile. INDEPENDENT: pod-security-context (dup, see above) | **READY** |
|
||||
| m02 ServiceAccounts & RBAC (NEW) | Identity for workloads, least-privilege permissions | CORE: serviceaccount-pod, cks-automount-token, rbac-role, cks-rbac-clusterrole, cks-rbac-least-privilege. INDEPENDENT: rbac-clusterrole (dup, see above) | PARTIAL (lesson authored; no separate demo/guided) |
|
||||
| m03 Network & Supply Chain Security (NEW) | NetworkPolicy as a security control, image/registry trust, TLS ingress | CORE: cks-network-policy, cks-egress-namespace, cks-tls-ingress, cks-image-pull-secret. GUIDED: networkpolicy-egress. ADVANCED: cks-network-policy-metadata. ASSESSMENT BANK: cks-mcq-image-footprint | PARTIAL |
|
||||
| m04 Cluster Hardening & Runtime Security (NEW) | Pod Security Admission, audit policy, runtime sandboxing — survey-level, not daily-driver core | CORE: cks-psa-namespace, cks-immutable-secret. ADVANCED: cks-audit-policy, cks-runtime-class. ASSESSMENT BANK: cks-mcq-api-server, cks-mcq-audit-policy, cks-mcq-kubelet-auth, cks-mcq-psp-replacement, cks-mcq-runtime-security, cks-mcq-sandboxing | PARTIAL |
|
||||
|
||||
"ASSESSMENT BANK" scenarios are intentionally NOT wired as required module activities — they're
|
||||
recall-style MCQs well suited to a future structured exam pool (`rootnode-quick-assessment`-style
|
||||
definition), not hands-on core teaching. Not wired into `module_activities` this phase; flagged
|
||||
here so a future phase doesn't have to re-discover them.
|
||||
|
||||
## ROOT 10 — Kubernetes Troubleshooting & Observability
|
||||
|
||||
The gold-standard Health Probes module (`root-10-m01`) is **preserved unmodified** — it's the
|
||||
template, not a page to be made "consistent" with anything else.
|
||||
|
||||
| Module | Objective | Lesson | Demo | KC | Guided | Scenario(s) | Real Ops | Status |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| m01 Health Probes & Troubleshooting (existing, UNCHANGED) | — | Yes | Yes | Yes | Yes | rootnode-probe-port-mismatch | Yes | **READY** |
|
||||
| m03 The RootNode Troubleshooting Framework (NEW) | Apply Observe→Evidence→Hypothesis→Test→Repair→Validate→Prevent to a CrashLoopBackOff and a broken Service | Yes | Yes | Yes | No | crashloop-fix, endpoint-fix-task, container-logging-mcq (newly wired) | "a repeatable framework beats guessing under a 2am page" | PARTIAL (no separate guided exercise — the framework lesson + demo carry that role) |
|
||||
| m02 General Troubleshooting (existing) | Bare-lab bank | No | No | No | No | (unchanged) | — | PARTIAL (unchanged) |
|
||||
|
||||
## ROOT 12 — RootNode Real Ops
|
||||
|
||||
| Module | Objective | Lesson | Scenario(s) | Framing | Status |
|
||||
|---|---|---|---|---|---|
|
||||
| m01 Real Ops Incidents (existing, extended with framing) | Apply Detect→Triage→Diagnose→Mitigate→Recover→Verify→Learn to real incidents | Yes (new incident-response framework lesson) | rootnode-runaway-monitor, broken-deployment, rootnode-probe-port-mismatch (reused), rootnode-enforce-limits | Each lab gets a short incident-context intro + post-incident reflection (not full lesson/demo/KC per incident — see `docs/PHASE6B_ROOT_CURRICULUM_EXPANSION.md` for why) | PARTIAL |
|
||||
|
||||
`rootnode-real-ops` bundle contains exactly these 4 scenario ids (verified against
|
||||
`scenarios/bundles/rootnode-real-ops.json`) — no additional upstream scenario was intentionally
|
||||
included there beyond what `rootnode-path.json`'s `real-ops` stage already lists.
|
||||
|
||||
## ROOT CAPSTONE
|
||||
|
||||
| Module | Content | Status |
|
||||
|---|---|---|
|
||||
| m01 Operate and Recover (existing) | The real `rootnode-capstone` assessment | PARTIAL (unchanged — assessment engine, not lesson content) |
|
||||
| m02 Capstone Readiness (NEW) | What the capstone measures, prerequisites, how the timed/random draw works, time management, troubleshooting strategy, what completion means — derives real readiness from curriculum state, never publishes the draw or answers | **READY** |
|
||||
|
||||
## Scenarios deliberately left out of structured curriculum entirely
|
||||
|
||||
`cluster-upgrade-mcq`, `etcd-backup-mcq` (ROOT 11 territory, out of scope), and the 6
|
||||
ASSESSMENT-BANK-classified CKS MCQs above (not yet wired to an actual assessment definition —
|
||||
future work, not silently dropped). All remain reachable via the existing Practice tab/bundles,
|
||||
unaffected by any of this.
|
||||
@@ -0,0 +1,221 @@
|
||||
# ROOT_ Curriculum Inventory
|
||||
|
||||
Phase 6A §1 — a factual inventory of what already exists in this repository, gathered by
|
||||
inspecting `scenarios/data/`, `scenarios/bundles/`, `scenarios/rootnode-path.json`,
|
||||
`assessment_definitions` (013_assessments.sql), `programs` (019_programs.sql), `achievements`,
|
||||
and the curriculum/progress APIs — before any Phase 6A design work happened. Nothing here was
|
||||
modified to produce this document.
|
||||
|
||||
## 1. The current program model
|
||||
|
||||
One program exists: `root-kubernetes-foundations` (`programs` table, family `root`,
|
||||
`delivery_mode = self_paced`, `curriculum_path = rootnode-path`,
|
||||
`completion_assessment_id = rootnode-capstone`). `init`/`compile` families are supported by the
|
||||
schema's CHECK constraint but have zero seeded programs — no init_/compile_ curriculum exists
|
||||
anywhere in this repository.
|
||||
|
||||
## 2. `scenarios/rootnode-path.json` — the existing stage structure
|
||||
|
||||
The one curriculum path that exists, "RootNode Learning Path," version 1, with 7 stages:
|
||||
|
||||
| Stage | Scenario count | Completion rule |
|
||||
|---|---|---|
|
||||
| `foundation` | 20 | all_required |
|
||||
| `workloads` | 8 | all_required |
|
||||
| `networking` | 2 | all_required |
|
||||
| `configuration` | 5 | all_required |
|
||||
| `troubleshooting` | 5 | all_required |
|
||||
| `real-ops` (bundle `rootnode-real-ops`) | 4 | all_required |
|
||||
| `capstone` (type `capstone`) | 0 (references `rootnode-capstone`) | unlocks when all 6 content stages above are complete |
|
||||
|
||||
44 of the repository's 98 total scenarios are wired into this path. The remaining 54 exist as
|
||||
real, working scenario content but are not referenced by `rootnode-path.json` at all — see §4.
|
||||
|
||||
## 3. `scenarios/bundles/` — exam-prep bundle groupings
|
||||
|
||||
Five bundles exist (`1-k8s-basics`, `2-k8s-appdev`, `3-k8s-admin`, `4-k8s-security`,
|
||||
`rootnode-real-ops`), each a flat list of scenario ids used for CKS-style exam-prep grouping in
|
||||
the existing Practice area — overlapping by design (a scenario can appear in more than one
|
||||
bundle). These are unrelated to `rootnode-path.json`'s stages and untouched by Phase 6A.
|
||||
|
||||
## 4. Full scenario inventory
|
||||
|
||||
For every scenario in `scenarios/data/` (98 total): id, title, category, difficulty, task vs.
|
||||
MCQ, which bundle(s) it's in, which `rootnode-path.json` stage (if any) it's wired into, whether
|
||||
it appears in the `rootnode-capstone` or `rootnode-quick-assessment` `assessment_definitions`
|
||||
pool_spec, and whether it's RootNode-authored or generic upstream content.
|
||||
|
||||
Only 3 scenarios are RootNode-specific (`rootnode-runaway-monitor`,
|
||||
`rootnode-probe-port-mismatch`, `rootnode-enforce-limits`) — every other scenario, including all
|
||||
44 wired into `rootnode-path.json`, is generic upstream Kubernetes/CKS content. This matters for
|
||||
Phase 6A's course design: the "gold-standard module" spec explicitly asked for a RootNode-
|
||||
authored incident as the lab, which narrows the real choice to those 3 (plus `broken-deployment`,
|
||||
generic but already framed as an incident in the `real-ops` stage).
|
||||
|
||||
<!-- Generated from scenarios/data/*.json, scenarios/bundles/*.json, scenarios/rootnode-path.json,
|
||||
and platform/api/migrations/013_assessments.sql's seeded pool_specs — not hand-typed. -->
|
||||
|
||||
| Scenario ID | Title | Category | Difficulty | Type | Bundles | Path Stage | In Assessment | Origin |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| annotate-resource-basics | Annotating Kubernetes Resources | Core Concepts | Easy | task | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| broken-deployment | Debug a Failing Deployment | Troubleshooting | Medium | task | 3-k8s-admin, rootnode-real-ops | real-ops | capstone, quick-assessment | generic upstream |
|
||||
| cks-apparmor-profile | AppArmor Profiles | Workload Security | Hard | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-audit-policy | Write a Kubernetes Audit Policy | Cluster Setup | Hard | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-automount-token | Disable ServiceAccount Token | Workload Security | Easy | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-drop-capabilities | Drop Linux Capabilities | Workload Security | Medium | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-egress-namespace | Restrict Egress to Namespace | Network Security | Hard | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-image-pull-secret | Private Registry Image Pull Secret | Supply Chain Security | Easy | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-immutable-secret | Immutable Secrets | Cluster Security | Easy | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-mcq-api-server | API Server Authentication | Cluster Setup | Easy | mcq | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-mcq-audit-policy | Kubernetes Audit Logging Levels | Cluster Setup | Medium | mcq | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-mcq-image-footprint | Minimizing Image Footprint | Supply Chain Security | Easy | mcq | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-mcq-kubelet-auth | Securing the Kubelet | Cluster Setup | Medium | mcq | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-mcq-psp-replacement | Pod Security Evolution | Cluster Security | Easy | mcq | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-mcq-runtime-security | Runtime Security Tools | System Hardening | Medium | mcq | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-mcq-sandboxing | Container Runtime Sandboxing | System Hardening | Hard | mcq | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-network-policy | Default Deny Network Policy | Network Security | Medium | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-network-policy-metadata | Block Cloud Metadata | Network Security | Hard | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-non-root-enforce | Enforce Non-Root Container | Workload Security | Medium | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-pod-security-context | Pod Security Context | Workload Security | Medium | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-psa-namespace | Pod Security Admission | Cluster Security | Easy | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-rbac-clusterrole | Cluster-Level RBAC | Cluster Security | Medium | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-rbac-least-privilege | RBAC Least Privilege | Cluster Security | Medium | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-readonly-filesystem | Read-Only Root Filesystem | Workload Security | Easy | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-runtime-class | RuntimeClass for Sandboxed Workloads | System Hardening | Medium | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-seccomp-profile | Seccomp Profiles | Workload Security | Hard | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cks-tls-ingress | Secure Ingress with TLS | Network Security | Hard | task | 4-k8s-security | — | — | generic upstream |
|
||||
| cluster-upgrade-mcq | Cluster Upgrade Order | Cluster Administration | Hard | mcq | 3-k8s-admin | — | — | generic upstream |
|
||||
| configmap-volume | Mount ConfigMap as Volume | Configuration | Medium | task | 1-k8s-basics, 2-k8s-appdev | configuration | — | generic upstream |
|
||||
| container-logging-mcq | Container Logging and Monitoring | Troubleshooting | Easy | mcq | 2-k8s-appdev, 3-k8s-admin | — | — | generic upstream |
|
||||
| crashloop-fix | Fix a CrashLoopBackOff Pod | Troubleshooting | Medium | task | 2-k8s-appdev, 3-k8s-admin | — | — | generic upstream |
|
||||
| cronjob-task | Create a CronJob | Workloads & Scheduling | Easy | task | 2-k8s-appdev, 3-k8s-admin | — | — | generic upstream |
|
||||
| daemonset-mcq | Understanding DaemonSets | Workloads | Easy | mcq | 1-k8s-basics | workloads | quick-assessment | generic upstream |
|
||||
| delete-by-label-basics | Bulk Deletion by Label | Core Concepts | Easy | task | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| deploy-nginx | Deploy and Expose Nginx | Workloads | Easy | task | 1-k8s-basics, 2-k8s-appdev | workloads | capstone, quick-assessment | generic upstream |
|
||||
| deployment-rollback | Rolling Back a Deployment | Workloads & Scheduling | Easy | task | 2-k8s-appdev, 3-k8s-admin | — | — | generic upstream |
|
||||
| deployment-strategy-mcq | Deployment Update Strategies | Workloads | Medium | mcq | 1-k8s-basics | workloads | — | generic upstream |
|
||||
| dns-resolution-mcq | Kubernetes DNS Resolution | Services & Networking | Easy | mcq | 3-k8s-admin | — | — | generic upstream |
|
||||
| dry-run-manifest-basics | Generating YAML Manifests | Core Concepts | Easy | task | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| edit-deployment-basics | Updating Deployments | Workloads | Easy | task | 1-k8s-basics | workloads | capstone | generic upstream |
|
||||
| emptydir-pod | Shared emptyDir Volume Between Containers | Storage | Easy | task | 2-k8s-appdev, 3-k8s-admin | — | — | generic upstream |
|
||||
| endpoint-fix-task | Fix a Broken Service Selector | Troubleshooting | Medium | task | 3-k8s-admin | — | — | generic upstream |
|
||||
| env-vars-configmap | Environment Variables from ConfigMap | Configuration | Easy | task | 1-k8s-basics, 2-k8s-appdev | configuration | capstone | generic upstream |
|
||||
| etcd-backup-mcq | etcd Backup and Restore | Cluster Administration | Hard | mcq | 3-k8s-admin | — | — | generic upstream |
|
||||
| exec-command-basics | Execute Commands in Pods | Troubleshooting | Easy | task | 1-k8s-basics | troubleshooting | capstone, quick-assessment | generic upstream |
|
||||
| expose-service-basics | Exposing an Application | Services & Networking | Easy | task | 1-k8s-basics | networking | quick-assessment | generic upstream |
|
||||
| extract-logs-basics | Extract Container Logs | Troubleshooting | Easy | task | 1-k8s-basics | troubleshooting | capstone, quick-assessment | generic upstream |
|
||||
| field-selector-basics | Filtering Resources with Field Selectors | Core Concepts | Easy | task | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| ingress-task | Create an Ingress Resource | Services & Networking | Medium | task | 2-k8s-appdev, 3-k8s-admin | — | — | generic upstream |
|
||||
| init-container-task-basics | Pod with Init Container | Core Concepts | Medium | task | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| init-containers-mcq | Init Containers Behaviour | Core Concepts | Easy | mcq | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| job-task | Create a One-Shot Job | Workloads & Scheduling | Easy | task | 2-k8s-appdev, 3-k8s-admin | — | — | generic upstream |
|
||||
| jobs-cronjobs-mcq | Jobs vs CronJobs | Workloads | Easy | mcq | 1-k8s-basics | workloads | — | generic upstream |
|
||||
| jsonpath-basics | JSONPath Data Extraction | Core Concepts | Medium | task | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| kubectl-cp-basics | Copy Files to Containers | Troubleshooting | Medium | task | 1-k8s-basics | troubleshooting | capstone, quick-assessment | generic upstream |
|
||||
| kubectl-diff-basics | Previewing Changes with kubectl diff | Core Concepts | Medium | task | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| kubectl-essentials-mcq | Essential kubectl Commands | Core Concepts | Easy | mcq | 1-k8s-basics, 2-k8s-appdev | foundation | — | generic upstream |
|
||||
| kubectl-explain-basics | Exploring the API with kubectl explain | Core Concepts | Easy | task | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| kubectl-output-format-mcq | kubectl Output Formats | Core Concepts | Easy | mcq | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| kubectl-patch-basics | Patching Resources with kubectl patch | Core Concepts | Medium | task | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| labels-selectors-mcq | Labels and Selectors | Core Concepts | Easy | mcq | 1-k8s-basics, 2-k8s-appdev | foundation | — | generic upstream |
|
||||
| limitrange-task | LimitRange for Default Container Limits | Workloads & Scheduling | Medium | task | 3-k8s-admin | — | — | generic upstream |
|
||||
| liveness-probe-task | Configure a Liveness Probe | Workloads | Medium | task | 1-k8s-basics, 2-k8s-appdev | workloads | capstone, quick-assessment | generic upstream |
|
||||
| multi-container-pod-basics | Create a Multi-Container Pod | Core Concepts | Easy | task | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| namespaces-basics | Working with Namespaces | Core Concepts | Easy | task | 1-k8s-basics, 2-k8s-appdev | foundation | — | generic upstream |
|
||||
| network-policy | Isolate Traffic with NetworkPolicy | Networking | Hard | task | 3-k8s-admin | — | — | generic upstream |
|
||||
| networkpolicy-egress | Egress NetworkPolicy | Security | Hard | task | 3-k8s-admin | — | — | generic upstream |
|
||||
| node-cordon-basics | Cordoning and Uncordoning a Node | Cluster Management | Easy | task | 1-k8s-basics | troubleshooting | capstone, quick-assessment | generic upstream |
|
||||
| node-label-selector | Node Labels and nodeSelector | Cluster Administration | Easy | task | 3-k8s-admin | — | — | generic upstream |
|
||||
| node-taint-toleration | Taints and Tolerations | Cluster Administration | Medium | task | 3-k8s-admin | — | — | generic upstream |
|
||||
| nodeport-task | Expose a Deployment via NodePort | Services & Networking | Easy | task | 2-k8s-appdev, 3-k8s-admin | — | — | generic upstream |
|
||||
| pod-affinity-mcq | Pod Affinity vs Anti-Affinity | Workloads & Scheduling | Medium | mcq | 3-k8s-admin | — | — | generic upstream |
|
||||
| pod-basics-mcq | Pod Lifecycle States | Core Concepts | Easy | mcq | 1-k8s-basics, 2-k8s-appdev | foundation | — | generic upstream |
|
||||
| pod-disruption-budget-task | Configure a PodDisruptionBudget | Cluster Administration | Medium | task | 3-k8s-admin | — | — | generic upstream |
|
||||
| pod-env-vars-inline | Pod with Inline Environment Variables | Configuration | Easy | task | 1-k8s-basics | configuration | — | generic upstream |
|
||||
| pod-labeling-basics | Labelling Resources | Core Concepts | Easy | task | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| pod-node-assignment-mcq | Assigning Pods to Nodes | Scheduling | Medium | mcq | 1-k8s-basics | troubleshooting | — | generic upstream |
|
||||
| pod-security-context | Pod Security Context | Security | Medium | task | 2-k8s-appdev, 3-k8s-admin | — | — | generic upstream |
|
||||
| priority-class-task | Pod Priority and PriorityClass | Workloads & Scheduling | Medium | task | 3-k8s-admin | — | — | generic upstream |
|
||||
| pv-pvc-mount | Persistent Storage with PV and PVC | Storage | Medium | task | 3-k8s-admin | — | — | generic upstream |
|
||||
| pvc-dynamic-task | Dynamic PVC Provisioning | Storage | Medium | task | 2-k8s-appdev, 3-k8s-admin | — | — | generic upstream |
|
||||
| rbac-clusterrole | ClusterRole and ClusterRoleBinding | Cluster Administration | Medium | task | 3-k8s-admin | — | — | generic upstream |
|
||||
| rbac-role | RBAC: Create Role and RoleBinding | Security | Hard | task | 2-k8s-appdev, 3-k8s-admin | — | — | generic upstream |
|
||||
| readiness-probe-task | Configure a Readiness Probe | Workloads & Scheduling | Medium | task | 2-k8s-appdev, 3-k8s-admin | — | — | generic upstream |
|
||||
| resource-limits-task | Resource Requests and Limits | Configuration | Easy | task | 1-k8s-basics, 2-k8s-appdev | configuration | capstone | generic upstream |
|
||||
| resource-quota-ns | ResourceQuota for a Namespace | Cluster Administration | Medium | task | 3-k8s-admin | — | — | generic upstream |
|
||||
| restart-policy-mcq | Pod Restart Policies | Core Concepts | Easy | mcq | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| rolling-update-task | Perform a Rolling Update | Workloads | Easy | task | 1-k8s-basics, 2-k8s-appdev | workloads | capstone, quick-assessment | generic upstream |
|
||||
| rootnode-enforce-limits | Stop the Next Outage: Enforce Resource Defaults | Observability & Ops | Hard | task | rootnode-real-ops | real-ops | capstone, quick-assessment | RootNode-specific |
|
||||
| rootnode-probe-port-mismatch | Stuck Not-Ready: The Probe Port Mismatch | Observability & Ops | Medium | task | rootnode-real-ops | real-ops | capstone, quick-assessment | RootNode-specific |
|
||||
| rootnode-runaway-monitor | Recover the Monitoring Stack | Observability & Ops | Medium | task | rootnode-real-ops | real-ops | capstone, quick-assessment | RootNode-specific |
|
||||
| scale-deployment | Scaling a Deployment | Workloads | Easy | task | 1-k8s-basics, 2-k8s-appdev | workloads | capstone, quick-assessment | generic upstream |
|
||||
| secrets-basics | Creating and Using Secrets | Configuration | Easy | task | 1-k8s-basics, 2-k8s-appdev | configuration | capstone | generic upstream |
|
||||
| serviceaccount-pod | ServiceAccount for Pod API Access | Cluster Administration | Medium | task | 3-k8s-admin | — | — | generic upstream |
|
||||
| services-mcq | Kubernetes Service Types | Networking | Easy | mcq | 1-k8s-basics, 2-k8s-appdev | networking | quick-assessment | generic upstream |
|
||||
| sort-by-basics | Sorting kubectl Output | Core Concepts | Easy | task | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| static-pods-mcq | Static Pods in Kubernetes | Core Concepts | Medium | mcq | 1-k8s-basics | foundation | — | generic upstream |
|
||||
| storageclass-mcq | StorageClass Reclaim Policies | Storage | Easy | mcq | 3-k8s-admin | — | — | generic upstream |
|
||||
|
||||
## 5. Unused content bank, grouped for future ROOT_ modules
|
||||
|
||||
The 54 scenarios with no Path Stage above, grouped by category — a real, working content bank
|
||||
ROOT 07/08/09/10 can draw on in Phase 6B, not content that needs to be invented:
|
||||
|
||||
- **Workload Security (7):** cks-apparmor-profile, cks-automount-token, cks-drop-capabilities, cks-non-root-enforce, cks-pod-security-context, cks-readonly-filesystem, cks-seccomp-profile
|
||||
- **Cluster Setup (4):** cks-audit-policy, cks-mcq-api-server, cks-mcq-audit-policy, cks-mcq-kubelet-auth
|
||||
- **Network Security (4):** cks-egress-namespace, cks-network-policy-metadata, cks-network-policy, cks-tls-ingress
|
||||
- **Supply Chain Security (2):** cks-image-pull-secret, cks-mcq-image-footprint
|
||||
- **Cluster Security (5):** cks-immutable-secret, cks-mcq-psp-replacement, cks-psa-namespace, cks-rbac-clusterrole, cks-rbac-least-privilege
|
||||
- **System Hardening (3):** cks-mcq-runtime-security, cks-mcq-sandboxing, cks-runtime-class
|
||||
- **Cluster Administration (8):** cluster-upgrade-mcq, etcd-backup-mcq, node-label-selector, node-taint-toleration, pod-disruption-budget-task, rbac-clusterrole, resource-quota-ns, serviceaccount-pod
|
||||
- **Troubleshooting (3):** container-logging-mcq, crashloop-fix, endpoint-fix-task
|
||||
- **Workloads & Scheduling (7):** cronjob-task, deployment-rollback, job-task, limitrange-task, pod-affinity-mcq, priority-class-task, readiness-probe-task
|
||||
- **Services & Networking (3):** dns-resolution-mcq, ingress-task, nodeport-task
|
||||
- **Storage (4):** emptydir-pod, pv-pvc-mount, pvc-dynamic-task, storageclass-mcq
|
||||
- **Networking (1):** network-policy
|
||||
- **Security (3):** networkpolicy-egress, pod-security-context, rbac-role
|
||||
|
||||
## 6. Other existing systems inspected
|
||||
|
||||
- **Assessment definitions** (`assessment_definitions`): `rootnode-capstone` (90 min, hint_policy
|
||||
`flagged`, required = the 4 real-ops scenarios, random draw across workloads/config/
|
||||
troubleshooting) and `rootnode-quick-assessment` (60 min exam, fully random draw). Both real,
|
||||
both working, neither touched by Phase 6A — the new capstone course activity references
|
||||
`rootnode-capstone` by id only.
|
||||
- **Achievements** (`achievements`): 4 types already implemented — `stage_completed`,
|
||||
`curriculum_completed`, `assessment_completed`, `capstone_completed`. Phase 6A's
|
||||
`auto_assessment` completion behavior reads `assessment_completed`/`capstone_completed` rows
|
||||
by `metadata->>'assessment_definition_id'` — no new achievement type needed.
|
||||
- **Progress/attempt model** (`progress`, `scenario_attempts`): unchanged. `progress` is what
|
||||
`curriculum-rules.js` and the new `auto_scenario` completion behavior both read.
|
||||
- **Curriculum APIs**: `curriculum.js` loads scenarios/bundles/path from disk into an in-process
|
||||
cache; `curriculum-rules.js` computes stage lock/completion/percent purely from that path +
|
||||
a progress map. Phase 6A's `course-progress.js` follows the identical shape one level higher
|
||||
(course/module/activity instead of stage/scenario).
|
||||
- **Self-paced dashboard** (`ProgramsTab.jsx`) and **classroom dashboard** (`ClassroomTab.jsx`,
|
||||
admin `ClassroomView.jsx`): both currently show curriculum STATE (percent, next scenario) with
|
||||
no notion of lessons/modules — Phase 6A's `CourseView.jsx`/`CurriculumView.jsx` add that layer
|
||||
alongside, not instead of, these.
|
||||
- **Admin curriculum/program tooling** (`admin-learning-routes.js`, `admin-programs-routes.js`):
|
||||
read-only scenario/stage views and program/enrollment CRUD; no course/module concept existed
|
||||
before this phase.
|
||||
|
||||
## 7. What this inventory rules out
|
||||
|
||||
- No init_/compile_ curriculum content exists anywhere — confirmed by a full scan of
|
||||
`scenarios/data/`, not an assumption.
|
||||
- No DevOps/CI/CD/IaC/GitOps scenario content exists (ROOT 11 is necessarily 100% PLANNED).
|
||||
- No Linux-fundamentals, Git, or Docker-specific scenario content exists (ROOT 01-04 are
|
||||
necessarily 100% PLANNED) — everything in `scenarios/data/` is Kubernetes/CKS content.
|
||||
|
||||
## 8. Phase 6B update — scenario coverage after curriculum authoring
|
||||
|
||||
Phase 6B wired 44 more of the scenarios above into structured curriculum (verified by direct
|
||||
query against `module_activities.scenario_id`, not estimated): **88 of 98 scenarios (89.8%)**
|
||||
are now in structured curriculum, up from 44 at the end of Phase 6A. The remaining 10 —
|
||||
`cluster-upgrade-mcq`, `etcd-backup-mcq` (ROOT 11 territory), 7 CKS-style MCQs classified as
|
||||
future ASSESSMENT BANK material, and `readiness-probe-task` (a genuine, documented gap for
|
||||
Phase 6C) — are listed with their specific reasons in
|
||||
`docs/PHASE6B_ROOT_CURRICULUM_EXPANSION.md` §5. The full course-by-course mapping, including
|
||||
which scenarios were classified CORE/GUIDED/INDEPENDENT/ADVANCED/ASSESSMENT-BANK and why, lives
|
||||
in `docs/ROOT_CURRICULUM_COVERAGE_MATRIX.md` — this file is intentionally not duplicated there.
|
||||
@@ -0,0 +1,118 @@
|
||||
# ROOT Pilot Curriculum — Authoritative Manifest
|
||||
|
||||
This is the single authoritative answer to "what are we actually teaching in the first paid
|
||||
pilot." It uses honest READY / PARTIAL / DEFERRED / NOT IN PILOT labels — nothing here is
|
||||
marketed as more finished than it is. See
|
||||
[`docs/ROOT_PILOT_READINESS_STANDARD.md`](ROOT_PILOT_READINESS_STANDARD.md) for what READY
|
||||
means and [`docs/ROOT11_DEVOPS_PLATFORM_SCOPE.md`](ROOT11_DEVOPS_PLATFORM_SCOPE.md) for the
|
||||
ROOT 11 scope decision referenced below.
|
||||
|
||||
## Pilot delivery mode: instructor-led (cohort-based)
|
||||
|
||||
The first pilot is scoped as **instructor-led**, not fully self-paced. Every module below that
|
||||
is marked READY meets the self-paced bar (no "your instructor will explain this" gaps). A
|
||||
smaller set of modules — flagged explicitly — are READY for instructor-led delivery only,
|
||||
because they are dense hands-on practice banks whose sibling module in the same course already
|
||||
carries the taught concept; running them live lets an instructor supply framing a self-paced
|
||||
student wouldn't get. Fully self-paced delivery of those specific modules is deferred to a
|
||||
future phase pending an authored teaching layer for each. See Section 16 discussion below.
|
||||
|
||||
## Program: ROOT_ (root-kubernetes-foundations)
|
||||
|
||||
| Course | Pilot status | Notes |
|
||||
|---|---|---|
|
||||
| ROOT 01 Computing & Linux Foundations | **NOT IN PILOT** | Never authored (status `planned`, zero modules). Out of Phase 6C's scope; students are assumed to arrive with basic Linux/CLI comfort for this pilot. |
|
||||
| ROOT 02 Linux Administration & Networking | **NOT IN PILOT** | Same as above. |
|
||||
| ROOT 03 Git & Collaborative Development | **NOT IN PILOT** | Same as above. |
|
||||
| ROOT 04 Containers & Docker | **NOT IN PILOT** | Same as above. |
|
||||
| ROOT 05 Kubernetes Foundations | **IN PILOT** | See modules below. |
|
||||
| ROOT 06 Kubernetes Application Workloads | **IN PILOT** | See modules below. |
|
||||
| ROOT 07 Kubernetes Networking & Configuration | **IN PILOT** | See modules below. |
|
||||
| ROOT 08 Kubernetes Storage & Scheduling | **DEFERRED — optional supplementary** | Does not gate the capstone (`rootnode-path.json`'s six stages never included storage/scheduling). Both modules remain PARTIAL (lesson+knowledge-check only, no demo/guided/reflection). Offer as optional supplementary content to instructors who want it; do not count it in the pilot's required completion path. |
|
||||
| ROOT 09 Kubernetes Security | **IN PILOT** | See modules below. |
|
||||
| ROOT 10 Kubernetes Troubleshooting & Observability | **IN PILOT** | See modules below. |
|
||||
| ROOT 11 DevOps & Platform Engineering | **IN PILOT — orientation only** | Lesson-only closing module. Hands-on GitOps/CI-CD/IaC/Observability content explicitly deferred; see scope doc. |
|
||||
| ROOT 12 Real Ops | **IN PILOT** | READY. |
|
||||
| ROOT Capstone | **IN PILOT** | READY. |
|
||||
|
||||
## Module-level detail (courses IN pilot only)
|
||||
|
||||
### ROOT 05 — Kubernetes Foundations
|
||||
| Module | Status | Prerequisite | Outcome | Delivery | Required/Optional | Labs/Assessment |
|
||||
|---|---|---|---|---|---|---|
|
||||
| root-05-m00-kubernetes-mental-model | READY | none | Explain desired vs. actual state; run first kubectl commands | Self-paced or instructor-led | 6 required | 1 lab |
|
||||
| root-05-m01-pods-and-kubectl-basics | READY | m00 | Broad kubectl command fluency (labels, JSONPath, patch, dry-run, etc.) | Self-paced or instructor-led (lesson+reflection added Phase 6C) | 22 required | 20 labs/MCQs |
|
||||
|
||||
### ROOT 06 — Kubernetes Application Workloads
|
||||
| Module | Status | Prerequisite | Outcome | Delivery | Required/Optional | Labs/Assessment |
|
||||
|---|---|---|---|---|---|---|
|
||||
| root-06-m02-deployments-scaling-updates | READY | ROOT 05 | Scale and roll out/back Deployments with confidence | Self-paced or instructor-led | 7 required | 1 lab |
|
||||
| root-06-m03-jobs-cronjobs-daemonsets | READY | m02 | Choose the correct workload primitive for a given job | Self-paced or instructor-led (upgraded Phase 6C) | 7 required | 2 labs |
|
||||
| root-06-m01-deployments-and-workloads | **PARTIAL — instructor-led only** | ROOT 05 | Deeper Deployment practice bank (8 tasks, no bookending lesson) | **Instructor-led only this pilot**; self-paced deferred | 8 required | 8 labs |
|
||||
|
||||
### ROOT 07 — Kubernetes Networking & Configuration
|
||||
| Module | Status | Prerequisite | Outcome | Delivery | Required/Optional | Labs/Assessment |
|
||||
|---|---|---|---|---|---|---|
|
||||
| root-07-m03-services-and-service-discovery | READY | ROOT 06 | Explain Service types and discovery | Self-paced or instructor-led | 6 required | 1 lab |
|
||||
| root-07-m02-configuration-and-secrets | READY | m03 | Choose ConfigMap vs. Secret vs. env var vs. volume correctly | Self-paced or instructor-led (new lesson/demo/KC/guided/reflection Phase 6C) | 10 required | 5 labs |
|
||||
| root-07-m04-ingress-networkpolicy-foundations | READY | m02 | Trace request routing through Ingress; plan network isolation | Self-paced or instructor-led (upgraded Phase 6C) | 7 required | 2 labs |
|
||||
| root-07-m01-services-and-networking | **PARTIAL — instructor-led only** | ROOT 06 | Deeper Services/networking practice bank (4 tasks, no bookending lesson) | **Instructor-led only this pilot**; self-paced deferred | 4 required | 4 labs |
|
||||
|
||||
### ROOT 09 — Kubernetes Security
|
||||
| Module | Status | Prerequisite | Outcome | Delivery | Required/Optional | Labs/Assessment |
|
||||
|---|---|---|---|---|---|---|
|
||||
| root-09-m01-security-model-and-securitycontext | READY | ROOT 07 | Harden a Pod's SecurityContext with reasoning, not memorization | Self-paced or instructor-led | 9 required + 3 optional | labs |
|
||||
| root-09-m02-serviceaccounts-and-rbac | READY | m01 | Write least-privilege RBAC | Self-paced or instructor-led (upgraded Phase 6C) | 10 required + 1 optional | 6 labs |
|
||||
| root-09-m03-network-and-supplychain-security | READY | m02 | Apply default-deny NetworkPolicy and supply-chain controls | Self-paced or instructor-led (upgraded Phase 6C) | 9 required + 2 optional | 6 labs |
|
||||
| root-09-m04-cluster-hardening-and-runtime | READY (survey depth, documented exception) | m03 | Survey-level exposure to runtime/cluster hardening topics, checked with a real assessment | Self-paced or instructor-led | 4 required + 2 optional | 4 labs + **RootNode Kubernetes Security Assessment** (9-scenario timed exam) |
|
||||
|
||||
### ROOT 10 — Kubernetes Troubleshooting & Observability
|
||||
| Module | Status | Prerequisite | Outcome | Delivery | Required/Optional | Labs/Assessment |
|
||||
|---|---|---|---|---|---|---|
|
||||
| root-10-m01-health-probes-and-troubleshooting | READY | ROOT 09 | Configure and diagnose liveness/readiness probes | Self-paced or instructor-led | 6 required + 1 optional | 1 required lab + 1 optional lab (readiness-probe-task, wired Phase 6C) |
|
||||
| root-10-m03-troubleshooting-framework | READY | m01 | Apply the seven-step Observe→Evidence→...→Prevent-Recurrence framework to novel failures | Self-paced or instructor-led (upgraded Phase 6C) | 7 required | 2 labs |
|
||||
| root-10-m02-general-troubleshooting | **PARTIAL — instructor-led only** | m01 | Deeper troubleshooting practice bank (6 tasks, no bookending lesson) | **Instructor-led only this pilot**; self-paced deferred | 6 required | 6 labs |
|
||||
|
||||
### ROOT 11 — DevOps & Platform Engineering
|
||||
| Module | Status | Prerequisite | Outcome | Delivery | Required/Optional | Labs/Assessment |
|
||||
|---|---|---|---|---|---|---|
|
||||
| root-11-m01-devops-platform-thinking | PARTIAL (honest, lesson-only by design) | ROOT 10 (course-level gate) | Name DevOps/platform engineering practices already exercised in ROOT 05-10/12; understand what's still missing for GitOps/CI-CD/IaC/observability | Self-paced or instructor-led | 3 required | none — orientation only |
|
||||
|
||||
### ROOT 12 — Real Ops
|
||||
| Module | Status | Prerequisite | Outcome | Delivery | Required/Optional | Labs/Assessment |
|
||||
|---|---|---|---|---|---|---|
|
||||
| root-12-m01-real-ops-incidents | READY (documented exception — deliberately unguided by design) | ROOT 10 | Run structured incident response without a hint system | Instructor-led recommended for first exposure; self-paced viable given deliberate design | 6 required | 4 incident labs |
|
||||
|
||||
### ROOT Capstone
|
||||
| Module | Status | Prerequisite | Outcome | Delivery | Required/Optional | Labs/Assessment |
|
||||
|---|---|---|---|---|---|---|
|
||||
| root-capstone-m02-capstone-readiness | READY | ROOT 12 | Confirm readiness before attempting the capstone | Self-paced or instructor-led | 4 required | checklist/prep activities |
|
||||
| root-capstone-m01-operate-and-recover | READY (documented exception — the capstone IS the module) | m02 + capstone unlock gate | Operate and recover a production-like environment under real ops discipline | Instructor-observed recommended | 1 required | capstone exam |
|
||||
|
||||
## Instructor prep requirements
|
||||
|
||||
Every module marked "instructor-led only" or "instructor-led recommended" above requires the
|
||||
instructor to have read that module's Instructor Notes section (timing, misconceptions,
|
||||
discussion prompts) before delivering it live — this content already exists for every upgraded
|
||||
module (see `docs/ROOT_PILOT_READINESS_STANDARD.md` checklist item 6). No additional
|
||||
instructor-only prep material was created outside the per-activity Instructor Notes.
|
||||
|
||||
## What is intentionally NOT in the first pilot
|
||||
|
||||
- ROOT 01-04 (never authored).
|
||||
- ROOT 08 Kubernetes Storage & Scheduling (non-capstone-gating; both modules remain PARTIAL).
|
||||
- ROOT 11 modules 2-7 (GitOps, CI/CD, IaC, hands-on Observability, hands-on Reliability) — see
|
||||
scope doc.
|
||||
- Fully self-paced delivery of root-06-m01, root-07-m01, and root-10-m02 specifically — these
|
||||
three run instructor-led in the first pilot; self-paced delivery for exactly these three
|
||||
modules is deferred pending an authored teaching layer for each.
|
||||
- `cluster-upgrade-mcq` and `etcd-backup-mcq` (Cluster Administration category) — no current
|
||||
ROOT module teaches kubeadm-based cluster administration or etcd backup/restore; these two
|
||||
scenarios stay unwired pending a future Cluster Administration module, not due to neglect.
|
||||
|
||||
## Bottom line
|
||||
|
||||
Every module the pilot's required completion path depends on (ROOT 05-07, 09-10, 12, Capstone)
|
||||
meets the READY bar for instructor-led delivery. Three specific modules run instructor-led only
|
||||
this pilot. ROOT 08 and most of ROOT 11 are explicitly out of the required path. Given that
|
||||
scope, the pilot curriculum, delivered instructor-led, is ready.
|
||||
@@ -0,0 +1,83 @@
|
||||
# ROOT_ Pilot Readiness Standard
|
||||
|
||||
Phase 6C §2 — the explicit bar a module must clear to be called READY for the first paid ROOT_
|
||||
pilot. Extends `docs/COURSE_AUTHORING_STANDARD.md` with a pass/fail checklist rather than
|
||||
general authoring guidance, and replaces ad hoc judgment calls with a documented rule every
|
||||
module in `docs/ROOT_PILOT_CURRICULUM.md` is checked against.
|
||||
|
||||
## The standard is about sufficiency, not a fixed activity count
|
||||
|
||||
Phase 6A/6B's gold-standard pattern (lesson → demonstration → knowledge check → guided exercise
|
||||
→ lab → reflection) is a strong DEFAULT for an operational skill module, not a mechanical
|
||||
checklist to force onto every module regardless of subject. The actual test is:
|
||||
|
||||
> **Does the learner have enough instruction and scaffolding to reasonably perform the practical
|
||||
> work, understand why it matters, and check their own understanding — without an instructor
|
||||
> filling a gap the content itself should have closed?**
|
||||
|
||||
A module can be READY with fewer activities if the subject genuinely doesn't need all six. A
|
||||
module can have all six activities and still not be READY if any of them fails its own bar
|
||||
below (a "lab" with no framing, a "knowledge check" that only tests recall of the immediately
|
||||
preceding paragraph).
|
||||
|
||||
## The READY checklist
|
||||
|
||||
A module is READY only if ALL of the following are true:
|
||||
|
||||
1. **Concept instruction exists and is substantive.** A `lesson` (or equivalent) activity
|
||||
explains the underlying mental model — what problem this solves, how it actually works, what
|
||||
goes wrong when it's misunderstood — not a restated scenario description or a bullet list of
|
||||
commands with no explanation of why.
|
||||
2. **The learner sees the reasoning demonstrated, not just described**, wherever the subject
|
||||
involves diagnosing or configuring something non-trivial. This can be a `demonstration`
|
||||
activity, OR — for a purely conceptual/orientation module where there's nothing to
|
||||
diagnose — folded into the lesson itself with a worked example. A module that skips this
|
||||
entirely for an operational/troubleshooting subject is not READY.
|
||||
3. **The learner can check their own understanding** before attempting independent work — a
|
||||
`knowledge_check` (or, for an orientation module with no lab, a reflection that serves the
|
||||
same self-check purpose) that tests INTERPRETATION and REASONING (see
|
||||
`docs/COURSE_AUTHORING_STANDARD.md` §9's examples), not trivia.
|
||||
4. **The learner is not dropped into a lab cold.** Where a `lab` activity exists, either a
|
||||
`guided_exercise` bridges lesson to lab, OR the lesson/demonstration already gave the learner
|
||||
everything they need to attempt the lab's specific task without guessing at the mechanism
|
||||
being tested. A bare lab reference with a one-line "go do this" intro and no prior teaching
|
||||
of the concept it tests is NOT READY, regardless of how good the lab's own hints are — Labs
|
||||
are the laboratory, not the whole course (Phase 6A's own founding principle).
|
||||
5. **The module closes with operational context**, where the subject has real production
|
||||
consequences — a `reflection` connecting what was practiced to why it matters beyond passing
|
||||
validation (see `docs/COURSE_AUTHORING_STANDARD.md` §15's Real Ops takeaway examples). A
|
||||
purely conceptual orientation module (e.g. Capstone Readiness) may satisfy this through its
|
||||
own closing content rather than a separate reflection activity.
|
||||
6. **Instructor notes exist wherever a live class would benefit from them** — at minimum on the
|
||||
lesson and any demonstration/guided-exercise activity: timing, a likely misconception, and
|
||||
(for a demonstration) a demo plan. Not required on every activity type (a bare lab activity
|
||||
with no `content_path` has nothing to attach notes to).
|
||||
7. **Self-paced sufficiency**: nothing in the module says or implies "your instructor will
|
||||
explain this" for content available to self-paced students. If an activity is genuinely
|
||||
instructor-only (rare in this content model — everything self-paced students can reach uses
|
||||
the same file), it must be excluded from self-paced delivery structurally, not by a
|
||||
disclaimer.
|
||||
8. **Every reference resolves** — `scenario_id` points at a real scenario file,
|
||||
`assessment_definition_id` at a real definition, `content_path` at a real file that actually
|
||||
loads (`infra/scripts/validate-course-content.js` proves this mechanically; a human read-
|
||||
through confirms the CONTENT itself, not just that the file exists).
|
||||
|
||||
## What does NOT make a module READY
|
||||
|
||||
- Having a `lab` activity. A scenario reference is necessary, never sufficient.
|
||||
- Having headings that look like a lesson. Prose volume is not instruction quality.
|
||||
- Existing in the database with `status = 'ready'`. That field is the AUTHOR's claim — this
|
||||
document is the actual bar it's checked against, and `docs/ROOT_PILOT_CURRICULUM.md` is where
|
||||
every claim gets verified against this checklist by a human, not just by the field's value.
|
||||
- Passing `e2e-phase6c.js`'s structural checks alone. Automated tests prove the DATA MODEL is
|
||||
correct (references resolve, completion derives correctly) — they cannot judge whether a
|
||||
lesson's explanation is actually good. That judgment is made once, explicitly, per module,
|
||||
and recorded in the pilot manifest.
|
||||
|
||||
## DEFERRED / NOT IN PILOT is not a failure
|
||||
|
||||
A module that doesn't meet this bar is not a defect to apologize for — it's an honest status.
|
||||
`docs/ROOT_PILOT_CURRICULUM.md` uses DEFERRED (real content exists, not yet pilot-quality) and
|
||||
NOT IN PILOT (deliberately excluded — e.g. ROOT 01-04, most of ROOT 11) as first-class,
|
||||
legitimate outcomes. Marking something READY that isn't is a worse outcome than an honestly
|
||||
smaller pilot.
|
||||
@@ -0,0 +1,109 @@
|
||||
# ROOT_ Program Architecture
|
||||
|
||||
Phase 6A §2 — the canonical course architecture for `root_` (Linux → systems → containers →
|
||||
Kubernetes → DevOps/platform engineering, ages ~15-18+/college/adult per the youth ladder — see
|
||||
`docs/PHASE6A_ROOT_COURSE_FOUNDATION.md` §1). This document is the design; the actual seed data
|
||||
lives in `platform/api/migrations/022_course_content.sql`, generated to match this exactly.
|
||||
|
||||
All 13 rows below are `courses` under the ONE existing program, `root-kubernetes-foundations`
|
||||
(`programs` table, unchanged — see §"why one program, not thirteen" below). Every status claim
|
||||
here is grounded in `docs/ROOT_CURRICULUM_INVENTORY.md`, not aspirational.
|
||||
|
||||
**Updated for Phase 6B** (`platform/api/migrations/023_phase6b_curriculum.sql`,
|
||||
`docs/PHASE6B_ROOT_CURRICULUM_EXPANSION.md`): ROOT 08 and 09 moved from PLANNED to PARTIAL, and
|
||||
6 courses gained new teaching modules — the module-level detail and evidence for every new
|
||||
placement lives in `docs/ROOT_CURRICULUM_COVERAGE_MATRIX.md`, not duplicated here.
|
||||
|
||||
## Course sequence
|
||||
|
||||
| # | Course | Status | Why |
|
||||
|---|---|---|---|
|
||||
| 1 | Computing & Linux Foundations | **PLANNED** | No lab content exists in this repository — every scenario is Kubernetes/CKS. |
|
||||
| 2 | Linux Administration & Networking | **PLANNED** | Same — no Linux-admin scenario content exists. |
|
||||
| 3 | Git & Collaborative Development | **PLANNED** | No Git scenario content exists. |
|
||||
| 4 | Containers & Docker | **PLANNED** | No Docker-specific scenario content exists (the Lab itself runs k3s, but no student-facing "learn Docker" scenarios exist). |
|
||||
| 5 | Kubernetes Foundations | **PARTIAL** | Phase 6B added a READY front-door module (`root-05-m00-kubernetes-mental-model` — full lesson/demo/knowledge-check/guided-exercise/lab/reflection) sequenced before the existing 20-scenario bare-lab bank (`root-05-m01`), which now sits behind it. |
|
||||
| 6 | Kubernetes Application Workloads | **PARTIAL** | Phase 6B added a READY `root-06-m02-deployments-scaling-updates` and a PARTIAL `root-06-m03-jobs-cronjobs-daemonsets`, both in front of the existing bare-lab bank. |
|
||||
| 7 | Kubernetes Networking & Configuration | **PARTIAL** | Phase 6B added a READY `root-07-m03-services-and-service-discovery` and a PARTIAL `root-07-m04-ingress-networkpolicy-foundations`. The existing configuration bank (`root-07-m02`) still has no authored teaching content — an honest, documented gap. |
|
||||
| 8 | Kubernetes Storage & Scheduling | **PARTIAL** | Phase 6B wired the real scenario content the inventory identified (PV/PVC, StorageClass, taints/tolerations, PDB, quotas/limits) into two new PARTIAL modules — moved off PLANNED for the first time. |
|
||||
| 9 | Kubernetes Security | **PARTIAL** | Phase 6B built the 22-scenario CKS-style bank into four modules (Security Model & SecurityContext — READY; ServiceAccounts & RBAC, Network & Supply Chain Security, Cluster Hardening & Runtime — all PARTIAL), classifying scenarios CORE/GUIDED/INDEPENDENT/ADVANCED/ASSESSMENT-BANK rather than exposing them as 22 disconnected tasks — moved off PLANNED. |
|
||||
| 10 | Kubernetes Troubleshooting & Observability | **PARTIAL** | The Phase 6A gold-standard module (READY, unmodified) plus a new PARTIAL `root-10-m03-troubleshooting-framework` (the generalized Observe/Evidence/Hypothesis/Test/Repair/Validate/Prevent framework) and the existing bare-lab bank, now sequenced last. |
|
||||
| 11 | DevOps & Platform Engineering Foundations | **PLANNED** | No CI/CD/IaC/GitOps scenario content exists anywhere in this repository — unchanged, out of Phase 6B's authoring scope. |
|
||||
| 12 | RootNode Real Ops | **PARTIAL** | Phase 6B added an incident-response framing lesson and a post-incident reflection wrapping the existing 4-scenario module — the incidents themselves are unchanged, still deliberately un-guided. |
|
||||
| 13 | ROOT Capstone — Operate and Recover a Production-Like Environment | **PARTIAL** | Phase 6B added a READY Capstone Readiness module (orientation, no answers) sequenced before the existing capstone assessment module — see `docs/ROOT_CAPSTONE_READINESS.md`. |
|
||||
|
||||
No course is marked READY at the course level in Phase 6A — that's honest: a course only earns
|
||||
READY once every module in it has real teaching content, and Phase 6A deliberately built exactly
|
||||
one complete module (the gold standard) as the authoring template, not a finished curriculum.
|
||||
|
||||
## Prerequisite chain
|
||||
|
||||
```
|
||||
01 Linux Foundations → 02 Linux Admin/Networking → 03 Git → 04 Containers/Docker
|
||||
│
|
||||
▼
|
||||
05 Kubernetes Foundations
|
||||
│
|
||||
▼
|
||||
06 Application Workloads
|
||||
│ │
|
||||
┌─────────────────────┤ │
|
||||
▼ ▼ ▼
|
||||
07 Networking/Config 09 Security 10 Troubleshooting
|
||||
│ │ │
|
||||
▼ │ ▼
|
||||
08 Storage/Scheduling │ 11 DevOps/Platform
|
||||
▼
|
||||
12 RootNode Real Ops
|
||||
│
|
||||
▼
|
||||
CAPSTONE — Operate & Recover
|
||||
```
|
||||
|
||||
01-04 have no prerequisite chain enforced against each other yet (all PLANNED, zero content —
|
||||
locking a PLANNED course behind another PLANNED course would just be decorative). 05 is the
|
||||
first course with a real prerequisite gate. See `docs/COURSE_CONTENT_MODEL.md`'s "prerequisite/
|
||||
locking rule" for why a PLANNED/content-free prerequisite never actually blocks anything.
|
||||
|
||||
## Deviations from the spec's suggested sequence, and why
|
||||
|
||||
The authorization's suggested sequence listed Security (09) after Storage/Scheduling (08) and
|
||||
before Troubleshooting (10). This document keeps that numbering for continuity, but the
|
||||
**prerequisite graph** branches 09 and 10 both directly off 06 (Application Workloads) rather
|
||||
than chaining 08→09→10 linearly — the existing content doesn't actually build 09 on top of 08's
|
||||
concepts (RBAC/PSA/network-policy security content is orthogonal to PV/PVC/scheduling content),
|
||||
so a straight linear chain would impose a false dependency. This is exactly the case the
|
||||
authorization's "if existing content suggests a better grouping, document the reasoning" line
|
||||
anticipated.
|
||||
|
||||
## Real Ops and Capstone placement
|
||||
|
||||
Real Ops (12) sits after Troubleshooting (10), matching the existing curriculum's own stage
|
||||
order (`rootnode-path.json`'s `real-ops` stage already comes after `troubleshooting`) — Real Ops
|
||||
scenarios assume the diagnostic skills Troubleshooting teaches. The Capstone sits after Real Ops
|
||||
alone, not after every course, because that's what the existing `rootnode-capstone` assessment
|
||||
definition's own `unlock_rule` already requires (all 6 content stages of `rootnode-path.json`
|
||||
complete — which necessarily includes 05-06-07-10-12's underlying scenarios, plus networking/
|
||||
configuration). 08, 09, and 11 are not currently gating the capstone, because their scenario
|
||||
content isn't part of `rootnode-path.json`'s stage list at all yet — see
|
||||
`docs/COURSE_CONTENT_MODEL.md` for how course-level completion is computed independently of that
|
||||
assessment-engine unlock_rule (the two systems currently agree by construction, not by a shared
|
||||
implementation).
|
||||
|
||||
## Why one program, not thirteen
|
||||
|
||||
`programs` (Phase 5B) models the commercial/academic unit a student enrolls in. root_ is
|
||||
described by this authorization as one continuous ladder — "Linux → systems → containers →
|
||||
Kubernetes → DevOps/platform engineering" — not thirteen separate purchases. Attaching all 13
|
||||
courses to the single existing `root-kubernetes-foundations` program means:
|
||||
|
||||
- Zero changes to the `programs` table, enrollment model, or entitlement logic — an enrolled
|
||||
student automatically has structured access to the whole ladder as it's built out.
|
||||
- No renaming/redefinition of the existing program (which would risk a real, if cosmetic, break
|
||||
for anyone already enrolled) — see Phase 6A's explicit "do not redesign working systems"
|
||||
constraint.
|
||||
|
||||
If a future phase needs `root_` split into separately-purchasable programs (e.g. "Kubernetes
|
||||
Foundations" sold standalone from "DevOps"), that's a `programs`-table decision to make
|
||||
deliberately later, informed by real pricing/business requirements — not something to guess at
|
||||
here.
|
||||
@@ -0,0 +1,186 @@
|
||||
# Runbook: Google Workspace Email Configuration
|
||||
|
||||
Everything in this document happens OUTSIDE this repository, in the Google Admin Console and
|
||||
Google Cloud Console, and is marked **REQUIRES GOOGLE WORKSPACE ADMIN ACTION**. Nothing here can
|
||||
be scripted or automated from the codebase — Chase (or whoever holds Workspace super-admin) has
|
||||
to do these steps by hand, once, before RootNode Academy can send real email.
|
||||
|
||||
See `docs/PHASE5C_ENROLLMENT_COMMUNICATIONS.md` §2 for why this mechanism was chosen over SMTP
|
||||
or a human login.
|
||||
|
||||
## 1. Selected mechanism
|
||||
|
||||
**Gmail API (`users.messages.send`) via a Google Cloud service account using Workspace
|
||||
domain-wide delegation**, impersonating one dedicated sending mailbox. No human password is
|
||||
ever stored by RootNode. The delegated scope is exactly `https://www.googleapis.com/auth/
|
||||
gmail.send` — nothing broader.
|
||||
|
||||
## 2. REQUIRES GOOGLE WORKSPACE ADMIN ACTION: create the sending mailbox
|
||||
|
||||
Decide the address RootNode sends as — e.g. `[email protected]` or
|
||||
`[email protected]`. This repository does not assume or hard-code one; it's whatever
|
||||
you set `EMAIL_FROM_ADDRESS` to (§5).
|
||||
|
||||
1. In Google Admin Console (admin.google.com) → Directory → Users, create this mailbox as a
|
||||
real Workspace user (or a Google Group with "Collaborative Inbox" if you'd rather route
|
||||
replies to a shared team inbox — either works with this mechanism, but a Group needs its own
|
||||
delegation setup; a plain user mailbox is simpler and is what this runbook assumes).
|
||||
2. Consider giving it a strong, unique, never-reused password and enabling 2-Step Verification
|
||||
on it anyway, even though RootNode's application will never use that password — this account
|
||||
still exists as a real Workspace identity and should be secured like any other.
|
||||
|
||||
**If the sending address is an ALIAS of an existing account (a real cost-saving option — you
|
||||
don't need a whole extra paid seat just for this):** that's fine and supported, but note it for
|
||||
step 4 below. Domain-wide delegation authorizes the service account to act as a real Workspace
|
||||
**user account**, not an arbitrary alias string — if `[email protected]` is an alias of,
|
||||
say, `[email protected]`, the delegation in step 4 is granted for `chase@`'s account
|
||||
(the alias's owner), and RootNode's config keeps the alias as the visible sending address
|
||||
separately (see §5's `GOOGLE_DELEGATED_USER` vs. `EMAIL_FROM_ADDRESS`). Gmail already lets any
|
||||
account send as its own verified aliases, so this works cleanly — you just need both pieces of
|
||||
information (the alias, and which account owns it) when configuring RootNode.
|
||||
|
||||
## 3. REQUIRES GOOGLE WORKSPACE ADMIN ACTION: create the service account (Google Cloud Console)
|
||||
|
||||
1. In Google Cloud Console (console.cloud.google.com), create or choose a GCP project associated
|
||||
with your Workspace organization.
|
||||
2. Enable the **Gmail API** for that project (APIs & Services → Library → search "Gmail API" →
|
||||
Enable).
|
||||
3. IAM & Admin → Service Accounts → Create Service Account. Name it something identifiable
|
||||
(e.g. `rootnode-academy-mailer`). No project-level IAM roles are needed for this — it only
|
||||
ever calls the Gmail API as the impersonated user, not as itself.
|
||||
4. On that service account, create a JSON key (Keys → Add Key → Create new key → JSON). This
|
||||
downloads a file containing `client_email` and `private_key` — **this is the sensitive
|
||||
credential**, treat it exactly like a password (see §8).
|
||||
5. Copy the service account's **Client ID** (a long numeric string, on the service account's
|
||||
detail page) — needed for the next step.
|
||||
|
||||
## 4. REQUIRES GOOGLE WORKSPACE ADMIN ACTION: authorize domain-wide delegation
|
||||
|
||||
1. Google Admin Console → Security → Access and data control → API controls → Domain-wide
|
||||
Delegation → Add new.
|
||||
2. Client ID: the numeric ID from step 3.5 above.
|
||||
3. OAuth Scopes: `https://www.googleapis.com/auth/gmail.send` — **only this scope**. Do not add
|
||||
broader Gmail scopes (readonly, modify, etc.) — RootNode's adapter never needs them and
|
||||
granting them would violate least-privilege for no benefit.
|
||||
4. Authorize.
|
||||
|
||||
This grants the service account permission to act as **one specific Workspace user account**.
|
||||
If the sending address is that account's own primary login, that's the account. If the sending
|
||||
address is an ALIAS (§2), this authorization is inherently for the alias's owner account — there
|
||||
is no separate "authorize an alias" step, because delegation only ever targets real accounts.
|
||||
|
||||
## 5. RootNode-side environment variables
|
||||
|
||||
Set these on the Platform API's environment (production: real secret storage — see §8; local
|
||||
dev: `platform/.env`, already gitignored):
|
||||
|
||||
| Variable | Value | Notes |
|
||||
|---|---|---|
|
||||
| `EMAIL_PROVIDER` | `gws` | Unset/anything else defaults to the dev provider — never real mail. |
|
||||
| `GOOGLE_SERVICE_ACCOUNT_JSON` | the full JSON key file's contents, as one string | The sensitive credential — see §8. |
|
||||
| `EMAIL_FROM_ADDRESS` | the address students see mail arrive from, e.g. `[email protected]` | Always the visible `From:` header. |
|
||||
| `GOOGLE_DELEGATED_USER` | **only needed if `EMAIL_FROM_ADDRESS` is an alias** — the real account that alias belongs to, e.g. `[email protected]` | This is what domain-wide delegation actually authorized in step 4. If `EMAIL_FROM_ADDRESS` is itself a primary account, leave this unset — the code falls back to `EMAIL_FROM_ADDRESS` automatically. |
|
||||
| `EMAIL_FROM_NAME` | e.g. `RootNode Academy` | Display name only, not sensitive. |
|
||||
| `EMAIL_REPLY_TO` | optional, e.g. `[email protected]` | Where a student's reply actually goes, if different from the sending address. |
|
||||
| `PUBLIC_APP_ORIGIN` | e.g. `https://learn.rootnodeacademy.com` | The origin activation/reset links are built from (§7 of the Phase 5C authorization) — required for any token-bearing email to render a working link. |
|
||||
|
||||
**Concrete example matching the alias setup described above:** `[email protected]`
|
||||
(what students see), `[email protected]` (whose account actually
|
||||
owns that alias and was granted delegation in step 4).
|
||||
|
||||
## 6. DNS: SPF, DKIM, DMARC (REQUIRES DNS ADMIN ACTION — not performed from this repository)
|
||||
|
||||
Google Workspace's own documentation is authoritative here; this is a summary of what must
|
||||
exist before mail sent via the Gmail API from a Workspace mailbox reliably lands in inboxes
|
||||
instead of spam:
|
||||
|
||||
- **SPF**: your domain's TXT record must include Google's SPF include
|
||||
(`include:_spf.google.com`) if Workspace is your MX provider — check Admin Console → Apps →
|
||||
Google Workspace → Gmail → Authenticate email for the exact current record Google recommends
|
||||
for your domain.
|
||||
- **DKIM**: Admin Console → Apps → Google Workspace → Gmail → Authenticate email → generate a
|
||||
DKIM key for the domain, then publish the TXT record it gives you, then return to that page
|
||||
and turn DKIM signing ON. Gmail API sends from a Workspace mailbox are signed the same way
|
||||
regular Gmail sends are once this is on.
|
||||
- **DMARC**: publish a `_dmarc` TXT record (start with `p=none` to monitor before enforcing) once
|
||||
SPF and DKIM both pass consistently.
|
||||
|
||||
**Do not change DNS from this repository or these instructions** — this is exactly the kind of
|
||||
external, admin-console/DNS-provider action that stays outside the codebase. Verify current
|
||||
exact record values in Google's own Workspace admin help before publishing anything.
|
||||
|
||||
## 7. Testing procedure
|
||||
|
||||
**Local/dev — no live credentials needed at all.** `EMAIL_PROVIDER` unset (or anything other
|
||||
than `gws`) uses the deterministic dev provider (`email/dev-provider.js`) — every notification
|
||||
still goes through the full queue/worker/retry pipeline, but nothing ever reaches a real
|
||||
network. `platform/test/e2e-phase5c.js` and the Phase 5C drills run entirely this way.
|
||||
|
||||
**Live delivery — opt-in, one message, once credentials exist:**
|
||||
```bash
|
||||
RUN_GWS_EMAIL_TEST=true \
|
||||
TEST_EMAIL_RECIPIENT=you@example.com \
|
||||
GOOGLE_SERVICE_ACCOUNT_JSON="$(cat /path/to/service-account-key.json)" \
|
||||
EMAIL_FROM_ADDRESS=academy@rootnodeacademy.com \
|
||||
GOOGLE_DELEGATED_USER=chase@rootnodeacademy.com \
|
||||
EMAIL_FROM_NAME="RootNode Academy" \
|
||||
node infra/scripts/gws-email-test.js
|
||||
```
|
||||
(Omit `GOOGLE_DELEGATED_USER` entirely if `EMAIL_FROM_ADDRESS` is itself a primary account
|
||||
rather than an alias — see §5.) Sends exactly one clearly-marked test message to
|
||||
`TEST_EMAIL_RECIPIENT` — never inferred from production/student data, never run automatically as
|
||||
part of any regression suite. Without `RUN_GWS_EMAIL_TEST=true` it does nothing and exits
|
||||
cleanly.
|
||||
|
||||
## 8. Production secret handling
|
||||
|
||||
- `GOOGLE_SERVICE_ACCOUNT_JSON` is exactly as sensitive as a password — store it in whatever
|
||||
secret manager/environment-injection mechanism the production deployment uses (see
|
||||
`docs/PRODUCTION_DEPLOYMENT.md` for the general pattern this repo follows for other secrets
|
||||
like `PROVISIONER_SHARED_SECRET`). **Never commit it, never put it in a Docker image layer,
|
||||
never log it.** `gws-provider.js` reads it fresh from the environment on every send and never
|
||||
writes it to disk.
|
||||
- Nothing in this codebase ever returns `GOOGLE_SERVICE_ACCOUNT_JSON`'s contents, an access
|
||||
token, or any part of it through an API response, log line, or audit event.
|
||||
|
||||
## 9. Credential rotation
|
||||
|
||||
1. Google Cloud Console → the service account → Keys → Add Key → Create new key (JSON) — this
|
||||
creates an ADDITIONAL valid key without invalidating the old one yet.
|
||||
2. Update `GOOGLE_SERVICE_ACCOUNT_JSON` in production's secret store to the new key, redeploy/
|
||||
restart the Platform API.
|
||||
3. Confirm delivery still works (§7's live test, or just watch `/api/admin/notifications` for a
|
||||
real send after the restart).
|
||||
4. Google Cloud Console → the service account → Keys → delete the OLD key.
|
||||
|
||||
Domain-wide delegation authorization (§4) is tied to the service account's Client ID, not to any
|
||||
individual key — rotating keys does not require re-authorizing delegation.
|
||||
|
||||
## 10. Revocation (compromised credential / offboarding)
|
||||
|
||||
1. Google Cloud Console → the service account → Keys → delete the affected key immediately —
|
||||
this invalidates it instantly, before anything else.
|
||||
2. If the whole service account should stop being able to send at all: Google Admin Console →
|
||||
Security → API controls → Domain-wide Delegation → remove that Client ID's authorization.
|
||||
3. Rotate `EMAIL_FROM_ADDRESS`'s own Workspace login password as a precaution even though the
|
||||
application never used it — a compromised service-account key doesn't imply the mailbox's own
|
||||
credentials were exposed, but treat it as a "review this account" trigger.
|
||||
|
||||
## 11. Troubleshooting
|
||||
|
||||
| Symptom | Likely cause |
|
||||
|---|---|
|
||||
| Every send fails with `gws_not_configured` | `GOOGLE_SERVICE_ACCOUNT_JSON` or `EMAIL_FROM_ADDRESS` unset/malformed in the Platform API's environment. |
|
||||
| `gws_auth_failed` | Domain-wide delegation not authorized for this Client ID/scope, or the service account key was deleted/revoked. Re-check §4. **If this happens intermittently right after newly authorizing delegation** (the exact same request succeeds, then fails with `unauthorized_client`, then succeeds again, over roughly a minute) — that's Google's own auth infrastructure propagation delay, not a config error. It self-resolves; the notification worker's existing retry/backoff already absorbs it (`gws-provider.js` classifies `unauthorized_client` as retryable), so no action is needed beyond waiting. |
|
||||
| `gws_http_403` | The delegated `sub` (EMAIL_FROM_ADDRESS) isn't a real mailbox in this Workspace domain, or the Gmail API isn't enabled on the GCP project (§3.2). |
|
||||
| `gws_rate_limited` (HTTP 429) | Gmail API per-user sending limits hit — see §12. The notification worker retries this automatically with backoff. |
|
||||
| Mail sends successfully but lands in spam | SPF/DKIM/DMARC (§6) not fully configured/passing yet — this is a deliverability issue, not an application error. |
|
||||
|
||||
## 12. Google Workspace quotas/limits relevant to this mechanism
|
||||
|
||||
Gmail API sending inherits the same per-user daily sending limits as regular Gmail/Workspace
|
||||
sending (limits vary by Workspace edition — check current figures in Google's own Workspace
|
||||
admin documentation, as these change over time and are not something to hard-code here). For
|
||||
RootNode Academy's transactional volume (activation/enrollment/reset emails, not bulk/marketing
|
||||
— explicitly out of scope, see the Phase 5C authorization §5/§28), this should sit comfortably
|
||||
under any Workspace edition's per-user limit; revisit if enrollment volume grows substantially.
|
||||
@@ -0,0 +1,67 @@
|
||||
# Runbook: Self-Paced Programs & Enrollment
|
||||
|
||||
Covers managing programs and enrollments from `/admin` → Programs. See
|
||||
`docs/PHASE5B_SELF_PACED_ENROLLMENT.md` for the underlying design and
|
||||
`docs/WEBSITE_ENROLLMENT_INTEGRATION.md` for the separate website-driven path.
|
||||
|
||||
**Create a program** (Programs → Create program, admin/owner only):
|
||||
|
||||
Program ID is a slug (e.g. `root-linux-fundamentals`) and can't be changed later. Pick a
|
||||
family (`init_`/`compile_`/`root_`) and delivery mode. New programs default to
|
||||
`curriculum_path: rootnode-path` — the only curriculum content that exists today. A program for
|
||||
`init_`/`compile_` can be created now as a placeholder and wired to real content later without a
|
||||
schema change; it just won't have real lessons behind it yet.
|
||||
|
||||
**Activate/deactivate a program** (Programs → \[program\] → Activate/Deactivate):
|
||||
|
||||
An inactive program can't accept new enrollments (admin-manual or via the website integration
|
||||
endpoint) but existing enrollments in it are untouched — students already enrolled keep their
|
||||
access exactly as it was.
|
||||
|
||||
**Manually enroll a student** (Programs → \[program\] → Enroll a student):
|
||||
|
||||
Enter the student's ID (find it on the Students tab). Choose `pending` (the default — the
|
||||
student sees the enrollment but can't start until you Activate it) or `active` (skip that step).
|
||||
Optional start/end dates enforce access windows server-side — a future start date hides Lab
|
||||
access until then; an end date cuts off new Lab launches after it while leaving prior progress
|
||||
visible.
|
||||
|
||||
As of Phase 5C, this also automatically emails the student — an activation link if they've never
|
||||
had credentials, or an enrollment confirmation if they already do. See
|
||||
`docs/RUNBOOK_STUDENT_ACTIVATION.md` for checking delivery status and resending.
|
||||
|
||||
**Activate a pending enrollment** (Programs → \[program\] → Enrollments table → Activate):
|
||||
|
||||
Only needed if you enrolled as `pending`. This is the moment a student actually gains entry.
|
||||
|
||||
**Cancel an enrollment** (Programs → \[program\] → Enrollments table → Cancel, or Enrollments
|
||||
list on a student's own page):
|
||||
|
||||
Immediately blocks new Lab launches. Does not touch the student's progress/achievements/attempt
|
||||
history — those are permanent regardless of enrollment status. Cancelling is final for that
|
||||
enrollment; if the student should have access again later, enroll them again (a new enrollment
|
||||
row) rather than trying to "reactivate" a cancelled one.
|
||||
|
||||
**Check a student's enrollment history** (Students → \[student\] → "Enrolled programs" card):
|
||||
|
||||
Shows every program the student has ever been enrolled in — pending, active, completed,
|
||||
cancelled, expired — with progress and last activity. A plain instructor sees this scoped to
|
||||
enrollments they created or that belong to a student sharing one of their cohorts; owner/admin
|
||||
see everything.
|
||||
|
||||
**A student says "Continue Learning" isn't working / shows the wrong thing:**
|
||||
|
||||
1. Check their enrollment status on the Programs or Student page. `pending` → activate it.
|
||||
`expired`/`cancelled` → that's working as intended; enroll them again if they should continue.
|
||||
2. If `active` and still stuck, check `/api/admin/students/:id/environment` for a stuck Lab —
|
||||
same troubleshooting as any other Lab issue (`docs/RUNBOOK_STUCK_LAB_CLEANUP.md`). Destroying
|
||||
the Lab never loses their program progress; a fresh launch resumes from the same place.
|
||||
|
||||
**A student's completion didn't register:**
|
||||
|
||||
Completion is detected the next time their program status is *read* (their dashboard, or an
|
||||
admin viewing their enrollment) after they finish the capstone — not pushed instantly. Refresh
|
||||
the page or reopen their student detail in `/admin`; if it still doesn't show `completed` after
|
||||
that, check `GET /api/achievements` for that student for a `capstone_completed` record matching
|
||||
the program's configured capstone — if that's missing, the capstone itself wasn't actually
|
||||
finished, not an enrollment bug.
|
||||
@@ -1,15 +1,20 @@
|
||||
# Runbook: Student Account Lifecycle
|
||||
|
||||
Covers creating, converting, resetting, and revoking student accounts. See
|
||||
`docs/PHASE5A_STUDENT_IDENTITY_REMOTE_CLASSROOM.md` for the underlying design.
|
||||
`docs/PHASE5A_STUDENT_IDENTITY_REMOTE_CLASSROOM.md` for the underlying design and
|
||||
`docs/PHASE5C_ENROLLMENT_COMMUNICATIONS.md`/`docs/RUNBOOK_STUDENT_ACTIVATION.md` for the
|
||||
automatic-email side of this, added in Phase 5C.
|
||||
|
||||
**Create a new credentialed student** (`/admin` → Students → Add student):
|
||||
|
||||
Fill in display name plus **either** email (older/self-paced) or login ID (younger/instructor-
|
||||
managed) — leave both blank for a plain pilot account instead. Submitting shows an activation
|
||||
link **once**: `https://<domain>/?activate=<token>`. Copy it immediately (print it, read it
|
||||
aloud, hand it over) — it is never shown again and never stored in plaintext anywhere. The
|
||||
student opens the link and sets their own password; the instructor never sees it.
|
||||
link **once**: `https://<domain>/?activate=<token>`. As of Phase 5C, if the student has an
|
||||
email, RootNode also sends it to them automatically — the on-screen link remains a manual-relay
|
||||
fallback, useful when there's no deliverable address or the automatic email needs backup. Copy
|
||||
it if you might need it (print it, read it aloud, hand it over) — it is never shown again and
|
||||
never stored in plaintext anywhere. The student opens the link and sets their own password; the
|
||||
instructor never sees it.
|
||||
|
||||
**Convert an existing pilot student to a real account** (Students → \[student\] → Convert):
|
||||
|
||||
@@ -40,5 +45,8 @@ students list — deactivation takes effect on their very next request, not just
|
||||
login, because `studentSession` middleware re-checks `status='active'` every time.
|
||||
|
||||
**If a student says their activation/reset link "doesn't work":** tokens are single-use and
|
||||
time-limited (activation 7 days, reset 1 hour). Just issue a new one via Convert or Reset
|
||||
password — there's no way to "resend" the same token, and that's intentional.
|
||||
time-limited (activation 7 days, reset 1 hour). Use Students → \[student\] → **Resend
|
||||
activation** (or Reset password, for an already-activated account) to issue a new one — there's
|
||||
no way to "resend" the same token, and that's intentional (only the most recent link is ever
|
||||
valid, so there's never ambiguity about which one to use). See `docs/RUNBOOK_STUDENT_
|
||||
ACTIVATION.md` for checking whether the automatic email actually delivered.
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
# Runbook: Student Activation Pipeline
|
||||
|
||||
Covers the day-to-day operation of the enrollment → activation → login pipeline from `/admin`.
|
||||
See `docs/PHASE5C_ENROLLMENT_COMMUNICATIONS.md` for the underlying design and
|
||||
`docs/RUNBOOK_GOOGLE_WORKSPACE_EMAIL.md` for the email delivery mechanism itself.
|
||||
|
||||
## What happens automatically
|
||||
|
||||
Every time a student account gains a new enrollment — whether the website's checkout triggered
|
||||
it (`POST /api/integrations/enrollments`) or an instructor enrolled them manually from
|
||||
`/admin` → Programs — RootNode decides and sends the right email on its own:
|
||||
|
||||
- **Never activated before** → an activation email, with a one-time link to set their password.
|
||||
- **Already has real credentials** → an enrollment confirmation email pointing them back to
|
||||
sign in — no link/token involved.
|
||||
|
||||
Nothing needs to be triggered by hand for the normal case. The controls below are for when the
|
||||
normal case didn't work.
|
||||
|
||||
## Checking delivery status
|
||||
|
||||
**Per student**: Students → \[student\] → the "Notifications" card shows every notification ever
|
||||
sent to (or attempted for) that student, with type/status/attempt count. `sent` means it
|
||||
actually reached the provider successfully; `failed` means it gave up (see below); `queued`/
|
||||
`sending` means it's still in flight — check back in a few seconds, delivery happens through a
|
||||
background worker, not the same request that triggered it.
|
||||
|
||||
**Everywhere**: `/admin` → Notifications (owner/admin only) — the same information across every
|
||||
student, most recent first.
|
||||
|
||||
## If a student never got their activation email
|
||||
|
||||
1. Check Students → \[student\] → Notifications. If it shows `failed`, note the error — a
|
||||
`no_recipient_address` failure means the student truly has no email or contact address on
|
||||
file and can only be activated by an instructor manually relaying a link (see below); other
|
||||
error codes usually mean a delivery-infrastructure issue worth checking
|
||||
`docs/RUNBOOK_GOOGLE_WORKSPACE_EMAIL.md`'s troubleshooting table for.
|
||||
2. Students → \[student\] → Credentials → **Resend activation**. This invalidates whatever link
|
||||
was outstanding and sends a fresh one — safe to click as many times as needed; only the most
|
||||
recent link is ever valid, so there's never a "which link do I use" confusion for the student.
|
||||
3. If the student genuinely has no deliverable address at all (a younger student with a
|
||||
login-only account and no guardian contact email on file), convert/create them without email
|
||||
in the first place (see `docs/RUNBOOK_STUDENT_ACCOUNT_LIFECYCLE.md`) and relay the one-time
|
||||
activation link shown in the admin UI directly (print it, read it aloud) — this manual path
|
||||
still works exactly as it did before Phase 5C added automatic email.
|
||||
|
||||
## If a student says they didn't get an enrollment confirmation
|
||||
|
||||
Check Students → \[student\] → Notifications for the `enrollment_confirmation` row. If it's
|
||||
`sent`, the email genuinely went out — check their spam folder, or confirm the address on file
|
||||
(Students → \[student\]) is actually correct. If none exists for a given enrollment,
|
||||
Programs → \[program\] → Enrollments → find the enrollment → (student detail page also works)
|
||||
and use **Send enrollment confirmation** to trigger one explicitly.
|
||||
|
||||
## Password reset
|
||||
|
||||
Students should use "Forgot password" on the login screen themselves wherever possible — it's
|
||||
delivered the same way as everything else here and the flow is self-service end to end.
|
||||
|
||||
If a student can't complete self-service reset (e.g. they no longer have access to the email a
|
||||
guardian originally provided and it needs an instructor to re-trigger with a corrected address),
|
||||
Students → \[student\] → Credentials → **Send password reset** issues a fresh reset link and
|
||||
revokes all of that student's existing sessions immediately. In production, the instructor
|
||||
never sees the token itself — the response is a confirmation that an email was queued, not the
|
||||
link. If automatic delivery still can't reach the student, use `docs/RUNBOOK_STUDENT_ACCOUNT_
|
||||
LIFECYCLE.md`'s manual-relay procedure, which is unaffected by any of this.
|
||||
|
||||
## What "resend" does NOT do
|
||||
|
||||
Resending activation does not touch the student's enrollment status, progress, achievements, or
|
||||
attempt history — it only affects the credential/token side of their account. It's always safe.
|
||||
@@ -0,0 +1,220 @@
|
||||
# Website → RootNode Academy Enrollment Integration
|
||||
|
||||
Audience: whoever builds `rootnodeacademy.com` (marketing, pricing, checkout) and needs to turn
|
||||
a successful purchase/signup into a real RootNode Academy student enrollment. This document is
|
||||
meant to be enough on its own — you should not need to read the Platform API's source to
|
||||
integrate against it.
|
||||
|
||||
## Responsibility boundary
|
||||
|
||||
**The website is responsible for:** marketing, program selection, pricing, checkout/payment (now
|
||||
or in the future), collecting enrollment/customer information, and sending ONE verified
|
||||
enrollment event to the endpoint below once payment/signup is confirmed.
|
||||
|
||||
**RootNode's platform is responsible for everything after that:** student identity, program
|
||||
entitlement, account activation, the entire password lifecycle, all operational enrollment
|
||||
email (activation, enrollment confirmation, password reset — see
|
||||
`docs/PHASE5C_ENROLLMENT_COMMUNICATIONS.md`), learning, Labs, progress, assessment, and
|
||||
completion. **The website does not send email for platform activation or any other account
|
||||
lifecycle event — RootNode owns that entirely, as of Phase 5C.** Do not build a parallel
|
||||
"welcome" or "here's how to log in" email on the website side; it would either duplicate or
|
||||
conflict with what RootNode already sends automatically.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
rootnodeacademy.com (marketing, pricing, checkout — built separately, not in this repo)
|
||||
|
|
||||
| your backend verifies payment/signup with your payment provider
|
||||
v
|
||||
your backend calls the RootNode Enrollment API (below) — server-to-server only
|
||||
v
|
||||
RootNode Academy Platform API
|
||||
creates/matches a student account
|
||||
creates or reuses (idempotently) an enrollment
|
||||
v
|
||||
the student logs in at the RootNode Academy platform and sees their program
|
||||
```
|
||||
|
||||
**RootNode never talks to your payment provider, and your payment provider never talks to
|
||||
RootNode.** Your backend is the only thing that calls this endpoint, after it has already
|
||||
confirmed payment (or whatever your enrollment trigger is) on its own. This endpoint knows
|
||||
nothing about Stripe, PayPal, or any other provider's object model — it only understands a
|
||||
small, generic enrollment event described below.
|
||||
|
||||
## The endpoint
|
||||
|
||||
```
|
||||
POST /api/integrations/enrollments
|
||||
```
|
||||
|
||||
Reachable at the RootNode Academy platform's public origin (the same host your students'
|
||||
browsers already use), via HTTPS in any real deployment. This is the **only** endpoint your
|
||||
backend needs.
|
||||
|
||||
## Authentication
|
||||
|
||||
A pre-shared secret, sent as a header:
|
||||
|
||||
```
|
||||
X-RootNode-Integration-Key: <the secret>
|
||||
```
|
||||
|
||||
- This is **not** a student or admin login. It has nothing to do with cookies, sessions, or
|
||||
browser auth of any kind.
|
||||
- **The secret must live only in your backend's server-side configuration.** It must never be
|
||||
embedded in any browser-served JavaScript, mobile app bundle, or client-side config — anyone
|
||||
who obtains it can create enrollments.
|
||||
- A request with a missing or wrong key gets `401 { "error": "unauthorized" }` and nothing else
|
||||
happens — no student is looked up, nothing is created.
|
||||
- RootNode will provide you with the production secret out of band (not over email/chat in
|
||||
plaintext — ask the RootNode team for the current secure-sharing process). Rotate it by
|
||||
asking RootNode to issue a new one; there is no self-service rotation yet.
|
||||
|
||||
## Request
|
||||
|
||||
```json
|
||||
{
|
||||
"event_id": "order_9f8e7d6c5b4a",
|
||||
"source": "website",
|
||||
"program_slug": "root-kubernetes-foundations",
|
||||
"status": "paid",
|
||||
"student": {
|
||||
"email": "jamie@example.com",
|
||||
"login_identifier": null,
|
||||
"display_name": "Jamie Rivera",
|
||||
"contact_email": null
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Required | Notes |
|
||||
|---|---|---|
|
||||
| `event_id` | **yes** | A string, unique per real-world event, from *your* system (an order ID, a payment intent ID — anything stable). This is the idempotency key — see below. Not a RootNode-generated value. |
|
||||
| `source` | no | Free text, e.g. `"website"`. Recorded for audit only, defaults to `"integration"` if omitted. |
|
||||
| `program_slug` | **yes** | The RootNode program's id, e.g. `"root-kubernetes-foundations"`. Ask RootNode for the current list — there is no public program-catalog endpoint. |
|
||||
| `status` | **yes** | One of: `"paid"`, `"active"`, `"confirmed"` (all mean "grant access now") or `"pending"`, `"awaiting_payment"` (means "record the enrollment but don't grant access yet"). This is a small, fixed, provider-neutral vocabulary — translate whatever your payment provider says into one of these five values before calling. Anything else is rejected. |
|
||||
| `student.email` | one of email/login_identifier required | The student's own email, if they have one. |
|
||||
| `student.login_identifier` | one of email/login_identifier required | Use this instead of/alongside email for a student who doesn't have their own email address (a minor enrolled by a parent/guardian — see below). Any stable, unique string you control — e.g. an internal user ID from your own system works well. |
|
||||
| `student.display_name` | no | Shown in the RootNode instructor portal and, once the student logs in, to themselves. Falls back to the email or login_identifier if omitted. |
|
||||
| `student.contact_email` | no | **Only** for a student with no email of their own (§ below) — where to send THIS enrollment's activation/confirmation email (typically a guardian's address collected at checkout). Never treated as the student's own identity/login. |
|
||||
|
||||
### What you must NOT send
|
||||
|
||||
- No card numbers, CVVs, bank details, or any other payment-card data. This endpoint has no
|
||||
field for it and never will — RootNode never touches payment-card data at all.
|
||||
- No raw provider webhook payloads (Stripe event objects, PayPal IPN bodies, etc.) — translate
|
||||
to the shape above first. This endpoint is deliberately provider-neutral.
|
||||
- No password. RootNode issues its own credential/activation flow (see "What the student
|
||||
experiences" below) — you cannot set a student's password through this endpoint, and should
|
||||
never try to invent one.
|
||||
|
||||
## Response
|
||||
|
||||
**Success — a new enrollment was created** (`201`):
|
||||
```json
|
||||
{ "enrollment_id": "6dd6d898-f895-477f-97d8-1e5ce89c27cd", "student_id": "student-042", "program_id": "root-kubernetes-foundations", "status": "active", "created": true }
|
||||
```
|
||||
|
||||
**Success — this exact `event_id` was already processed** (`200`, idempotent replay — see below):
|
||||
```json
|
||||
{ "enrollment_id": "6dd6d898-f895-477f-97d8-1e5ce89c27cd", "student_id": "student-042", "program_id": "root-kubernetes-foundations", "status": "active", "created": false }
|
||||
```
|
||||
|
||||
**Errors:**
|
||||
|
||||
| Status | Body `error` | Meaning |
|
||||
|---|---|---|
|
||||
| `401` | `unauthorized` | Missing/wrong `X-RootNode-Integration-Key`. |
|
||||
| `400` | `event_id is required` | — |
|
||||
| `400` | `program_slug is required` | — |
|
||||
| `400` | `status must be one of: paid, active, confirmed, pending, awaiting_payment` | — |
|
||||
| `400` | `student.email or student.login_identifier is required` | — |
|
||||
| `404` | `PROGRAM_NOT_FOUND` | `program_slug` doesn't exist. |
|
||||
| `409` | `PROGRAM_INACTIVE` | The program exists but isn't currently open for enrollment. |
|
||||
| `500` | `internal_error` | Something unexpected broke server-side. Safe to retry (see below). |
|
||||
|
||||
## Idempotency and retries
|
||||
|
||||
**`event_id` is the whole mechanism.** Send the exact same `event_id` again (your payment
|
||||
provider retried a webhook, your own backend retried a timed-out request, anything) and you get
|
||||
back the **same enrollment**, `created: false`, `200` instead of `201` — never a duplicate. This
|
||||
is enforced at the database level (a unique constraint), not just "best effort," so it's safe
|
||||
under concurrent/parallel retries too, not just sequential ones.
|
||||
|
||||
**Retry policy:** on a `5xx` or a network error/timeout, retry with the *same* `event_id`
|
||||
(standard exponential backoff is fine — a few attempts over a couple of minutes is plenty). On a
|
||||
`4xx`, do not blindly retry — fix the request (see the error table above) or, for `401`, check
|
||||
your key.
|
||||
|
||||
**Never generate a new `event_id` for a retry of the same real-world event.** Doing so would
|
||||
create a second enrollment. Reserve a fresh `event_id` only for a genuinely new enrollment event
|
||||
(e.g. a renewal, a second purchase).
|
||||
|
||||
## Student matching rules
|
||||
|
||||
RootNode looks for an existing student by `email` (case-insensitive) or `login_identifier`
|
||||
(exact match), in that priority. If found, that student is enrolled. If not found, a new student
|
||||
account is created automatically — this is a trusted server-to-server call from your backend,
|
||||
equivalent to an instructor adding a student through the admin portal, not a public signup form.
|
||||
|
||||
## Minor / no-email students
|
||||
|
||||
RootNode explicitly supports students without their own email address (younger students,
|
||||
enrolled by a parent/guardian). Send `student.login_identifier` instead of (or in addition to)
|
||||
`email` — any stable string you control, such as an internal account/order reference. You must
|
||||
send **at least one** of `email` or `login_identifier`; sending neither is rejected.
|
||||
|
||||
If the enrolled student has no email at all, send `student.contact_email` too (e.g. the
|
||||
guardian's address collected at checkout) so RootNode has somewhere to deliver the activation
|
||||
email — otherwise the enrollment is created but nobody is notified automatically, and an
|
||||
instructor has to relay the activation link manually. `contact_email` is stored only against
|
||||
that specific enrollment, never copied into the student's own identity record.
|
||||
|
||||
There is no guardian/parent account system yet — the enrollment is created directly against the
|
||||
student identity you provide. If you need to support "a parent manages multiple children's
|
||||
enrollments" as a first-class concept later, talk to the RootNode team before building around a
|
||||
workaround; the schema has room for it but it isn't implemented.
|
||||
|
||||
## What the student experiences after you call this (as of Phase 5C)
|
||||
|
||||
If `status` mapped to "active": RootNode sends the student (or their enrollment's
|
||||
`contact_email`) a real email automatically —an activation email with a one-time setup link if
|
||||
they're new, or an enrollment confirmation if they already have a RootNode account — and they
|
||||
can log in as soon as they act on it. **You do not need to send any email yourself for this** —
|
||||
see "Responsibility boundary" above. If `status` mapped to "pending": the enrollment exists but
|
||||
the student can't start until a RootNode instructor activates it — useful if your flow needs a
|
||||
manual review step before access is granted; no email goes out until activation happens.
|
||||
|
||||
If the student has neither an `email` nor a `contact_email` on file, RootNode still creates the
|
||||
account and enrollment, but nothing gets emailed automatically — an instructor has to relay the
|
||||
activation link by hand. Send whichever address you have whenever you have one.
|
||||
|
||||
## Local/staging testing
|
||||
|
||||
Set `INTEGRATION_SHARED_SECRET` in your test environment's config (in local dev,
|
||||
`platform/.env`; example dev value used by this repo's own test suite:
|
||||
`dev-integration-secret-not-for-production` — never use this value anywhere real). Then:
|
||||
|
||||
```bash
|
||||
curl -s -X POST http://localhost:4200/api/integrations/enrollments \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H 'X-RootNode-Integration-Key: dev-integration-secret-not-for-production' \
|
||||
-d '{
|
||||
"event_id": "test-order-001",
|
||||
"source": "website",
|
||||
"program_slug": "root-kubernetes-foundations",
|
||||
"status": "paid",
|
||||
"student": { "email": "test-student@example.com", "display_name": "Test Student" }
|
||||
}'
|
||||
```
|
||||
|
||||
All example data on this page is fake — do not reuse these values against a real deployment.
|
||||
|
||||
## Production secret handling
|
||||
|
||||
- The production `X-RootNode-Integration-Key` value must be requested from the RootNode team
|
||||
and stored in your backend's secret manager/environment configuration — never in source
|
||||
control, never in a browser-reachable config, never logged.
|
||||
- RootNode's own logs and audit trail never record the secret's value, only that an
|
||||
authenticated call happened (or a `401` if it didn't).
|
||||
@@ -9,6 +9,9 @@ import AccountView from './AccountView'
|
||||
import LearningView from './LearningView'
|
||||
import AssessmentsView from './AssessmentsView'
|
||||
import ClassroomView from './ClassroomView'
|
||||
import ProgramsView from './ProgramsView'
|
||||
import NotificationsView from './NotificationsView'
|
||||
import CurriculumView from './CurriculumView'
|
||||
import styles from './Admin.module.css'
|
||||
|
||||
// Instructor/Admin portal — clearly separate from the student experience (/admin route,
|
||||
@@ -21,9 +24,12 @@ const NAV = [
|
||||
{ key: 'dashboard', label: 'Dashboard' },
|
||||
{ key: 'cohorts', label: 'Cohorts' },
|
||||
{ key: 'students', label: 'Students' },
|
||||
{ key: 'programs', label: 'Programs' },
|
||||
{ key: 'curriculum', label: 'Curriculum' },
|
||||
{ key: 'classroom', label: 'Classroom' },
|
||||
{ key: 'learning', label: 'Learning' },
|
||||
{ key: 'assessments', label: 'Assessments' },
|
||||
{ key: 'notifications', label: 'Notifications', roles: ['owner', 'admin'] },
|
||||
{ key: 'instructors', label: 'Instructors', roles: ['owner'] },
|
||||
{ key: 'audit', label: 'Audit', roles: ['owner', 'admin'] },
|
||||
]
|
||||
@@ -61,9 +67,12 @@ export default function AdminPortal() {
|
||||
{view === 'dashboard' && <AdminDashboard />}
|
||||
{view === 'cohorts' && <CohortsView role={instructor.role} />}
|
||||
{view === 'students' && <StudentsView role={instructor.role} />}
|
||||
{view === 'programs' && <ProgramsView role={instructor.role} />}
|
||||
{view === 'curriculum' && <CurriculumView />}
|
||||
{view === 'classroom' && <ClassroomView />}
|
||||
{view === 'learning' && <LearningView />}
|
||||
{view === 'assessments' && <AssessmentsView role={instructor.role} />}
|
||||
{view === 'notifications' && ['owner', 'admin'].includes(instructor.role) && <NotificationsView />}
|
||||
{view === 'instructors' && instructor.role === 'owner' && <InstructorsView currentInstructorId={instructor.instructor_id} />}
|
||||
{view === 'audit' && ['owner', 'admin'].includes(instructor.role) && <AuditView />}
|
||||
{view === 'account' && <AccountView instructor={instructor} />}
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user