Compare commits
49
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b6516a74db | ||
|
|
f5379afd5c | ||
|
|
15b616274b | ||
|
|
87526bcf7f | ||
|
|
3c815cfeea | ||
|
|
bd0a10fbfb | ||
|
|
01d3764138 | ||
|
|
903b0ead43 | ||
|
|
7f6e1857c0 | ||
|
|
2db136875f | ||
|
|
505783e5f7 | ||
|
|
e0f954e714 | ||
|
|
54b35e8ad7 | ||
|
|
de6c5b3286 | ||
|
|
831486d664 | ||
|
|
960a638ec9 | ||
|
|
73cea1ba1d | ||
|
|
dbcc210889 | ||
|
|
86ea035e9b | ||
|
|
25409d3bd7 | ||
|
|
046a0fba18 | ||
|
|
bee57d89fd | ||
|
|
82f17a05e8 | ||
|
|
fd38d06055 | ||
|
|
8ffc923a84 | ||
|
|
db927b8081 | ||
|
|
8c09ef8f3e | ||
|
|
28c925069b | ||
|
|
e2298e95de | ||
|
|
fe702754cd | ||
|
|
f152b9a489 | ||
|
|
e4cbcf6c40 | ||
|
|
4c0a950afb | ||
|
|
7baf7074e5 | ||
|
|
777efd168d | ||
|
|
b38c46aaec | ||
|
|
be92be81cd | ||
|
|
f25535027c | ||
|
|
3c7f0ca59c | ||
|
|
9d382c95c5 | ||
|
|
5183a6af09 | ||
|
|
1c08a30c7f | ||
|
|
23f881884d | ||
|
|
f89a769540 | ||
|
|
4bc202e5f3 | ||
|
|
1b6da0a966 | ||
|
|
17160da89e | ||
|
|
611e1ac4ba | ||
|
|
895e16fea4 |
+19
@@ -42,6 +42,25 @@ RUN set -eux && \
|
||||
ln -sf /usr/local/bin/k3s /usr/local/bin/kubectl && \
|
||||
ln -sf /usr/local/bin/k3s /usr/local/bin/crictl
|
||||
|
||||
# ── Docker Engine (ROOT 04) + DNS tooling (ROOT 02) ──────────────────────────
|
||||
# A second, independent container runtime nested inside this already-Privileged Lab container
|
||||
# (see docs/ROOT_FOUNDATIONS_LAB_STRATEGY.md §3 for the security/resource evaluation and the
|
||||
# real test results this decision is based on) — its own dockerd, its own
|
||||
# /var/run/docker.sock, entirely separate from the Lab Host's real Docker socket, which is
|
||||
# never mounted into this image. docker-compose v2 isn't bundled by Ubuntu's docker.io package,
|
||||
# so fetch the static binary the same way k3s's own binary is fetched above, rather than adding
|
||||
# a whole new apt repository for one plugin.
|
||||
RUN apt-get update && apt-get install -y docker.io dnsutils && rm -rf /var/lib/apt/lists/*
|
||||
RUN set -eux && \
|
||||
case "$(uname -m)" in \
|
||||
x86_64) COMPOSE_ARCH="x86_64" ;; \
|
||||
aarch64) COMPOSE_ARCH="aarch64" ;; \
|
||||
*) echo "Unsupported arch: $(uname -m)" && exit 1 ;; \
|
||||
esac && \
|
||||
mkdir -p /usr/local/lib/docker/cli-plugins && \
|
||||
curl -fsSL "https://github.com/docker/compose/releases/latest/download/docker-compose-linux-${COMPOSE_ARCH}" \
|
||||
-o /usr/local/lib/docker/cli-plugins/docker-compose && \
|
||||
chmod +x /usr/local/lib/docker/cli-plugins/docker-compose
|
||||
|
||||
# ── App files ─────────────────────────────────────────────────────────────────
|
||||
WORKDIR /app
|
||||
|
||||
@@ -163,6 +163,19 @@ its core:
|
||||
classes from `/admin` (presence, Lab/learning status, help-request queue, announcements,
|
||||
guidance-only current-activity) — RootNode owns identity/coordination, an external service
|
||||
(Meet/Zoom/Teams/etc.) owns the actual video/voice.
|
||||
- Students can now be enrolled in a self-paced program (as of Phase 5B) — "My Programs" on the
|
||||
student dashboard, entitlement enforced server-side (pending/future/expired/cancelled
|
||||
enrollments cannot launch a Lab no matter what the browser sends), completion server-derived
|
||||
from the existing capstone rules. `/admin` gained a Programs area (program authoring, manual
|
||||
enrollment, activate/cancel). A separate, provider-neutral `POST /api/integrations/enrollments`
|
||||
boundary — server-to-server only, no browser session involved — is what the future public
|
||||
`rootnodeacademy.com` website/checkout will call to create these enrollments; see
|
||||
`docs/PHASE5B_SELF_PACED_ENROLLMENT.md` and `docs/WEBSITE_ENROLLMENT_INTEGRATION.md`.
|
||||
- As of Phase 5C, RootNode actually tells students what's happening — real activation,
|
||||
enrollment-confirmation, and password-reset email, delivered asynchronously through a
|
||||
provider-neutral notification worker (Google Workspace via the Gmail API in production, a
|
||||
deterministic dev provider everywhere else — no test/dev run ever emails a real person). See
|
||||
`docs/PHASE5C_ENROLLMENT_COMMUNICATIONS.md` and `docs/RUNBOOK_GOOGLE_WORKSPACE_EMAIL.md`.
|
||||
|
||||
**Never publish the raw privileged container** — front it with isolation (that's what the
|
||||
Gateway is). The one-container `docker run --privileged -p 7554:80 ...` command above stays
|
||||
@@ -200,7 +213,8 @@ Key properties of the platform (not this image):
|
||||
Production topology and what's built vs. not yet: `docs/LAB_ARCHITECTURE.md`,
|
||||
`docs/PHASE1_PLATFORM_FOUNDATION.md`, `docs/PHASE2A_GATEWAY_AND_STUDENT_FLOW.md`,
|
||||
`docs/PHASE3A_IDENTITY_AND_INSTRUCTOR_ACCESS.md`, `docs/PHASE3B_LEARNING_AND_ASSESSMENT.md`,
|
||||
`docs/PHASE5A_STUDENT_IDENTITY_REMOTE_CLASSROOM.md`.
|
||||
`docs/PHASE5A_STUDENT_IDENTITY_REMOTE_CLASSROOM.md`, `docs/PHASE5B_SELF_PACED_ENROLLMENT.md`,
|
||||
`docs/PHASE5C_ENROLLMENT_COMMUNICATIONS.md`.
|
||||
|
||||
The production infrastructure package (Proxmox/Ansible/Terraform/firewall/monitoring/backup/CI)
|
||||
that runs all of this on real hardware lives in `infra/` — see
|
||||
|
||||
+66
@@ -0,0 +1,66 @@
|
||||
# How Computers Run Software
|
||||
|
||||
## Hardware, OS, and application: three different layers
|
||||
|
||||
When you run a program, three distinct layers cooperate:
|
||||
|
||||
- **Hardware** — the physical CPU, RAM, and storage. It has no idea what a "file" or a "process"
|
||||
is; it just executes instructions and stores bits.
|
||||
- **The operating system (kernel)** — the layer that turns raw hardware into something usable:
|
||||
it decides which program gets the CPU next, hands out memory, and organizes storage into
|
||||
files and directories. Linux is a kernel plus a large set of standard tools around it.
|
||||
- **Applications (userspace)** — the programs you actually run. They ask the kernel for
|
||||
resources (memory, files, network) through a well-defined interface (system calls) instead of
|
||||
touching hardware directly.
|
||||
|
||||
This module gives you just enough of this model to reason about what you're actually looking at
|
||||
for the rest of ROOT_ — not a full computer architecture course.
|
||||
|
||||
## CPU: the thing that actually executes instructions
|
||||
|
||||
A CPU has some number of **cores** — each core can run one stream of instructions at a time.
|
||||
`nproc` tells you how many cores/threads Linux believes it has available. When a system feels
|
||||
"slow," it's very often because more processes want CPU time than there are cores to give them.
|
||||
|
||||
## RAM: fast, temporary, and finite
|
||||
|
||||
RAM holds the data and code a running program is actively using. It's fast but **volatile** —
|
||||
its contents disappear when power is lost, which is exactly why persistent data lives on
|
||||
storage instead. `free -h` shows you total memory, how much is in use, and how much is free.
|
||||
|
||||
## Storage: slower, but persistent
|
||||
|
||||
Storage (an SSD/disk) keeps data even when the machine is off. It's organized into a
|
||||
**filesystem** — the tree of directories and files you already have some intuition for. `df -h`
|
||||
shows how much space exists and how much is used on each mounted filesystem.
|
||||
|
||||
## Processes: a running program
|
||||
|
||||
A **process** is a program actually being executed — code plus its own private memory, tracked
|
||||
by the kernel with a unique ID (a PID). One program can have many processes running at once
|
||||
(open two terminals running the same shell, and you have two separate processes). ROOT 01
|
||||
Module 5 goes much deeper into this; for now, just know: everything you run becomes a process
|
||||
the kernel is responsible for scheduling onto a CPU core.
|
||||
|
||||
## Why this matters for everything after this module
|
||||
|
||||
Every concept the rest of ROOT_ builds on is a refinement of this same idea. A container is
|
||||
(mostly) an isolated process with its own filesystem view. A Kubernetes Pod is a group of
|
||||
containers the scheduler places onto a specific machine's CPU/RAM. None of that will make sense
|
||||
as anything other than magic if you don't first have this base model of what a computer is
|
||||
actually managing.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Lesson goal:** Give students just enough of a mental model to stop treating "the computer" as
|
||||
an opaque box — without turning this into a full CS/architecture course.
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live with discussion.
|
||||
|
||||
**Common misconception to watch for:** students often conflate RAM and storage ("my computer
|
||||
has 8GB" meaning disk space). Draw the volatile-vs-persistent distinction explicitly and early.
|
||||
|
||||
**Discussion question:** "Why can't a program just talk to the hardware directly instead of
|
||||
going through the kernel?" (Answer: the kernel is what lets many programs safely share one CPU/
|
||||
one block of RAM without stepping on each other — this is worth naming explicitly, as it's the
|
||||
root of why containers and Kubernetes are even possible.)
|
||||
+35
@@ -0,0 +1,35 @@
|
||||
# Guided Exercise: Predict Before You Look
|
||||
|
||||
Before you run the real inspection commands in the Lab, commit to a guess — this is what makes
|
||||
the real numbers meaningful instead of just scrolling past them.
|
||||
|
||||
## Your task
|
||||
|
||||
On paper or in a scratch file, write down your best GUESS for:
|
||||
|
||||
1. How many CPU cores does this machine have?
|
||||
2. How much total RAM, roughly, in GB?
|
||||
3. How much free disk space is on the root filesystem?
|
||||
|
||||
Then think through: which of these three would you expect a "student Lab" environment to be
|
||||
MOST constrained on, and why? (Hint: think about what happens if 50 students are each running
|
||||
their own Lab on one shared physical machine at the same time.)
|
||||
|
||||
## A worked shape to check yourself against
|
||||
|
||||
There's no single right numeric answer here — the point is the habit of predicting first. A
|
||||
reasonable answer to the "most constrained" question: memory and CPU are typically the tightest
|
||||
shared resource on a multi-tenant host, since every additional Lab needs its own slice of both,
|
||||
while disk is comparatively cheap to over-provision.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Have students commit to a number verbally or on a shared doc BEFORE anyone
|
||||
runs the real commands — the value is in noticing the gap between guess and reality, not in
|
||||
getting it right.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
|
||||
**Optional extension:** ask what would happen to their guess if this were a "real" laptop
|
||||
they personally own vs. a shared cloud server — good primer for later resource-limit
|
||||
discussions (ROOT 06's resource requests/limits, ROOT 09's resource governance).
|
||||
+27
@@ -0,0 +1,27 @@
|
||||
# Reflection: You Just Looked Inside the Box
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. How close was your guess to the real numbers? What surprised you most?
|
||||
2. Name one thing you now understand about "the computer" that you couldn't have explained
|
||||
before this module.
|
||||
3. When you eventually run `kubectl describe pod` and see `requests`/`limits` for CPU and
|
||||
memory, what do you now expect those numbers to actually be constraining?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**Every resource limit you'll ever configure — in Docker, in Kubernetes, in a systemd unit — is
|
||||
a promise about exactly the three things this module named: CPU, memory, and storage.** You
|
||||
can't reason well about "why did my container get killed" or "why is my Pod stuck Pending"
|
||||
without this base model. This module isn't computer-science trivia — it's the vocabulary the
|
||||
rest of ROOT_ assumes you have.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 3 is worth spending real time on — it's
|
||||
the first moment students connect this abstract lesson to something they'll type as a real
|
||||
command many courses from now. Naming that connection explicitly here pays off later.
|
||||
|
||||
**Approximate timing:** 5-8 minutes live discussion; 5 minutes self-paced writing.
|
||||
+58
@@ -0,0 +1,58 @@
|
||||
# Linux and the Shell
|
||||
|
||||
## What "Linux" actually means
|
||||
|
||||
Linux is a **kernel** — the core piece of software from Module 1 that manages CPU, memory, and
|
||||
storage. What most people call "a Linux system" (Ubuntu, Debian, Fedora, Alpine...) is a
|
||||
**distribution**: the Linux kernel plus a chosen set of standard tools, package manager, and
|
||||
configuration conventions layered on top. Different distributions, same kernel underneath —
|
||||
which is why a command you learn on Ubuntu mostly still works on Alpine.
|
||||
|
||||
## Shell vs. terminal
|
||||
|
||||
These get used interchangeably but are different things:
|
||||
|
||||
- The **terminal** is the window/program that displays text and sends your keystrokes somewhere.
|
||||
- The **shell** is the program actually interpreting what you type as commands — `bash` is the
|
||||
shell this Lab uses. The terminal is just the display; the shell is the brain.
|
||||
|
||||
## Command structure
|
||||
|
||||
Every command follows roughly the same shape:
|
||||
|
||||
```
|
||||
command [options] [arguments]
|
||||
```
|
||||
|
||||
`ls -la /var/log` — `ls` is the command, `-la` are options (flags) that change its behavior,
|
||||
`/var/log` is the argument (what to act on). Options can usually be combined (`-l -a` = `-la`).
|
||||
|
||||
## Exit codes: how a command reports success or failure
|
||||
|
||||
Every command returns a number when it finishes: `0` means success, anything else (1-255) means
|
||||
some kind of failure. This isn't a Linux quirk — it's the exact mechanism scripts, CI pipelines,
|
||||
and (much later) Kubernetes health checks all rely on to know whether something worked. You can
|
||||
read the last command's exit code with the special variable `$?`.
|
||||
|
||||
## Getting help without leaving the terminal
|
||||
|
||||
- **Tab completion** — press Tab while typing a command or path; the shell completes it for you
|
||||
or shows you the options. This is a genuine time-saver, not just a convenience.
|
||||
- **Command history** — press the Up arrow to recall previous commands; `history` lists them.
|
||||
- **`man <command>`** — the manual page for almost any command, with every option explained.
|
||||
`man ls`, `man grep`, `man chmod` are all real, useful references — reaching for `man` before
|
||||
a search engine is a habit worth building now.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Lesson goal:** Establish shell fluency vocabulary (shell vs. terminal, exit codes, options vs.
|
||||
arguments) that every later module assumes without re-explaining.
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live with a short live demo of tab
|
||||
completion and `man`.
|
||||
|
||||
**Common misconception to watch for:** students often think "Linux" and "the terminal" are the
|
||||
same thing — clarify the kernel/distribution/shell/terminal layering explicitly.
|
||||
|
||||
**Demo idea:** live-type a command that fails (e.g. `ls /nonexistent`), then run `echo $?`
|
||||
immediately after to show the exit code in real time — more convincing than describing it.
|
||||
+49
@@ -0,0 +1,49 @@
|
||||
# Demonstration: Watching Exit Codes in Real Time
|
||||
|
||||
## OBSERVE
|
||||
|
||||
Run a command you know will succeed, then immediately check its exit code:
|
||||
|
||||
```
|
||||
$ true
|
||||
$ echo $?
|
||||
0
|
||||
```
|
||||
|
||||
## HYPOTHESIS
|
||||
|
||||
Now predict: what will the exit code be if the command fails instead?
|
||||
|
||||
## INSPECT
|
||||
|
||||
Run a command that fails, and check immediately:
|
||||
|
||||
```
|
||||
$ ls /this-does-not-exist
|
||||
ls: cannot access '/this-does-not-exist': No such file or directory
|
||||
$ echo $?
|
||||
2
|
||||
```
|
||||
|
||||
## CHANGE
|
||||
|
||||
Try a few other failing commands and note their codes: `grep nomatch /etc/hostname` (exit 1 —
|
||||
"no matches found," a different kind of failure than "file not found").
|
||||
|
||||
## VERIFY
|
||||
|
||||
Notice: exit codes aren't just "0 or 1" — different failures often get different non-zero
|
||||
codes. `0` always means success; the exact non-zero value's MEANING varies by command, but ANY
|
||||
non-zero value reliably means "something went wrong," which is the part every script actually
|
||||
relies on (`if command; then ... else ...`).
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo goal:** Make the abstract "exit code" concept concrete by actually watching `$?` change
|
||||
in real time, immediately after each command — this is far more convincing live than described
|
||||
in prose.
|
||||
|
||||
**Approximate timing:** 8 minutes live.
|
||||
|
||||
**Common mistake to watch for:** running `echo $?` after ANY other command in between (even
|
||||
`ls`) silently checks the WRONG command's exit code — emphasize "check it immediately."
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
# Knowledge Check: Linux and the Shell
|
||||
|
||||
---
|
||||
|
||||
**1. What's the difference between a shell and a terminal?**
|
||||
|
||||
*Answer:* The terminal is the window/program that displays text and captures keystrokes; the
|
||||
shell (e.g. `bash`) is the program actually interpreting what you type as commands. The terminal
|
||||
is the display, the shell is the brain.
|
||||
|
||||
---
|
||||
|
||||
**2. A command exits with code `0`. What does that tell you?**
|
||||
|
||||
*Answer:* It succeeded. `0` is the universal "success" convention every script and pipeline
|
||||
relies on; any non-zero value means some kind of failure occurred.
|
||||
|
||||
---
|
||||
|
||||
**3. You've typed half a long path and can't remember the rest. What should you press?**
|
||||
|
||||
*Answer:* Tab — tab completion finishes the path (or shows the options) without you needing to
|
||||
type or remember it exactly.
|
||||
|
||||
---
|
||||
|
||||
**4. Where would you look to learn every option a command supports, without leaving the
|
||||
terminal?**
|
||||
|
||||
*Answer:* Its manual page — `man <command>`.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 1 is the one students most often blur together — worth confirming
|
||||
verbally before moving on.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+31
@@ -0,0 +1,31 @@
|
||||
# Guided Exercise: Predict the Exit Code Before You Run It
|
||||
|
||||
This is a paper exercise — commit to an answer before checking it in the Lab.
|
||||
|
||||
## Your task
|
||||
|
||||
For each command below, predict whether its exit code will be `0` (success) or non-zero
|
||||
(failure), and why:
|
||||
|
||||
1. `cat /etc/hostname` (a file that exists)
|
||||
2. `cat /etc/does-not-exist-at-all` (a file that doesn't)
|
||||
3. `grep root /etc/passwd` (a pattern that's very likely present)
|
||||
4. `grep zzz-not-a-real-user /etc/passwd` (a pattern that's very likely absent)
|
||||
5. `mkdir /tmp/already-exists && mkdir /tmp/already-exists` (running the same `mkdir` twice)
|
||||
|
||||
## A worked shape to check yourself against
|
||||
|
||||
1. `0` — the file exists and is readable.
|
||||
2. Non-zero — the file doesn't exist, `cat` fails.
|
||||
3. `0` — `grep` succeeds (exit 0) when it finds at least one match.
|
||||
4. Non-zero (specifically `1`) — `grep` "fails" when it finds NO matches, even though nothing
|
||||
went technically wrong. This is a common surprise: `grep`'s exit code means "did I find a
|
||||
match," not "did an error occur."
|
||||
5. The SECOND `mkdir` fails (non-zero) — the directory already exists. The first one succeeds.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Item 4 is the one worth pausing on — many students assume "no error message"
|
||||
means "exit code 0," and `grep`'s no-match convention is a good early counter-example.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced.
|
||||
+25
@@ -0,0 +1,25 @@
|
||||
# Reflection: Exit Codes Are Everywhere After This
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Before this module, did you know commands returned a machine-readable success/failure
|
||||
signal at all? What did you assume happened instead?
|
||||
2. Where do you think exit codes might matter later in ROOT_ — in a script, a CI pipeline, or
|
||||
a container?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**Every automated system you'll ever build or operate checks exit codes, not error messages.**
|
||||
A script deciding whether to continue, a CI pipeline deciding whether to deploy, and (much
|
||||
later) a container orchestrator deciding whether a process is healthy, all reduce to the exact
|
||||
same question this module taught you to answer by hand: did this exit `0` or not? Getting
|
||||
comfortable checking `$?` now means none of those later systems will feel like magic.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 2 previews Docker/Kubernetes health
|
||||
checks well before students get there — useful to name explicitly without going deep yet.
|
||||
|
||||
**Approximate timing:** 5-8 minutes live discussion; 5 minutes self-paced writing.
|
||||
+55
@@ -0,0 +1,55 @@
|
||||
# Navigating the Filesystem
|
||||
|
||||
## Everything is one tree, starting at `/`
|
||||
|
||||
Linux has exactly one filesystem tree, rooted at `/` (pronounced "root," unrelated to your
|
||||
username). Every file and directory, no matter what physical disk it's actually on, appears
|
||||
somewhere under this one tree — there's no separate "C:\" and "D:\" the way Windows shows
|
||||
multiple drives.
|
||||
|
||||
## Absolute vs. relative paths
|
||||
|
||||
An **absolute path** starts with `/` and always means the same location no matter where you
|
||||
currently are: `/home/student/notes.txt`. A **relative path** is interpreted relative to your
|
||||
CURRENT directory: `notes.txt` means something different depending on where you're standing
|
||||
when you type it. `.` means "this directory," `..` means "one directory up" — `cd ..` moves up
|
||||
one level, `cd ../..` moves up two.
|
||||
|
||||
## Key commands
|
||||
|
||||
- `pwd` — print your current directory (where am I?)
|
||||
- `ls` — list what's in a directory (`ls -la` shows hidden files and details)
|
||||
- `cd <path>` — change directory (absolute or relative)
|
||||
|
||||
## Hidden files
|
||||
|
||||
A file or directory starting with `.` (like `.bashrc` or `.config`) is "hidden" — `ls` skips it
|
||||
by default, `ls -a` shows it. This isn't a security feature, just a convention to keep
|
||||
configuration/metadata files out of the way of normal directory listings.
|
||||
|
||||
## The important top-level directories (conceptually, not memorized)
|
||||
|
||||
You don't need to memorize the entire Filesystem Hierarchy Standard, but recognizing these by
|
||||
purpose will save you real confusion later:
|
||||
|
||||
- **`/etc`** — system-wide configuration files (nginx's config lives at `/etc/nginx/nginx.conf`)
|
||||
- **`/var`** — variable data that changes while the system runs: logs (`/var/log`), among others
|
||||
- **`/home`** — personal directories for regular users
|
||||
- **`/tmp`** — temporary files, safe to assume anything here can disappear on reboot
|
||||
- **`/usr`** — most installed programs and their supporting files live somewhere under here
|
||||
- **`/proc`** — not real files on disk at all — a live, kernel-generated view into running
|
||||
processes and system state (you'll use this again in Module 5)
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Lesson goal:** Give students a working mental model of "one tree, absolute vs. relative
|
||||
paths" plus just enough top-level-directory recognition to not be lost later — not FHS
|
||||
memorization.
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live.
|
||||
|
||||
**Common misconception to watch for:** students coming from Windows sometimes expect separate
|
||||
drives/roots — emphasize the single unified tree explicitly.
|
||||
|
||||
**Demo idea:** `cd /` then `ls`, narrating what each top-level directory is for as you see it,
|
||||
rather than listing them in the abstract.
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
# Demonstration: The Same Word, Two Different Destinations
|
||||
|
||||
## OBSERVE
|
||||
|
||||
Starting from `/home/student`, run `cd project` — you land at `/home/student/project`.
|
||||
|
||||
## HYPOTHESIS
|
||||
|
||||
Now, from `/home/student/project`, what do you predict `cd project` will do this time?
|
||||
|
||||
## INSPECT
|
||||
|
||||
```
|
||||
$ pwd
|
||||
/home/student/project
|
||||
$ cd project
|
||||
bash: cd: project: No such file or directory
|
||||
```
|
||||
|
||||
It fails — there's no `project` directory INSIDE `project` (unless you happened to create one).
|
||||
The exact same relative path meant something completely different depending on where you stood.
|
||||
|
||||
## CHANGE
|
||||
|
||||
From `/home/student/project`, run `cd ../project` instead — this correctly goes back up one
|
||||
level, then into `project` again, landing you right back where you started.
|
||||
|
||||
## VERIFY
|
||||
|
||||
`pwd` after each `cd` to confirm exactly where you ended up — this habit (verify your location
|
||||
before assuming it) prevents a huge share of "why did my command run somewhere unexpected"
|
||||
confusion later.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo goal:** Make "relative paths depend on where you are" concrete and slightly surprising,
|
||||
rather than an abstract rule to memorize.
|
||||
|
||||
**Approximate timing:** 8 minutes live.
|
||||
|
||||
**Common mistake to watch for:** students assume a relative path always means the SAME thing
|
||||
once they've used it once — reinforce that it's re-evaluated fresh every time, from wherever
|
||||
they currently are.
|
||||
+36
@@ -0,0 +1,36 @@
|
||||
# Knowledge Check: Navigating the Filesystem
|
||||
|
||||
---
|
||||
|
||||
**1. What does `..` mean in a path?**
|
||||
|
||||
*Answer:* One directory up from your current location.
|
||||
|
||||
---
|
||||
|
||||
**2. You're at `/home/student/project`. Does `cd project` do the same thing here as it did at
|
||||
`/home/student`?**
|
||||
|
||||
*Answer:* No — a relative path is re-evaluated from your CURRENT location every time. At
|
||||
`/home/student`, `project` refers to `/home/student/project`; from inside that same directory,
|
||||
`project` would refer to `/home/student/project/project`, which likely doesn't exist.
|
||||
|
||||
---
|
||||
|
||||
**3. A file is "hidden" from a normal `ls`. What actually makes it hidden?**
|
||||
|
||||
*Answer:* Its name starts with a `.` — it's purely a naming convention `ls` respects by
|
||||
default, not a real permission or security feature. `ls -a` shows it.
|
||||
|
||||
---
|
||||
|
||||
**4. Where would you expect to find a running system's log files?**
|
||||
|
||||
*Answer:* `/var/log` — `/var` holds data that changes while the system runs.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2 is the one worth confirming out loud — it's the core "gotcha" of
|
||||
relative paths.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+39
@@ -0,0 +1,39 @@
|
||||
# Guided Exercise: Plan a Directory Structure on Paper First
|
||||
|
||||
Before you build anything in the Lab, plan it — this is the same discipline you'll use later
|
||||
when planning a Kubernetes namespace layout or a Git repository structure.
|
||||
|
||||
## Your task
|
||||
|
||||
You're setting up a small project called `weather-app`. On paper, sketch a directory tree
|
||||
containing:
|
||||
|
||||
- A place for source code
|
||||
- A place for documentation
|
||||
- A place for tests
|
||||
- A single hidden configuration file at the project's root
|
||||
|
||||
Write out the exact `mkdir` and file-creation commands you'd use, including whether each path
|
||||
you write is absolute or relative.
|
||||
|
||||
## A worked shape to check yourself against
|
||||
|
||||
```
|
||||
mkdir -p weather-app/src weather-app/docs weather-app/tests
|
||||
echo "ready" > weather-app/.config
|
||||
```
|
||||
|
||||
(Or with an absolute path: `mkdir -p /srv/weather-app/src ...`.) The `-p` flag matters here —
|
||||
without it, `mkdir` refuses to create `weather-app/src` if `weather-app` doesn't already exist.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Have students commit to their command in writing before revealing the worked
|
||||
answer — the value is in noticing what `-p` actually buys them, not just copying a correct
|
||||
command.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced.
|
||||
|
||||
**Common mistake to watch for:** forgetting `mkdir -p` and getting a "No such file or
|
||||
directory" error when creating a nested path in one command — a good moment to connect back to
|
||||
Module 2's exit-code lesson (that error IS a non-zero exit code).
|
||||
+27
@@ -0,0 +1,27 @@
|
||||
# Reflection: Structure Isn't Bureaucracy, It's Communication
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Why might a real engineering team insist on a consistent directory structure across every
|
||||
project, instead of everyone organizing files however they like?
|
||||
2. Which of `/etc`, `/var`, `/home`, `/tmp`, `/usr`, `/proc` do you feel most confident you could
|
||||
explain to someone else right now? Which one still feels fuzzy?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**A predictable filesystem layout is what lets tools (and other engineers) reason about a
|
||||
system without asking you first.** `/etc` always means configuration, `/var/log` always means
|
||||
logs — this convention is exactly why a monitoring tool can be pointed at "check `/var/log`" on
|
||||
any Linux machine and just work. Container images later inherit this same convention (an
|
||||
nginx container still keeps its config at `/etc/nginx/nginx.conf`) — you're not learning a
|
||||
one-off Linux fact, you're learning a convention that outlives this specific module.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 1 connects directly to why Kubernetes
|
||||
namespaces and Git repository conventions matter later — a good moment to name that connection
|
||||
without going deep yet.
|
||||
|
||||
**Approximate timing:** 5-8 minutes live discussion; 5 minutes self-paced writing.
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
# Files and Text
|
||||
|
||||
## Basic file operations
|
||||
|
||||
- `touch <file>` — create an empty file (or update its timestamp if it exists)
|
||||
- `mkdir <dir>` — create a directory (`-p` creates any missing parent directories too)
|
||||
- `cp <src> <dst>` — copy a file
|
||||
- `mv <src> <dst>` — move OR rename a file (Linux treats renaming as "moving to a new name in
|
||||
the same place")
|
||||
- `rm <file>` — delete a file (`-r` for a directory, `-f` to skip confirmation — dangerous
|
||||
combined: `rm -rf` deletes without asking, permanently)
|
||||
|
||||
## Reading text
|
||||
|
||||
- `cat <file>` — print the whole file at once
|
||||
- `less <file>` — page through a file interactively (press `q` to quit) — the right tool for
|
||||
anything too long to read in one screen
|
||||
- `head <file>` / `tail <file>` — the first/last 10 lines by default (`-n 20` for a different
|
||||
count); `tail -f` follows a file live as new lines are appended — exactly how you'd watch a
|
||||
log file update in real time
|
||||
|
||||
## Searching text: `grep`
|
||||
|
||||
`grep PATTERN file` prints only the lines matching PATTERN. This is one of the most-used
|
||||
commands in real operations work — "find the lines that matter" beats "read everything."
|
||||
|
||||
## Pipes and redirection: composing commands
|
||||
|
||||
- **A pipe (`|`)** sends one command's output as the next command's input:
|
||||
`cat app.log | grep ERROR` — read the file, then filter it.
|
||||
- **Redirection (`>`)** sends a command's output into a file instead of the screen (overwriting
|
||||
it): `grep ERROR app.log > errors.txt`. `>>` appends instead of overwriting.
|
||||
|
||||
## stdout vs. stderr
|
||||
|
||||
A command actually has TWO separate output streams: **stdout** (normal output) and **stderr**
|
||||
(error messages), even though both print to your screen by default. This is why
|
||||
`command 2>/dev/null` (redirecting stream `2`, stderr, to nowhere) can suppress error noise
|
||||
while still showing normal output — you'll use exactly this pattern in later Labs.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Lesson goal:** Build real fluency composing small commands together — this is the single
|
||||
highest-leverage Linux skill for troubleshooting work later in ROOT_.
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live.
|
||||
|
||||
**Common misconception to watch for:** students often don't realize stdout/stderr are separate
|
||||
streams until they see `2>/dev/null` change what's suppressed — worth demonstrating live.
|
||||
|
||||
**Demo idea:** run a command that produces both real output and an error in the same call
|
||||
(e.g. `ls /etc /nonexistent`) and show how `2>/dev/null` changes what appears.
|
||||
+56
@@ -0,0 +1,56 @@
|
||||
# Demonstration: Filtering a Log Down to What Matters
|
||||
|
||||
## OBSERVE
|
||||
|
||||
A raw log file has five lines mixing `INFO` and `ERROR`:
|
||||
|
||||
```
|
||||
$ cat app.log
|
||||
INFO: boot ok
|
||||
ERROR: disk full
|
||||
INFO: retry
|
||||
ERROR: timeout
|
||||
INFO: done
|
||||
```
|
||||
|
||||
## HYPOTHESIS
|
||||
|
||||
Predict what `grep ERROR app.log` will print, before running it.
|
||||
|
||||
## INSPECT
|
||||
|
||||
```
|
||||
$ grep ERROR app.log
|
||||
ERROR: disk full
|
||||
ERROR: timeout
|
||||
```
|
||||
|
||||
Only the matching lines — everything else is filtered out.
|
||||
|
||||
## CHANGE
|
||||
|
||||
Now redirect that filtered result into a new file instead of just viewing it:
|
||||
|
||||
```
|
||||
$ grep ERROR app.log > errors.txt
|
||||
$ cat errors.txt
|
||||
ERROR: disk full
|
||||
ERROR: timeout
|
||||
```
|
||||
|
||||
## VERIFY
|
||||
|
||||
Count how many lines actually matched, without eyeballing them: `wc -l < errors.txt` (or
|
||||
`grep -c ERROR app.log` directly). This composition — filter, then count, then act — is the
|
||||
exact shape of a huge share of real troubleshooting work.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo goal:** Show the filter → redirect → verify chain as one continuous, composable
|
||||
workflow, not three unrelated commands.
|
||||
|
||||
**Approximate timing:** 8 minutes live.
|
||||
|
||||
**Common mistake to watch for:** using `>` when they meant `>>` and accidentally overwriting a
|
||||
file they wanted to append to — worth calling out explicitly since it's a real, common mistake
|
||||
with no undo.
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
# Knowledge Check: Files and Text
|
||||
|
||||
---
|
||||
|
||||
**1. What's the difference between `>` and `>>`?**
|
||||
|
||||
*Answer:* `>` overwrites the target file's contents entirely; `>>` appends to the end of it,
|
||||
preserving what was already there.
|
||||
|
||||
---
|
||||
|
||||
**2. A command prints both normal output and an error message. What are these two streams
|
||||
called?**
|
||||
|
||||
*Answer:* stdout (normal output) and stderr (error messages) — two separate streams that both
|
||||
happen to print to your screen by default.
|
||||
|
||||
---
|
||||
|
||||
**3. You want to watch a log file update live, as new lines are appended, without re-running a
|
||||
command repeatedly.**
|
||||
|
||||
*Answer:* `tail -f <file>` — follows the file and prints new lines as they're written.
|
||||
|
||||
---
|
||||
|
||||
**4. `cat app.log | grep ERROR`. What does the `|` actually do?**
|
||||
|
||||
*Answer:* It pipes `cat`'s output directly into `grep`'s input, so `grep` filters the file's
|
||||
contents without needing a separate temporary file.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2 is worth a live show-of-hands check — many students haven't
|
||||
consciously registered that stdout/stderr are separate until this module.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+31
@@ -0,0 +1,31 @@
|
||||
# Guided Exercise: Which Tool Actually Answers the Question?
|
||||
|
||||
This is a paper exercise — commit to an answer before checking it in the Lab.
|
||||
|
||||
## Your task
|
||||
|
||||
For each situation, name the ONE command (or short pipeline) you'd reach for first:
|
||||
|
||||
1. You want to see just the last 20 lines of a 50,000-line log file.
|
||||
2. You want to know how many lines in a file contain the word "timeout."
|
||||
3. You want to watch a log file update live while a process is running.
|
||||
4. You want to see a file's full contents but it's too long for one screen and you want to
|
||||
scroll through it interactively.
|
||||
|
||||
## A worked shape to check yourself against
|
||||
|
||||
1. `tail -n 20 file.log`
|
||||
2. `grep -c timeout file.log`
|
||||
3. `tail -f file.log`
|
||||
4. `less file.log`
|
||||
|
||||
Notice: `cat` would technically "work" for #4 too, but it dumps everything at once with no way
|
||||
to scroll back — `less` is the actual right tool once a file is longer than one screen.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Item 4 is worth discussing explicitly — "it works" isn't the same bar as
|
||||
"it's the right tool for this situation," and that distinction matters more as commands compose
|
||||
into real troubleshooting workflows.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced.
|
||||
@@ -0,0 +1,26 @@
|
||||
# Reflection: Filtering Is a Reasoning Skill, Not a Syntax Trick
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Before this module, how would you have found "the important lines" in a huge log file?
|
||||
How does that compare to `grep`?
|
||||
2. Which combination (pipe, redirect, or a specific command like `tail -f`) do you expect to
|
||||
use the most once you're doing real troubleshooting work?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**"Read everything and eyeball it" does not scale — filtering does.** Every troubleshooting
|
||||
framework you'll learn later in ROOT_ (starting with ROOT 02's own version, then again more
|
||||
formally in ROOT 10) depends on being able to quickly narrow a huge amount of system output down
|
||||
to the handful of lines that actually matter. `grep`, pipes, and redirection are the first real
|
||||
tools you have for that — not a syntax exercise, a reasoning skill.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 1 often produces good answers ("I'd
|
||||
just scroll and read") that make the value of filtering obvious by contrast — let students
|
||||
articulate this themselves rather than stating it for them.
|
||||
|
||||
**Approximate timing:** 5-8 minutes live discussion; 5 minutes self-paced writing.
|
||||
@@ -0,0 +1,64 @@
|
||||
# Processes
|
||||
|
||||
## What a process actually is
|
||||
|
||||
A **process** is a running instance of a program — its own code, its own private memory, and a
|
||||
unique identifier the kernel tracks called a **PID** (Process ID). Run the same program twice
|
||||
(open two terminals both running `bash`), and you get two separate processes with two different
|
||||
PIDs, even though they're running identical code.
|
||||
|
||||
## Foreground vs. background
|
||||
|
||||
A command you run normally occupies your terminal until it finishes (**foreground**) — you
|
||||
can't type anything else until it's done. Append `&` to run it in the **background** instead,
|
||||
freeing your terminal immediately while the process keeps running. `sleep 300 &` starts a
|
||||
300-second sleep in the background and hands your prompt right back.
|
||||
|
||||
## Inspecting processes: `ps` and `top`
|
||||
|
||||
- `ps aux` — a snapshot of every process currently running, including its PID, how much CPU/
|
||||
memory it's using, and the exact command that started it.
|
||||
- `top` (or `htop`, if installed) — the same idea, but live-updating and sorted by resource
|
||||
usage, useful for "what's actually eating CPU/memory right now."
|
||||
|
||||
## Stopping a process: `kill` and signals
|
||||
|
||||
`kill <PID>` doesn't instantly destroy a process — it sends it a **signal**, a message the
|
||||
process can (usually) choose how to respond to. The default signal, `SIGTERM` (15), politely
|
||||
asks a process to shut down, giving it a chance to clean up first. `kill -9 <PID>` sends
|
||||
`SIGKILL`, which the kernel enforces immediately with no chance for the process to react —
|
||||
appropriate only when a process is truly stuck and ignoring `SIGTERM`. `kill -0 <PID>` is a
|
||||
special case: it sends NO real signal at all, just checks whether the PID still exists —
|
||||
exactly how you confirm "did that process actually stop" without side effects.
|
||||
|
||||
## Process hierarchy
|
||||
|
||||
Every process (except the very first one at boot) has a **parent** — the process that started
|
||||
it. Your shell is the parent of every command you run from it. This matters later: a parent
|
||||
process's own lifecycle can affect its children (this is part of why a container, which is
|
||||
often just one process tree, behaves the way it does when its main process exits).
|
||||
|
||||
## Why this matters for containers and Kubernetes
|
||||
|
||||
A container is, underneath, mostly an isolated process (or small process tree) with its own
|
||||
view of the filesystem and network — when people say "a container exited," they mean its main
|
||||
process exited, exactly the same event you'll cause on purpose in this module's Lab. Kubernetes'
|
||||
entire job, at its core, is deciding which machine a process (inside a container) runs on and
|
||||
restarting it if it dies unexpectedly — none of that will feel intuitive without this module's
|
||||
model of what a process actually is.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Lesson goal:** Build a real process mental model (PID, foreground/background, signals,
|
||||
parent/child) that ROOT 04 (containers) and ROOT 06 (Kubernetes workloads) both directly build
|
||||
on without re-explaining.
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live.
|
||||
|
||||
**Common misconception to watch for:** students often think `kill` always means "forcefully
|
||||
destroy" — clarify that plain `kill` sends a polite, ignorable request (SIGTERM), and `-9` is
|
||||
the forceful escalation, not the default.
|
||||
|
||||
**Discussion question:** "What do you think happens to a container if its main process exits on
|
||||
its own, versus if something outside kills it?" (Don't need a complete answer yet — just plant
|
||||
the question for later.)
|
||||
+53
@@ -0,0 +1,53 @@
|
||||
# Demonstration: Watching a Process Actually Terminate
|
||||
|
||||
## OBSERVE
|
||||
|
||||
Start a long-running background process and note its PID:
|
||||
|
||||
```
|
||||
$ sleep 300 &
|
||||
[1] 1421
|
||||
```
|
||||
|
||||
`1421` is the PID (yours will differ every time).
|
||||
|
||||
## HYPOTHESIS
|
||||
|
||||
Predict: will `kill -0 1421` print anything if the process is still running?
|
||||
|
||||
## INSPECT
|
||||
|
||||
```
|
||||
$ kill -0 1421 && echo "still running"
|
||||
still running
|
||||
```
|
||||
|
||||
`kill -0` sends no real signal — silence (or, here, our own `echo`) means the process exists.
|
||||
|
||||
## CHANGE
|
||||
|
||||
Now actually terminate it:
|
||||
|
||||
```
|
||||
$ kill 1421
|
||||
```
|
||||
|
||||
## VERIFY
|
||||
|
||||
```
|
||||
$ kill -0 1421 2>/dev/null && echo "still running" || echo "gone"
|
||||
gone
|
||||
```
|
||||
|
||||
The exact same check now reports the opposite result — real, observable proof the process is
|
||||
gone, not an assumption.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo goal:** Make "checking before and after" a visible habit, not just narrated — students
|
||||
should see the SAME command return a different real answer once state actually changed.
|
||||
|
||||
**Approximate timing:** 8 minutes live.
|
||||
|
||||
**Common mistake to watch for:** reaching for `kill -9` by default "to be safe" — reinforce that
|
||||
plain `kill` (SIGTERM) is the correct first attempt almost always.
|
||||
@@ -0,0 +1,38 @@
|
||||
# Knowledge Check: Processes
|
||||
|
||||
---
|
||||
|
||||
**1. What does a PID identify?**
|
||||
|
||||
*Answer:* One specific running instance of a program — the kernel's unique identifier for that
|
||||
process, distinct from every other running process, even other copies of the same program.
|
||||
|
||||
---
|
||||
|
||||
**2. What's the actual difference between `kill <pid>` and `kill -9 <pid>`?**
|
||||
|
||||
*Answer:* Plain `kill` sends SIGTERM, a polite request the process can choose to handle (clean
|
||||
up, then exit). `kill -9` sends SIGKILL, which the kernel enforces immediately with no chance
|
||||
for the process to react — an escalation for when SIGTERM is ignored, not the default choice.
|
||||
|
||||
---
|
||||
|
||||
**3. You want to confirm a process is still running WITHOUT affecting it in any way. What do
|
||||
you use?**
|
||||
|
||||
*Answer:* `kill -0 <pid>` — sends no real signal, only checks whether the PID still exists.
|
||||
|
||||
---
|
||||
|
||||
**4. Why might understanding processes matter for understanding containers later?**
|
||||
|
||||
*Answer:* A container is, underneath, mostly an isolated process (or small process tree) — a
|
||||
container "exiting" is really its main process exiting, the exact same event this module's Lab
|
||||
causes on purpose.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2 is worth confirming explicitly — "kill -9 by default" is a common,
|
||||
worth-correcting habit.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+28
@@ -0,0 +1,28 @@
|
||||
# Guided Exercise: Choosing the Right Signal
|
||||
|
||||
This is a paper exercise — commit to an answer before checking it in the Lab.
|
||||
|
||||
## Your task
|
||||
|
||||
For each situation, decide: plain `kill` (SIGTERM) first, or is `kill -9` (SIGKILL) actually
|
||||
justified?
|
||||
|
||||
1. A normal web server process you want to shut down for a routine restart.
|
||||
2. A process that received SIGTERM two minutes ago and is still running, showing no signs of
|
||||
responding.
|
||||
3. A script you just started by mistake, that hasn't done anything harmful yet.
|
||||
|
||||
## A worked shape to check yourself against
|
||||
|
||||
1. Plain `kill` — give it the chance to shut down cleanly (close connections, flush data, etc).
|
||||
2. `kill -9` is now justified — you already gave it the polite chance, and it didn't respond.
|
||||
3. Plain `kill` first, still — "just started" doesn't change the default; SIGKILL is a last
|
||||
resort, not a shortcut for convenience.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Item 3 sometimes trips students who reach for `-9` out of impatience — worth
|
||||
reinforcing that SIGTERM-first is the discipline REGARDLESS of urgency, since a process that
|
||||
ignores SIGTERM is the actual signal that escalation is warranted.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Reflection: You Just Stopped a Real Process, Not a Simulation
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Before checking `kill -0` after your `kill`, how confident were you the process was
|
||||
actually gone? What would you have done if it hadn't stopped?
|
||||
2. Why do you think SIGTERM-before-SIGKILL matters more for a real production service than for
|
||||
a throwaway script in a Lab?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**"I ran the kill command" is not the same as "I confirmed the process is gone."** This module's
|
||||
Lab required you to verify state, not just take an action and assume it worked — that
|
||||
verification step is exactly what separates a confident operator from someone hoping for the
|
||||
best. You'll repeat this exact pattern (act, then verify) constantly for the rest of ROOT_,
|
||||
starting as soon as ROOT 04's containers and continuing all the way through Kubernetes
|
||||
troubleshooting.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 2 is a good bridge to ROOT 06's later
|
||||
discussion of graceful shutdown/`terminationGracePeriodSeconds` — plant the idea now without
|
||||
naming Kubernetes specifics yet.
|
||||
|
||||
**Approximate timing:** 5-8 minutes live discussion; 5 minutes self-paced writing.
|
||||
+59
@@ -0,0 +1,59 @@
|
||||
# Users and Permissions Foundations
|
||||
|
||||
## Users, groups, and ownership
|
||||
|
||||
Every file on Linux has an **owner** (a user) and a **group**. `ls -la` shows both: the third
|
||||
column is the owning user, the fourth is the owning group. Multiple users can belong to the same
|
||||
group, which is how permissions get shared across a team without giving every individual user
|
||||
direct ownership.
|
||||
|
||||
## Read, write, execute — for three different audiences
|
||||
|
||||
Every file has permissions for THREE separate audiences: the owner, the group, and everyone
|
||||
else ("other"). Each audience gets some combination of:
|
||||
|
||||
- **r**ead — can view the file's contents (or list a directory's contents)
|
||||
- **w**rite — can modify the file (or create/delete files inside a directory)
|
||||
- e**x**ecute — can run the file as a program (or enter/traverse into a directory)
|
||||
|
||||
`ls -la` shows this as a 10-character string like `-rw-r--r--`: the first character is the file
|
||||
type, then three groups of `rwx` for owner/group/other.
|
||||
|
||||
## Reading and setting permissions as octal numbers
|
||||
|
||||
Each `rwx` triplet can be written as one digit: read=4, write=2, execute=1, summed together.
|
||||
`rw-` = 4+2 = `6`. `r--` = `4`. So `644` means: owner can read+write (6), group can read-only
|
||||
(4), everyone else can read-only (4) — a very common, sensible default for a regular file.
|
||||
`chmod 644 <file>` sets exactly that.
|
||||
|
||||
## `sudo`: temporary elevated privilege
|
||||
|
||||
Rather than logging in AS the root user (with permanent full access to everything), `sudo
|
||||
<command>` runs just that one command with elevated privileges, after checking you're allowed
|
||||
to. This is deliberate: the less time you spend with full privileges, the smaller the window for
|
||||
a mistake (or a compromised session) to do serious damage — the exact same reasoning behind
|
||||
"least privilege," which this module introduces by name for the first time.
|
||||
|
||||
## Least privilege: the principle, not just the command
|
||||
|
||||
**Least privilege** means: grant exactly the access something needs to do its job, and nothing
|
||||
more. A user who only needs to read a file shouldn't also be able to write it. A process that
|
||||
only needs to read one directory shouldn't have write access to the whole filesystem. This
|
||||
principle isn't specific to Linux permissions — you'll meet it again, in a much more elaborate
|
||||
form, in ROOT 09's Kubernetes RBAC and SecurityContext content. This module is where it starts.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Lesson goal:** Build real fluency with rwx/octal permissions AND plant the least-privilege
|
||||
PRINCIPLE explicitly — not just command syntax, since ROOT 09 depends on students already
|
||||
having this vocabulary.
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live.
|
||||
|
||||
**Common misconception to watch for:** students often memorize "644 = read/write" without being
|
||||
able to derive it from 4+2+1 — walk through the arithmetic explicitly rather than treating the
|
||||
numbers as magic.
|
||||
|
||||
**Discussion question:** "Why not just make everything readable and writable by everyone, to
|
||||
avoid permission headaches?" (Good setup for the least-privilege principle — let students
|
||||
generate the security argument themselves before you confirm it.)
|
||||
+55
@@ -0,0 +1,55 @@
|
||||
# Demonstration: A Real Permission Denial, and the Real Fix
|
||||
|
||||
## OBSERVE
|
||||
|
||||
A file is owned by root with permissions `600` (owner read/write only):
|
||||
|
||||
```
|
||||
$ stat -c '%a %U' data.txt
|
||||
600 root
|
||||
```
|
||||
|
||||
## HYPOTHESIS
|
||||
|
||||
Predict: can a different, non-root user read this file?
|
||||
|
||||
## INSPECT
|
||||
|
||||
```
|
||||
$ su - labuser -c "cat data.txt"
|
||||
cat: data.txt: Permission denied
|
||||
```
|
||||
|
||||
Confirmed — `600` grants nothing to group or other, so `labuser` (neither the owner nor root)
|
||||
is correctly denied.
|
||||
|
||||
## CHANGE
|
||||
|
||||
Grant read access to everyone, while keeping write access limited to the owner:
|
||||
|
||||
```
|
||||
$ chmod 644 data.txt
|
||||
$ stat -c '%a' data.txt
|
||||
644
|
||||
```
|
||||
|
||||
## VERIFY
|
||||
|
||||
```
|
||||
$ su - labuser -c "cat data.txt"
|
||||
sensitive
|
||||
```
|
||||
|
||||
The exact same command that failed before now succeeds — the permission bits, not the file's
|
||||
actual content, were the only thing that changed.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo goal:** Show a permission denial as a real, reproducible fact (not just a described
|
||||
error message) and the exact minimal fix, not an overcorrection like `chmod 777`.
|
||||
|
||||
**Approximate timing:** 8 minutes live.
|
||||
|
||||
**Common mistake to watch for:** students reaching for `chmod 777` "to make the error go away"
|
||||
— explicitly name why that's wrong (it also grants WRITE to everyone, not just the read access
|
||||
actually needed) and connect it back to least privilege.
|
||||
+38
@@ -0,0 +1,38 @@
|
||||
# Knowledge Check: Users and Permissions Foundations
|
||||
|
||||
---
|
||||
|
||||
**1. A colleague says "permission denied" and fixes it with `chmod 777`. What's wrong with
|
||||
that fix?**
|
||||
|
||||
*Answer:* `777` grants read, write, AND execute to the owner, group, AND everyone else — almost
|
||||
always far more access than the actual problem required. It "fixes" the symptom while violating
|
||||
least privilege, usually granting write access nobody asked for.
|
||||
|
||||
---
|
||||
|
||||
**2. What does the permission `644` actually mean, broken down?**
|
||||
|
||||
*Answer:* Owner: read+write (4+2=6). Group: read-only (4). Other: read-only (4). Nobody but the
|
||||
owner can modify the file; everyone can read it.
|
||||
|
||||
---
|
||||
|
||||
**3. Why does `sudo <command>` exist instead of just logging in as root permanently?**
|
||||
|
||||
*Answer:* It grants elevated privilege only for the one command that needs it, minimizing the
|
||||
time spent with full access — reducing the chance a mistake (or a compromised session) causes
|
||||
serious, wide damage.
|
||||
|
||||
---
|
||||
|
||||
**4. State the least-privilege principle in your own words.**
|
||||
|
||||
*Answer:* Grant exactly the access something needs to do its job, and nothing more.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 1 is worth discussing as a group — most students have seen `chmod
|
||||
777` somewhere online as "the fix," and unlearning that habit early matters.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+32
@@ -0,0 +1,32 @@
|
||||
# Guided Exercise: Pick the Minimal Permission, Not the Easy One
|
||||
|
||||
This is a paper exercise — commit to an answer before checking it in the Lab.
|
||||
|
||||
## Your task
|
||||
|
||||
For each situation, write the exact `chmod` octal value that grants the MINIMUM permission
|
||||
actually needed:
|
||||
|
||||
1. A private SSH key file — only the owner should ever read or write it, nobody else, no
|
||||
execute for anyone.
|
||||
2. A shared script every user on the system should be able to run, but only the owner should be
|
||||
able to edit.
|
||||
3. A log file every user should be able to read (for troubleshooting), but only a logging
|
||||
service (the owner) should write to it.
|
||||
|
||||
## A worked shape to check yourself against
|
||||
|
||||
1. `600` — owner read+write (6), nothing for group or other (0, 0). This is the real, standard
|
||||
permission for private SSH keys — SSH itself refuses to use a key file that's more open than
|
||||
this.
|
||||
2. `755` — owner read+write+execute (7), group and other read+execute (5, 5) so they can run it
|
||||
but not modify it.
|
||||
3. `644` — owner read+write (6), group and other read-only (4, 4).
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Item 1 is a good moment to mention that this isn't a hypothetical rule — SSH
|
||||
genuinely refuses to use an overly-permissive private key file, a real-world consequence of
|
||||
exactly this principle.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced.
|
||||
+33
@@ -0,0 +1,33 @@
|
||||
# Reflection: Least Privilege Is a Habit, Not a One-Time Fix
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Before this module, what would you have done the first time you hit a "permission denied"
|
||||
error? How does that compare to what you'd do now?
|
||||
2. Name one place outside of file permissions where you'd expect "grant only what's needed" to
|
||||
matter (a user account, an API key, a service's network access — anything).
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**Least privilege is the single idea from this entire module that reappears the most across
|
||||
the rest of ROOT_.** File permissions are just the first, most concrete place you'll apply it —
|
||||
ROOT 09 applies the exact same principle to Kubernetes RBAC (what can a user or workload
|
||||
actually DO) and SecurityContext (what can a container actually access). Getting comfortable
|
||||
asking "what's the minimum this actually needs" now, on a file you can see and touch, makes that
|
||||
same question much easier to ask later about something more abstract, like a ServiceAccount's
|
||||
permissions.
|
||||
|
||||
This module is complete. This is the end of ROOT 01 — continue to ROOT 02: Linux
|
||||
Administration & Networking.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 2 works well as a round-robin — the
|
||||
variety of answers (API keys, database credentials, network firewall rules) is itself useful
|
||||
evidence of how far this one principle travels.
|
||||
|
||||
**Approximate timing:** 5-8 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Milestone note:** this is the last module of ROOT 01 — consider a short recap of the whole
|
||||
course's arc (hardware → shell → filesystem → files/text → processes → permissions) before
|
||||
moving on, so students see it as one coherent foundation rather than six separate topics.
|
||||
+108
@@ -0,0 +1,108 @@
|
||||
# What Kubernetes Solves & the Desired-State Model
|
||||
|
||||
## Before Kubernetes: the problem
|
||||
|
||||
Imagine you're running a web application on three servers by hand. One server crashes at 2am —
|
||||
someone has to notice, log in, and restart the application. Traffic grows, so you need a fourth
|
||||
server — someone has to provision it, install dependencies, and configure it to match the other
|
||||
three. You ship a new version — someone has to update all four servers, one at a time, without
|
||||
taking the whole application down.
|
||||
|
||||
None of this is hard in isolation. It becomes unmanageable at scale, and it depends entirely on
|
||||
a human noticing and reacting fast enough. Kubernetes exists to make a computer do that
|
||||
noticing and reacting instead — continuously, automatically, and the same way every time.
|
||||
|
||||
## The core idea: desired state
|
||||
|
||||
You don't tell Kubernetes *how* to run your application step by step. You tell it *what you
|
||||
want to be true* — "3 replicas of this container should be running" — and Kubernetes
|
||||
continuously works to make that true, correcting drift whenever reality doesn't match.
|
||||
|
||||
This is the single most important mental shift for someone new to Kubernetes: you're not
|
||||
writing a script that runs once. You're declaring a **desired state**, and a background process
|
||||
never stops comparing the real world against it.
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: web
|
||||
spec:
|
||||
replicas: 3 # <- the desired state: "3 of these, always"
|
||||
...
|
||||
```
|
||||
|
||||
If a Pod crashes, Kubernetes doesn't "restart the crashed one" as a special case — it just
|
||||
notices reality (2 running) no longer matches desired state (3 wanted) and creates a
|
||||
replacement. Same mechanism, no special-casing.
|
||||
|
||||
## The cluster mental model
|
||||
|
||||
A Kubernetes cluster has two conceptually different kinds of machines:
|
||||
|
||||
- **The control plane** — the "brain." It stores the desired state (in `etcd`), watches actual
|
||||
state, and makes decisions (the API server, scheduler, and controllers all live here). You
|
||||
talk to the control plane through `kubectl`; you never SSH into it to manage workloads.
|
||||
- **Worker nodes** — where your containers actually run. Each node runs a `kubelet` (an agent
|
||||
that starts/stops containers as the control plane instructs) and a container runtime.
|
||||
|
||||
You describe desired state to the control plane. The control plane decides which nodes should
|
||||
run what. The nodes' kubelets make it actually happen. You never manually place a Pod on a
|
||||
specific node in normal operation — that placement decision belongs to the scheduler.
|
||||
|
||||
## Declarative, not imperative
|
||||
|
||||
"Imperative" means giving step-by-step instructions ("start a container, then another, then
|
||||
configure networking between them"). "Declarative" means describing an end state and letting
|
||||
the system figure out the steps. Kubernetes' API objects (Pods, Deployments, Services, and
|
||||
everything else you'll learn in ROOT_) are declarative — you write YAML describing what should
|
||||
exist, `kubectl apply` submits it, and the control plane does the rest.
|
||||
|
||||
This is also why Kubernetes YAML looks the way it does: it's not a script, it's a specification.
|
||||
|
||||
## Pods, Namespaces, and labels — the vocabulary you need first
|
||||
|
||||
- **Pod** — the smallest deployable unit. Usually one container, sometimes a small group of
|
||||
tightly-coupled containers that must run together on the same node and share networking.
|
||||
You'll rarely create bare Pods directly in real use (a Deployment manages them for you, see
|
||||
ROOT 06) but understanding the Pod itself is foundational to everything built on top of it.
|
||||
- **Namespace** — a way to partition one cluster into logically separate areas (e.g. `dev`,
|
||||
`staging`, teams, or projects), so names don't collide and access/quotas can be scoped.
|
||||
- **Labels** — arbitrary key/value tags attached to objects (`app: web`, `env: prod`). Labels
|
||||
are how almost everything in Kubernetes finds related objects — a Deployment finds "its" Pods
|
||||
by label selector, not by name pattern matching.
|
||||
|
||||
## kubectl: your one tool for all of this
|
||||
|
||||
`kubectl` is the command-line client that talks to the control plane's API server. Nearly
|
||||
everything you'll do in ROOT_ Labs goes through it: `kubectl get` (list/inspect),
|
||||
`kubectl describe` (detailed status + recent events — your primary troubleshooting tool),
|
||||
`kubectl apply -f` (submit a desired-state YAML file), `kubectl logs` (see what a container
|
||||
printed). You'll build real fluency with these across this module and the ones that follow.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read, 10-12 minutes for a live walkthrough with discussion.
|
||||
|
||||
**Demo plan:** Before the demonstration activity, it's worth drawing the control-plane/worker
|
||||
split on a whiteboard (or sharing a simple diagram) and tracing one request: `kubectl apply` ->
|
||||
API server -> etcd -> scheduler -> kubelet -> container runtime. Students retain the mental
|
||||
model much better when they've seen the path, not just read about it.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often assume `kubectl` talks directly to nodes/containers. It doesn't — it only ever
|
||||
talks to the API server. Worth stating explicitly.
|
||||
- "Declarative" is sometimes read as "automatic" in a vague sense. Be concrete: it means "you
|
||||
describe the end state; a controller loop makes it true," not "Kubernetes guesses what you
|
||||
want."
|
||||
|
||||
**Discussion question:** "If desired state says 3 replicas and you manually run `kubectl delete
|
||||
pod` on one of them, what happens next, and why?" (Answer: a new one is created almost
|
||||
immediately — deleting a Pod doesn't change the Deployment's desired replica count, so the
|
||||
control loop restores it.)
|
||||
|
||||
**Optional extension:** Ask students to predict what would happen if they edited a running Pod's
|
||||
container image directly with `kubectl edit pod` instead of updating the Deployment that manages
|
||||
it (answer: it works until that specific Pod is replaced for any reason, at which point the new
|
||||
Pod reverts to whatever the Deployment's template says — a good early illustration of "the
|
||||
Deployment is the source of truth, not the running Pod").
|
||||
+76
@@ -0,0 +1,76 @@
|
||||
# Demonstration: Desired State Reconciliation
|
||||
|
||||
Watch what actually happens when reality drifts from desired state — this is the mechanism the
|
||||
whole lesson describes, made concrete.
|
||||
|
||||
## Setup
|
||||
|
||||
A Namespace called `demo` exists with a Deployment `web` declaring `replicas: 2`. Right now,
|
||||
reality matches desired state:
|
||||
|
||||
```
|
||||
$ kubectl get pods -n demo
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
web-7d8f9c6b5d-abcde 1/1 Running 0 3m
|
||||
web-7d8f9c6b5d-fghij 1/1 Running 0 3m
|
||||
```
|
||||
|
||||
## Someone (or something) deletes one Pod
|
||||
|
||||
```
|
||||
$ kubectl delete pod web-7d8f9c6b5d-abcde -n demo
|
||||
pod "web-7d8f9c6b5d-abcde" deleted
|
||||
```
|
||||
|
||||
## What happens in the next second or two, without anyone doing anything else
|
||||
|
||||
```
|
||||
$ kubectl get pods -n demo
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
web-7d8f9c6b5d-fghij 1/1 Running 0 3m
|
||||
web-7d8f9c6b5d-klmno 0/1 ContainerCreating 0 1s
|
||||
```
|
||||
|
||||
A brand new Pod (`...-klmno`, a new random suffix — this is NOT the deleted Pod coming back)
|
||||
appeared on its own. Nobody ran `kubectl create` again. The Deployment's controller noticed
|
||||
actual state (1 running Pod matching its label selector) no longer matched desired state
|
||||
(`replicas: 2`), and created exactly one replacement to close the gap — not two, not zero.
|
||||
|
||||
A few seconds later:
|
||||
|
||||
```
|
||||
$ kubectl get pods -n demo
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
web-7d8f9c6b5d-fghij 1/1 Running 0 3m
|
||||
web-7d8f9c6b5d-klmno 1/1 Running 0 8s
|
||||
```
|
||||
|
||||
Back to 2/2 — desired state restored, entirely without human intervention.
|
||||
|
||||
## Why this matters beyond the demo
|
||||
|
||||
This exact mechanism is what makes a node crash, a Pod eviction, or a container being
|
||||
OOM-killed something Kubernetes recovers from automatically instead of something that pages a
|
||||
human at 2am. The controller loop doesn't know or care WHY a Pod disappeared — crash, manual
|
||||
delete, node failure — it only ever asks "does reality match desired state right now?" and acts
|
||||
if not. Understanding this one loop explains a huge amount of Kubernetes' behavior that would
|
||||
otherwise look like unrelated special cases.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Run this live rather than only showing the captured output above — have
|
||||
students watch `kubectl get pods -n demo --watch` in one terminal while you delete a Pod from
|
||||
another. Watching the new Pod name/age appear in real time lands the "this is a NEW Pod, not a
|
||||
restart" point far better than static text.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; 5 minutes as a self-paced read.
|
||||
|
||||
**Common misconception to correct:** students frequently describe this as "Kubernetes restarted
|
||||
the Pod." Correct this explicitly — a deleted Pod is gone permanently; a *new* Pod object was
|
||||
created to satisfy the replica count. This distinction matters later when debugging (the new
|
||||
Pod has zero history from the old one — no old logs, no old restart count).
|
||||
|
||||
**Optional extension:** ask what would happen if you deleted the Deployment itself instead of
|
||||
one Pod (answer: all of its Pods are deleted and NOT replaced, because the thing declaring
|
||||
desired state is now gone — a good way to reinforce that the Deployment, not any individual
|
||||
Pod, is the actual source of truth).
|
||||
@@ -0,0 +1,48 @@
|
||||
# Knowledge Check: The Kubernetes Mental Model
|
||||
|
||||
Answer each question yourself before reading the answer that follows it.
|
||||
|
||||
---
|
||||
|
||||
**1. You run `kubectl delete pod` on one Pod belonging to a Deployment with `replicas: 3`. A
|
||||
few seconds later, `kubectl get pods` shows 3 Pods again, but one has a different name than any
|
||||
of the original three. What actually happened?**
|
||||
|
||||
*Answer:* The deleted Pod is gone permanently. The Deployment's controller noticed actual state
|
||||
(2 Pods) no longer matched desired state (3 Pods) and created ONE new Pod to close the gap. It
|
||||
is not the old Pod restarting — it's a brand new object.
|
||||
|
||||
---
|
||||
|
||||
**2. A Deployment has `replicas: 3`, and there are 5 nodes in the cluster. Which of these
|
||||
decides WHICH node each of the 3 Pods actually lands on?**
|
||||
|
||||
*Answer:* The scheduler (part of the control plane), not the person who wrote the Deployment
|
||||
YAML and not the nodes themselves. You declare desired state; the scheduler makes the placement
|
||||
decision based on available resources and any placement rules configured.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: `kubectl` communicates directly with the kubelet on whichever node your
|
||||
Pod happens to be running on.**
|
||||
|
||||
*Answer:* False. `kubectl` only ever talks to the API server (part of the control plane). The
|
||||
API server is the single entry point for every read and write — it never bypasses the control
|
||||
plane to reach a node directly for a normal `get`/`apply`/`describe` call.
|
||||
|
||||
---
|
||||
|
||||
**4. Why is Kubernetes YAML described as "declarative" rather than a script?**
|
||||
|
||||
*Answer:* A script describes a sequence of steps to execute once. Kubernetes YAML describes an
|
||||
END STATE you want to exist ("3 replicas of this container should be running") — a controller
|
||||
loop then continuously works to keep reality matching that description, correcting drift
|
||||
whenever it appears, indefinitely, not just once at apply time.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Ask these aloud in a live classroom before revealing answers — question 1 in
|
||||
particular tends to surface the "it restarted" misconception the demonstration activity should
|
||||
have already addressed; if it hasn't landed, this is the moment to circle back.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+57
@@ -0,0 +1,57 @@
|
||||
# Guided Exercise: Planning a Namespace Layout
|
||||
|
||||
This is a paper-and-thinking exercise — work through it before the Lab, not during it.
|
||||
|
||||
## The scenario
|
||||
|
||||
You're setting up a small Kubernetes cluster for a class project with three teams (`frontend`,
|
||||
`backend`, `data`), each running their own set of Pods, and you also need a shared area for
|
||||
tools everyone uses (a shared logging dashboard, for example).
|
||||
|
||||
## Your task
|
||||
|
||||
Sketch out, on paper or in a scratch file:
|
||||
|
||||
1. What Namespaces would you create, and why? (Hint: think about who needs to see what, and
|
||||
whether any team's mistake — e.g. deleting the wrong Pod by accident — should be able to
|
||||
affect another team.)
|
||||
2. If `frontend` team members should be able to manage their own Pods but NOT touch `backend`'s
|
||||
or `data`'s, what does that suggest about how Namespaces relate to access control? (You'll
|
||||
learn the actual mechanism, RBAC, in ROOT 09 — for now just reason about why Namespace
|
||||
boundaries make that kind of restriction possible at all.)
|
||||
3. Would you put the shared logging dashboard in one of the three team Namespaces, or somewhere
|
||||
else? Justify your answer.
|
||||
4. If two teams both need to run something they each call `web`, what would happen if they were
|
||||
in the SAME Namespace? What changes if they're in separate Namespaces?
|
||||
|
||||
## A reasonable answer to check yourself against
|
||||
|
||||
1. Three Namespaces (`frontend`, `backend`, `data`) — one per team, so each team's objects are
|
||||
isolated from the others and a mistake in one Namespace (deleting the wrong Pod, applying a
|
||||
bad manifest) can't accidentally affect a different team's workloads.
|
||||
2. Namespace boundaries are what RBAC rules can be scoped TO — "this person can manage Pods in
|
||||
the `frontend` Namespace" is a real, common permission model. Without separate Namespaces,
|
||||
you'd have to distinguish access by resource name pattern instead of by boundary, which is
|
||||
far more error-prone.
|
||||
3. A separate Namespace (e.g. `shared-tools` or `platform`) — it isn't owned by any one team,
|
||||
and mixing it into a team's Namespace would make it unclear who's responsible for it or
|
||||
accidentally couple its lifecycle to that team's.
|
||||
4. In the SAME Namespace, object names must be unique — a second `web` Deployment would either
|
||||
fail to create or overwrite the first, depending on how it's applied. In SEPARATE
|
||||
Namespaces, both `frontend/web` and `backend/web` can coexist without conflict — the
|
||||
Namespace is part of what makes a name unique, not just the name itself.
|
||||
|
||||
If your answer differed meaningfully from this, re-read the lesson's Namespace section before
|
||||
continuing to the Lab.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Good as a 2-3 person small-group discussion in a live classroom before
|
||||
revealing the reasonable answer — question 4 in particular often produces a genuine "oh" moment
|
||||
once students realize name uniqueness is Namespace-scoped, not cluster-wide.
|
||||
|
||||
**Approximate timing:** 10 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what would need to be true for the `data` team to be ALLOWED to read
|
||||
(but not modify) something in `shared-tools` — a good bridge to RBAC's read vs. write verbs,
|
||||
covered properly in ROOT 09.
|
||||
@@ -0,0 +1,44 @@
|
||||
# Launch Lab: Working with Namespaces
|
||||
|
||||
Time to apply the mental model for real — you'll create an actual isolated Namespace and deploy
|
||||
into it, in your own disposable Lab environment.
|
||||
|
||||
## What you're doing
|
||||
|
||||
You'll create a Namespace (`team-alpha`) and deploy an application into it with a specific
|
||||
replica count — directly applying the "isolated area within one cluster" idea from the guided
|
||||
exercise, and the desired-state model from the lesson (once you declare `replicas: 2`, that
|
||||
becomes the standing desired state for that Deployment, enforced continuously).
|
||||
|
||||
## Before you start
|
||||
|
||||
Think back to the demonstration: once your Deployment exists with its desired replica count,
|
||||
Kubernetes will keep it there on its own. You don't need to "check back" and manually fix
|
||||
anything — if you want to prove this to yourself, try deleting one of your Pods after you've
|
||||
finished the task and watch what happens.
|
||||
|
||||
## Your task in this Lab
|
||||
|
||||
Create the Namespace, then deploy the application inside it with the exact replica count
|
||||
specified. Use `kubectl get namespace` and `kubectl get deployment -n <namespace>` to confirm
|
||||
your work — the same verification pattern you'll use constantly throughout ROOT_.
|
||||
|
||||
This Lab has its own in-context hints if you get stuck. Try working from the lesson and guided
|
||||
exercise first — the commands you need are exactly the ones described there.
|
||||
|
||||
When you're done, return to this module to close out with a reflection on why this isolation
|
||||
matters beyond just "keeping things organized."
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Classroom flow:** This is most students' first real `kubectl apply`/`create` in ROOT_ beyond
|
||||
whatever pilot exposure they've had — a good moment to circulate and check that basic
|
||||
`kubectl` command syntax isn't the blocker (as opposed to the Namespace concept itself, which
|
||||
the guided exercise should have already addressed).
|
||||
|
||||
**Approximate timing:** 10-15 minutes.
|
||||
|
||||
**Common mistake to watch for:** creating the Deployment in the DEFAULT namespace instead of
|
||||
`team-alpha` (forgetting `-n team-alpha` or the namespace field in the manifest) — this is a
|
||||
very common first mistake and a good real-world illustration of why Namespace isolation only
|
||||
works if you're actually specifying it every time.
|
||||
@@ -0,0 +1,44 @@
|
||||
# Reflection: Why Desired State Matters in Production
|
||||
|
||||
You just created an isolated Namespace and a Deployment with a declared replica count. Take a
|
||||
few minutes to connect this to why it matters beyond passing this Lab's validation.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. If your `team-alpha` Namespace were a real team's production environment, what's one mistake
|
||||
in a DIFFERENT namespace that Namespace isolation would have protected you from?
|
||||
2. You declared `replicas: 2` rather than manually creating two Pods. What would you have had
|
||||
to do manually, over and over, if Kubernetes only supported creating individual Pods with no
|
||||
concept of desired state?
|
||||
3. Think of one non-Kubernetes system you've used (a thermostat, a spell-checker, an
|
||||
auto-save feature) that also works by continuously comparing "what is" to "what should be."
|
||||
What made you think of it?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**Namespaces are your first real production safety boundary, and they're free.** A
|
||||
misconfigured `kubectl delete` scoped to the wrong Namespace is a common, entirely preventable
|
||||
class of incident — teams that consistently namespace their environments (dev/staging/prod, or
|
||||
per-team) limit the blast radius of almost any operator mistake to just that Namespace. This
|
||||
costs nothing to set up and prevents a real, common category of outage.
|
||||
|
||||
**Desired state is what makes self-healing possible at all.** Every "the platform recovered on
|
||||
its own" story you'll hear about Kubernetes in production traces back to exactly the mechanism
|
||||
you just watched in the demonstration: a controller comparing actual state to desired state and
|
||||
correcting the gap. Understanding this loop is what lets you predict Kubernetes' behavior
|
||||
instead of being surprised by it.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** use all three reflection questions as discussion
|
||||
prompts. Question 3 in particular tends to produce good analogies that make the abstract
|
||||
"controller loop" concept concrete and memorable.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to name one place in their own life (not computing) where
|
||||
"declare the goal and let a system enforce it" would work better than "manually do the thing
|
||||
every time" — a genuinely reflective prompt that reinforces the mental model beyond Kubernetes
|
||||
specifically.
|
||||
+49
@@ -0,0 +1,49 @@
|
||||
# Building kubectl Fluency: Why Twenty Small Reps Beat One Big Lecture
|
||||
|
||||
## What this module is, and why it looks different from the last one
|
||||
|
||||
ROOT 05's mental model module (m00) taught you the CONCEPT of desired state and gave you your
|
||||
first `kubectl` commands. This module does something deliberately different: it's a breadth
|
||||
drill across the everyday `kubectl` vocabulary you'll type dozens of times a day as a working
|
||||
Kubernetes engineer — labels, JSONPath, dry-run manifests, patch, field selectors, sort-by,
|
||||
diff, init containers, static Pods, and more.
|
||||
|
||||
There's no single narrative thread connecting these twenty tasks, and that's intentional. Real
|
||||
`kubectl` fluency isn't one big insight — it's having enough of these small, specific commands
|
||||
already in muscle memory that you're not stopping to look up syntax while you're mid-incident.
|
||||
Each task below explains the specific concept it covers in its own description; your job is to
|
||||
work through them, not to memorize a lecture first.
|
||||
|
||||
## How to approach this module
|
||||
|
||||
- **Don't try to do them in one sitting if you don't need to.** These are independent — skip
|
||||
around, come back to ones you found awkward, and don't feel obligated to treat this as one
|
||||
continuous session.
|
||||
- **Read each task's own explanation before reaching for the hint.** Every one of these
|
||||
twenty tasks includes real context in its description — that context IS the teaching content
|
||||
for that specific command. The hint is there for when you're stuck, not as the first thing to
|
||||
read.
|
||||
- **Notice which commands you reach for confidently versus which ones you have to think about.**
|
||||
That gap is exactly what the reflection at the end of this module asks you to name.
|
||||
- **This breadth matters more than it looks like it does.** `kubectl explain`, `--dry-run`,
|
||||
JSONPath output, and `kubectl patch` in particular are commands experienced engineers use
|
||||
constantly and newer engineers often don't know exist at all — this module exists specifically
|
||||
to close that gap early.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Lesson goal:** Frame this module correctly before students hit it — it's a breadth-practice
|
||||
bank, not a single-concept lesson, and the individual task descriptions carry the teaching
|
||||
content. Without this framing, students sometimes expect a lecture and are confused when they
|
||||
land directly on twenty short tasks.
|
||||
|
||||
**Approximate timing:** 5 minutes to introduce; the module itself runs long (20 tasks) and is
|
||||
meant to be worked through at each student's own pace, live or between sessions.
|
||||
|
||||
**Facilitation for a live cohort:** Consider live-demoing 2-3 of the less obvious ones
|
||||
(`kubectl explain`, JSONPath output, `kubectl patch`) rather than all twenty — the goal is to
|
||||
unblock students on the commands they're least likely to have seen before, not to walk through
|
||||
every task.
|
||||
|
||||
**Expected takeaway:** Kubernetes fluency is built from a large vocabulary of small, reliable
|
||||
commands, not just conceptual understanding — this module is where that vocabulary gets built.
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
# Reflection: Naming Your Gaps While They're Still Cheap to Close
|
||||
|
||||
You just worked through twenty independent `kubectl` tasks. Before moving on, take a few
|
||||
minutes to consolidate what actually stuck versus what you'd need to look up again.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Which THREE of these twenty commands do you feel genuinely confident using again right now,
|
||||
without a hint? Name them specifically.
|
||||
2. Which ONE command surprised you the most — either because you didn't know it existed, or
|
||||
because it did something different than you expected?
|
||||
3. `kubectl explain`, `kubectl patch`, and JSONPath output are commands a lot of engineers never
|
||||
learn early and end up rediscovering the hard way, mid-incident, years later. Now that you've
|
||||
used all three, where do you think you'd actually reach for one of them in a real debugging
|
||||
situation?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**Fluency gaps are cheap to close now and expensive to close later.** Right now, not knowing
|
||||
`kubectl patch` costs you a few minutes rereading this module. In a live incident six months
|
||||
from now, not knowing it costs you minutes you don't have while a Service is down. Naming your
|
||||
gaps here, while the stakes are zero, is how you make sure you close them before they matter.
|
||||
|
||||
**Breadth practice is not busywork.** It's easy to dismiss twenty small tasks as less valuable
|
||||
than one big project — but the twenty small tasks are what make the big project fast to execute
|
||||
later. This is the same discipline you'll rely on for the rest of ROOT_: build the vocabulary
|
||||
now, apply it under pressure later.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 3 works well as a round-robin — go
|
||||
person to person and have each name a different command, so the group collectively surfaces
|
||||
most of the twenty rather than converging on the same one or two obvious answers.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Expected takeaway:** Students should leave this module able to name their own remaining gaps
|
||||
specifically, not just feel generally "done" with kubectl basics.
|
||||
+81
@@ -0,0 +1,81 @@
|
||||
# Why Deployments Exist & How a Rolling Update Progresses
|
||||
|
||||
## The problem a bare Pod doesn't solve
|
||||
|
||||
A Pod, on its own, has no self-healing and no concept of "there should be more than one of
|
||||
these." If a Pod crashes, nothing recreates it. If you need 3 copies for capacity, you'd have to
|
||||
create 3 separately-named Pods by hand and keep track of all of them yourself. If you need to
|
||||
update the container image, you'd have to delete and recreate every one of them manually,
|
||||
almost certainly causing a visible outage while you do it.
|
||||
|
||||
**A Deployment solves all three problems at once**: it declares how many replicas of a Pod
|
||||
template should exist, keeps that count true continuously (the exact desired-state mechanism
|
||||
from ROOT 05), and knows how to change the Pod template over time — a version upgrade — without
|
||||
just deleting everything and starting over.
|
||||
|
||||
## ReplicaSets: the layer underneath, briefly
|
||||
|
||||
You'll rarely touch a ReplicaSet directly, but understanding it clarifies what a Deployment
|
||||
actually does. A Deployment doesn't manage Pods directly — it manages a ReplicaSet, which is
|
||||
what actually keeps N Pods matching a label selector running. When you update a Deployment's
|
||||
Pod template (like changing the image), the Deployment creates a NEW ReplicaSet with the new
|
||||
template and gradually shifts replicas from the old ReplicaSet to the new one. The old
|
||||
ReplicaSet isn't deleted — it's scaled to zero and kept around, which is exactly what makes
|
||||
rollback possible (more on that below).
|
||||
|
||||
## How a rolling update actually progresses
|
||||
|
||||
By default, a Deployment updates using the `RollingUpdate` strategy: it doesn't take all old
|
||||
Pods down and bring all new ones up at once (that would cause an outage). Instead, it
|
||||
incrementally creates new-version Pods and removes old-version ones, always keeping enough
|
||||
capacity available:
|
||||
|
||||
```
|
||||
Before: [v1] [v1] [v1]
|
||||
Step 1: [v1] [v1] [v1] [v2] <- one new Pod created
|
||||
Step 2: [v1] [v1] [v2] <- one old Pod removed once new one is Ready
|
||||
Step 3: [v1] [v1] [v2] [v2]
|
||||
Step 4: [v1] [v2] [v2]
|
||||
...
|
||||
After: [v2] [v2] [v2]
|
||||
```
|
||||
|
||||
Crucially, a new Pod only counts as "up" for the purposes of proceeding once it passes its
|
||||
readiness probe (see ROOT 10's gold-standard module) — this is exactly why a misconfigured
|
||||
readiness probe can make a rollout hang indefinitely instead of failing fast: Kubernetes is
|
||||
correctly waiting for a "ready" signal that will never come.
|
||||
|
||||
## Rollback: what it actually restores
|
||||
|
||||
`kubectl rollout undo` doesn't magically know how to "undo" arbitrary changes — it works because
|
||||
of the old ReplicaSet still sitting around at zero replicas, from the mechanism described above.
|
||||
Rolling back means: scale the OLD ReplicaSet back up and the current (broken) one back down,
|
||||
using the exact same rolling mechanism as a forward update. This is why rollback is fast and
|
||||
safe — it's not a special operation, it's the same well-tested rolling-update machinery running
|
||||
in reverse, against a Pod template that's already known to have worked before.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read, 10-12 minutes live with the ReplicaSet diagram.
|
||||
|
||||
**Demo plan:** Draw (or show) the ReplicaSet-swap diagram above on a whiteboard before the
|
||||
demonstration activity — students who see the "old ReplicaSet scaled to zero, not deleted"
|
||||
detail up front connect rollback to rollout much faster.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often think rollback "restores a backup" or reverts a Git-style diff. Correct this:
|
||||
it's scaling an existing (still-present) ReplicaSet back up, nothing is being restored from
|
||||
storage.
|
||||
- "Rolling update" is sometimes assumed to mean "one Pod at a time, always." The actual pace is
|
||||
controlled by `maxSurge`/`maxUnavailable` (not covered in depth here, but worth naming so
|
||||
students don't over-generalize from the simplified diagram).
|
||||
|
||||
**Discussion question:** "Why does a rolling update need readiness probes to work correctly,
|
||||
specifically?" (Answer: without a readiness signal, Kubernetes has no way to know when a new
|
||||
Pod is actually safe to receive traffic before removing an old one — it would either guess via a
|
||||
fixed delay, unreliable, or not wait at all, unsafe.)
|
||||
|
||||
**Optional extension:** ask what would happen to an in-progress rollout if the new image simply
|
||||
doesn't exist (a typo in the tag) — a good bridge to real troubleshooting: the rollout stalls
|
||||
indefinitely with `ImagePullBackOff` on the new Pods, never completing and never rolling back on
|
||||
its own.
|
||||
+77
@@ -0,0 +1,77 @@
|
||||
# Demonstration: Watching a Rollout, Good and Bad
|
||||
|
||||
## A healthy rollout
|
||||
|
||||
A Deployment `app-v1` running `nginx:1.24` at 3 replicas gets updated to `nginx:1.25`:
|
||||
|
||||
```
|
||||
$ kubectl set image deployment/app-v1 nginx=nginx:1.25
|
||||
deployment.apps/app-v1 image updated
|
||||
|
||||
$ kubectl rollout status deployment/app-v1
|
||||
Waiting for deployment "app-v1" rollout to finish: 1 out of 3 new replicas have been updated...
|
||||
Waiting for deployment "app-v1" rollout to finish: 2 out of 3 new replicas have been updated...
|
||||
Waiting for deployment "app-v1" rollout to finish: 1 old replicas are pending termination...
|
||||
deployment "app-v1" successfully rolled out
|
||||
```
|
||||
|
||||
`kubectl rollout status` blocks and reports exactly the incremental progress described in the
|
||||
lesson — this is your primary tool for watching (or scripting a wait on) a rollout, rather than
|
||||
repeatedly polling `kubectl get pods` yourself.
|
||||
|
||||
## Checking revision history
|
||||
|
||||
```
|
||||
$ kubectl rollout history deployment/app-v1
|
||||
REVISION CHANGE-CAUSE
|
||||
1 <none>
|
||||
2 <none>
|
||||
```
|
||||
|
||||
Revision 1 is the original ReplicaSet (now scaled to zero, still present); revision 2 is the
|
||||
current one. This history is exactly what `kubectl rollout undo` uses.
|
||||
|
||||
## A stalled rollout
|
||||
|
||||
Now imagine the same update, but to a typo'd tag, `nginx:1.25-typo`, which doesn't exist:
|
||||
|
||||
```
|
||||
$ kubectl rollout status deployment/app-v1
|
||||
Waiting for deployment "app-v1" rollout to finish: 1 out of 3 new replicas have been updated...
|
||||
```
|
||||
|
||||
...and it just sits there. `kubectl get pods` shows why:
|
||||
|
||||
```
|
||||
$ kubectl get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
app-v1-7d8f9c6b5d-abcde 1/1 Running 0 5m
|
||||
app-v1-7d8f9c6b5d-fghij 1/1 Running 0 5m
|
||||
app-v1-9c5b7d8f6b-klmno 0/1 ImagePullBackOff 0 30s
|
||||
```
|
||||
|
||||
The new Pod can never become Ready because its image can't even be pulled — so the rollout
|
||||
correctly refuses to proceed further (removing more old Pods would reduce capacity for no
|
||||
benefit, since the replacement will never come up). Nothing is "broken" in the platform; the
|
||||
platform is behaving exactly correctly given bad input.
|
||||
|
||||
**The fix here is `kubectl rollout undo`, not waiting longer** — this exact scenario is what
|
||||
`deployment-rollback`'s Lab, later in ROOT_, gives you real hands-on practice with.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** If time allows, run the healthy rollout live against a throwaway Deployment,
|
||||
then intentionally break the second one with a bad tag so students see `ImagePullBackOff` occur
|
||||
in real time rather than only in static text.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; 5-7 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students sometimes assume a stalled rollout will
|
||||
eventually time out and auto-rollback on its own. By default it will not — a Deployment has no
|
||||
automatic rollback trigger; a human (or automation) has to run `kubectl rollout undo` (or fix
|
||||
the underlying issue) explicitly. This is worth stating directly since it's a genuine and
|
||||
consequential production gotcha.
|
||||
|
||||
**Optional extension:** ask what `kubectl rollout status --timeout=2m` would change about this
|
||||
scenario (answer: the command itself would exit with an error after 2 minutes instead of
|
||||
blocking forever — useful in CI/CD pipelines that need to fail a deploy step rather than hang).
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
# Knowledge Check: Deployments & Rollouts
|
||||
|
||||
---
|
||||
|
||||
**1. A rolling update to a new image version has been "in progress" for 20 minutes with no
|
||||
error message, and `kubectl rollout status` never returns. What's the first thing you should
|
||||
check, and why?**
|
||||
|
||||
*Answer:* `kubectl get pods` to see the new Pod's status, then `kubectl describe pod` on it if
|
||||
it isn't `Running`. A stalled rollout with no explicit error almost always means a new Pod can't
|
||||
become Ready — commonly `ImagePullBackOff` (bad image/tag) or a failing readiness probe. The
|
||||
rollout isn't broken; it's correctly refusing to proceed until the new Pod proves healthy.
|
||||
|
||||
---
|
||||
|
||||
**2. True or false: `kubectl rollout undo` restores your application from a backup of its
|
||||
previous state.**
|
||||
|
||||
*Answer:* False. It scales the previous (still-present, just scaled-to-zero) ReplicaSet back up
|
||||
and the current one back down — the same rolling mechanism as a forward update, just pointed at
|
||||
an older, already-proven-working Pod template. Nothing is being restored from storage.
|
||||
|
||||
---
|
||||
|
||||
**3. A Deployment has `replicas: 3`. You need it to handle more traffic. Which is the correct
|
||||
action: edit the Pod template's resource limits, or change the replica count? Justify your
|
||||
answer for a stateless web application.**
|
||||
|
||||
*Answer:* Change the replica count (scale out) for a stateless application handling more
|
||||
concurrent traffic — more identical copies sharing the load. Changing resource limits affects
|
||||
how much each individual Pod can use, not how many Pods exist; it doesn't add capacity for more
|
||||
concurrent requests the way scaling does.
|
||||
|
||||
---
|
||||
|
||||
**4. Why does a Deployment keep an old, scaled-to-zero ReplicaSet around instead of deleting it
|
||||
immediately after a successful rollout?**
|
||||
|
||||
*Answer:* That scaled-down ReplicaSet is exactly what makes fast, safe rollback possible later —
|
||||
deleting it immediately would mean a "rollback" would have to rebuild the old Pod template from
|
||||
scratch (or fail entirely) instead of simply scaling existing, already-defined Pods back up.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 1 is the most important one to discuss out loud — it's the exact
|
||||
diagnostic instinct students will need repeatedly in ROOT 10.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+54
@@ -0,0 +1,54 @@
|
||||
# Guided Exercise: Planning a Safe Rollout Strategy
|
||||
|
||||
Paper-and-thinking exercise — work through this before the Lab.
|
||||
|
||||
## The scenario
|
||||
|
||||
Your team runs `api-server`, a Deployment with 6 replicas serving live traffic. You need to
|
||||
ship a new image version. The team lead asks: "what could go wrong, and how would we know
|
||||
before it takes down the whole service?"
|
||||
|
||||
## Your task
|
||||
|
||||
Work through these questions:
|
||||
|
||||
1. If the new image has a bug that crashes the container immediately on startup, at what point
|
||||
in the rolling update would you expect to notice — after 1 new Pod, after all 6, or somewhere
|
||||
in between? Why?
|
||||
2. What's the difference in blast radius between that crash-on-startup bug and a subtler bug
|
||||
that only fails under load, several minutes after a Pod starts serving traffic?
|
||||
3. Given your answer to question 2, what's the practical value of watching `kubectl rollout
|
||||
status` and `kubectl get pods` DURING a rollout, rather than only checking after it reports
|
||||
"successfully rolled out"?
|
||||
4. If you noticed a problem partway through, what's the single command that gets you back to
|
||||
the last known-good state fastest?
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. After the FIRST new Pod — a crash-on-startup bug means the new Pod never becomes Ready
|
||||
(`CrashLoopBackOff`), so the rollout stalls immediately, well before more old Pods are
|
||||
removed. This is actually the SAFEST kind of bug, because the blast radius stays tiny.
|
||||
2. A crash-on-startup bug is naturally self-limiting (the rollout stalls itself, as above). A
|
||||
bug that only appears under load is NOT self-limiting the same way — by the time it's
|
||||
noticeable, several old (working) replicas may already have been replaced, meaning a larger
|
||||
fraction of real traffic is already hitting the broken version.
|
||||
3. Watching during the rollout, not just checking the final "successfully rolled out" message,
|
||||
is what lets you catch the load-dependent case in question 2 early — waiting for the final
|
||||
status alone would let the rollout complete entirely (100% on the broken version) before you
|
||||
noticed anything wrong.
|
||||
4. `kubectl rollout undo deployment/api-server` — assuming the previous ReplicaSet is still
|
||||
present (the default), this is faster than trying to hand-diagnose and hot-fix the new
|
||||
version under pressure.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2's distinction (self-limiting vs. not) is the core insight worth
|
||||
making sure lands — it's why "the rollout said success" is necessary but not sufficient
|
||||
evidence that a deploy actually went well.
|
||||
|
||||
**Approximate timing:** 10-15 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what monitoring/observability signal (beyond `kubectl` output) a
|
||||
real team would want during a rollout to catch the load-dependent case even faster (a good
|
||||
forward pointer to ROOT 10's observability themes, without needing to build any actual
|
||||
monitoring infrastructure here).
|
||||
+39
@@ -0,0 +1,39 @@
|
||||
# Launch Lab: Perform a Rolling Update
|
||||
|
||||
Time to run a real rolling update against a real Deployment in your disposable Lab.
|
||||
|
||||
## What you're doing
|
||||
|
||||
A Deployment `app-v1` is running an older image version. Your task is to update it to a newer
|
||||
version and confirm every replica ends up running the new image — exactly the mechanism from
|
||||
the lesson and demonstration, on a real cluster this time.
|
||||
|
||||
## Before you start
|
||||
|
||||
Recall the demonstration: `kubectl rollout status` is your tool for watching progress rather
|
||||
than repeatedly polling `kubectl get pods` yourself, and a stalled rollout with no obvious error
|
||||
usually means a new Pod can't become Ready. If your update seems to hang, that's exactly the
|
||||
diagnostic instinct from the knowledge check to apply.
|
||||
|
||||
## Your task in this Lab
|
||||
|
||||
Update the Deployment's image and verify all replicas are running the new version. Use
|
||||
`kubectl rollout status` to confirm completion, not just a single `kubectl get pods` snapshot.
|
||||
|
||||
This Lab has its own in-context hints if you get stuck.
|
||||
|
||||
When you're done, return to this module for a short reflection on what "successfully rolled
|
||||
out" actually guarantees — and doesn't.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Classroom flow:** A good moment to have students keep `kubectl get pods --watch` open in a
|
||||
second terminal (if their Lab environment supports multiple panes) so they see the incremental
|
||||
old-Pod/new-Pod transition happen live, reinforcing the demonstration.
|
||||
|
||||
**Approximate timing:** 10-15 minutes.
|
||||
|
||||
**Common mistake to watch for:** running `kubectl edit deployment` and forgetting to save/apply
|
||||
correctly, or editing the wrong field (e.g. `replicas` instead of the container image) — a good
|
||||
moment to reinforce reading the task description precisely rather than pattern-matching to a
|
||||
similar-looking previous Lab.
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
# Reflection: Rollback Is a Plan, Not a Panic Button
|
||||
|
||||
You just completed a real rolling update. Take a few minutes to connect it to how this plays out
|
||||
in a real production team.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. "Successfully rolled out" (from `kubectl rollout status`) tells you every replica is Running
|
||||
and Ready. What does it NOT tell you about whether the new version is actually correct?
|
||||
2. If you were on-call and a rollout you just ran seemed to be causing problems, would you reach
|
||||
for `kubectl rollout undo` immediately or try to hot-fix the new version first? What would
|
||||
change your answer?
|
||||
3. Think back to the guided exercise's crash-on-startup vs. load-dependent distinction. Which
|
||||
one would a typical automated health check (like a readiness probe, ROOT 10) catch, and
|
||||
which one might slip through even with probes configured correctly?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**A rollout completing is not the same as a deploy succeeding.** "Every Pod is Running and
|
||||
Ready" only proves the new code started up and passed whatever checks are configured — it says
|
||||
nothing about whether the new version is functionally correct under real traffic. Real teams
|
||||
pair a rollout with actual monitoring (error rates, latency) before calling a deploy genuinely
|
||||
successful, not just a green `kubectl rollout status`.
|
||||
|
||||
**Rollback should be a rehearsed, low-drama action, not a last resort.** Teams that treat
|
||||
`kubectl rollout undo` as a normal, expected tool — something you reach for calmly the moment
|
||||
something looks wrong — recover from bad deploys in seconds. Teams that treat rollback as an
|
||||
admission of failure hesitate, try to hot-fix forward under pressure, and turn a small,
|
||||
recoverable mistake into a longer outage.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 2 is worth pushing on — many students'
|
||||
instinct is to try to fix forward, and it's worth surfacing WHY that instinct is often wrong
|
||||
under time pressure (unknown time-to-fix vs. known, fast rollback).
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to write the one-sentence Slack message they'd post to
|
||||
their team the moment they decide to roll back a bad deploy — reinforces that rollback is a
|
||||
normal, communicable operational action, not a private failure.
|
||||
+85
@@ -0,0 +1,85 @@
|
||||
# Jobs, CronJobs, and DaemonSets: The Other Workload Shapes
|
||||
|
||||
Everything in the previous module assumed a long-running application that should always have N
|
||||
replicas up. Not every workload looks like that. Kubernetes has three other primitives for
|
||||
genuinely different shapes of work — using a Deployment for any of these would be the wrong
|
||||
tool, not just a less elegant one.
|
||||
|
||||
## Job: run to completion, exactly once (or a fixed number of times)
|
||||
|
||||
A Job creates one or more Pods and considers itself done when they exit successfully — it does
|
||||
NOT restart a Pod that exits with success, and it DOES retry one that fails, up to a configured
|
||||
limit. This is fundamentally different from a Deployment, which expects its Pods to run forever
|
||||
and treats any exit as something to replace.
|
||||
|
||||
Use a Job for: a database migration, a batch data-processing task, a one-off cleanup script —
|
||||
anything with a defined "finished" state.
|
||||
|
||||
```yaml
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: migrate-db
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: migrate
|
||||
image: myapp:latest
|
||||
command: ["./migrate.sh"]
|
||||
restartPolicy: Never
|
||||
backoffLimit: 3
|
||||
```
|
||||
|
||||
## CronJob: a Job, on a schedule
|
||||
|
||||
A CronJob doesn't run Pods itself — it creates a new Job on a cron schedule (`schedule:
|
||||
"0 2 * * *"` for "every day at 2am"), and that Job behaves exactly as described above. Use a
|
||||
CronJob for anything that should run periodically and finish: nightly backups, scheduled
|
||||
reports, periodic cleanup.
|
||||
|
||||
A CronJob that reliably fails to complete (an image that always crashes, for example) will keep
|
||||
creating new Jobs on schedule, each failing the same way — this is a real, common way to
|
||||
silently exhaust cluster resources (job/pod history accumulating) if nobody's watching.
|
||||
|
||||
## DaemonSet: exactly one Pod per node, automatically
|
||||
|
||||
A DaemonSet doesn't declare a replica count at all — it ensures exactly one matching Pod runs
|
||||
on every node in the cluster (or every node matching a selector), and automatically adds a Pod
|
||||
when a new node joins. Use a DaemonSet for per-node infrastructure: a log-shipping agent, a
|
||||
node-level monitoring exporter, a network plugin component — things that fundamentally need to
|
||||
exist once per machine, not "N copies somewhere."
|
||||
|
||||
## The decision in one pass
|
||||
|
||||
| You need... | Use |
|
||||
|---|---|
|
||||
| Always-on, N identical copies, restarted forever | Deployment |
|
||||
| Run once (or a few times) until it finishes successfully | Job |
|
||||
| The above, but on a recurring schedule | CronJob |
|
||||
| Exactly one copy on every node | DaemonSet |
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live.
|
||||
|
||||
**Demo plan (if extending this PARTIAL module later):** live-create a Job with a command that
|
||||
deliberately fails once then succeeds (`backoffLimit: 2`), and watch `kubectl get pods` show
|
||||
the failed attempt alongside the eventual successful one — makes the retry-until-backoffLimit
|
||||
behavior concrete.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often assume a Job that finishes successfully gets cleaned up automatically. It
|
||||
doesn't by default — the completed Pod/Job objects remain until manually deleted or a
|
||||
`ttlSecondsAfterFinished` is configured (not covered here, worth a one-line mention if asked).
|
||||
- "DaemonSet" is sometimes confused with "one Pod, cluster-wide" (singular). It's the opposite —
|
||||
one Pod PER NODE, which can be many Pods total in a multi-node cluster.
|
||||
|
||||
**Discussion question:** "Why doesn't a Deployment support 'run once and be done'?" (Answer: a
|
||||
Deployment's whole contract is maintaining a standing replica count forever — repurposing it for
|
||||
one-shot work would fight that contract rather than use a primitive designed for it.)
|
||||
|
||||
**Optional extension:** ask what would go wrong if a CronJob's schedule ran more often than a
|
||||
single Job execution takes to complete (answer: overlapping Job runs, unless
|
||||
`concurrencyPolicy` is set to `Forbid` or `Replace` — a real gotcha worth naming even without
|
||||
building a full lab around it).
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
# Knowledge Check: Choosing the Right Workload Primitive
|
||||
|
||||
---
|
||||
|
||||
**1. You need to run a database migration script exactly once when a new version deploys. Which
|
||||
primitive fits, and why not a Deployment?**
|
||||
|
||||
*Answer:* A Job. A Deployment expects its Pods to run forever and replaces any that exit; a
|
||||
migration script is supposed to exit successfully when done, which a Deployment would treat as
|
||||
a failure to "fix" by restarting it — the wrong behavior for a task that's supposed to finish.
|
||||
|
||||
---
|
||||
|
||||
**2. A CronJob scheduled for "every 5 minutes" keeps creating Jobs that fail immediately due to
|
||||
a bad image tag. What actually happens over the next hour if nobody notices?**
|
||||
|
||||
*Answer:* A new failing Job (and its Pod) gets created roughly every 5 minutes, accumulating
|
||||
failed Job/Pod objects and consuming cluster resources — nothing stops this automatically. This
|
||||
is exactly the kind of slow, easy-to-miss resource exhaustion a real on-call engineer should
|
||||
watch for.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: a DaemonSet's Pod count scales with a `replicas` field, the same as a
|
||||
Deployment.**
|
||||
|
||||
*Answer:* False. A DaemonSet has no `replicas` field — its Pod count is implicitly "one per
|
||||
matching node," and it grows or shrinks automatically as nodes join or leave the cluster.
|
||||
|
||||
---
|
||||
|
||||
**4. You need a log-shipping agent that must run on every node, including any new node added
|
||||
later. Which primitive, and what would go wrong if you used a Deployment with a high replica
|
||||
count instead?**
|
||||
|
||||
*Answer:* A DaemonSet. A Deployment's replicas are placed by the scheduler wherever it decides
|
||||
is best — nothing guarantees one per node, and a newly-added node wouldn't automatically get a
|
||||
copy the way a DaemonSet guarantees.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2 is worth emphasizing — it's a genuinely realistic "how does a
|
||||
cluster quietly get into trouble" scenario, not just trivia.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+72
@@ -0,0 +1,72 @@
|
||||
# Demonstration: Watching a Job Actually Complete
|
||||
|
||||
## Creating the Job
|
||||
|
||||
```
|
||||
$ kubectl apply -f pi-job.yaml
|
||||
job.batch/pi-job created
|
||||
|
||||
$ kubectl get jobs
|
||||
NAME COMPLETIONS DURATION AGE
|
||||
pi-job 0/1 3s 3s
|
||||
|
||||
$ kubectl get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
pi-job-8x2kq 1/1 Running 0 3s
|
||||
```
|
||||
|
||||
The Job's own status (`0/1` completions) is tracked separately from its Pod's status — this
|
||||
matters because a Job can retry a FAILED Pod attempt without that looking like the healthy
|
||||
restart behavior you'd see on a Deployment.
|
||||
|
||||
## Once the Pod's process actually exits successfully
|
||||
|
||||
```
|
||||
$ kubectl get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
pi-job-8x2kq 0/1 Completed 0 8s
|
||||
|
||||
$ kubectl get jobs
|
||||
NAME COMPLETIONS DURATION AGE
|
||||
pi-job 1/1 7s 8s
|
||||
```
|
||||
|
||||
`STATUS: Completed` — not `Running`, not `CrashLoopBackOff`. This is the ONE status a
|
||||
long-running Deployment's Pod is never supposed to reach on its own; for a Job, it's success.
|
||||
The Job object itself now reports `1/1` and stops — nothing recreates this Pod, because
|
||||
"finished successfully" is the desired end state here, unlike a Deployment where exiting is
|
||||
always treated as something to fix.
|
||||
|
||||
## What a FAILING Job attempt looks like, briefly
|
||||
|
||||
```
|
||||
$ kubectl get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
bad-job-abc12 0/1 Error 0 5s
|
||||
bad-job-def34 0/1 Error 0 2s
|
||||
|
||||
$ kubectl describe job bad-job
|
||||
...
|
||||
Events:
|
||||
Warning BackoffLimitExceeded Job has reached the specified backoff limit
|
||||
```
|
||||
|
||||
Two DIFFERENT Pods (`abc12`, `def34`), not one Pod restarting twice — each failed attempt gets
|
||||
its own new Pod, up to `backoffLimit`. Once that limit is hit, the Job stops retrying and
|
||||
reports the failure via Events rather than trying forever.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Run the successful case live; showing the FAILING case is optional if time is
|
||||
short, but the `Completed` vs `Running`/`Error` distinction is worth pausing on regardless.
|
||||
|
||||
**Approximate timing:** 8 minutes live; 5 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students who've only seen Deployments sometimes expect
|
||||
`kubectl get pods` to show the Job's Pod disappear once it's done. It doesn't, by default — it
|
||||
stays as `Completed` until manually cleaned up or a TTL is configured, which is worth mentioning
|
||||
briefly (not a hands-on topic here).
|
||||
|
||||
**Optional extension:** ask what `kubectl get pods` would show for a CronJob's Pods across
|
||||
several scheduled runs (answer: one Completed Pod per past run, accumulating — the same
|
||||
"nothing auto-cleans this up" behavior, multiplied by the schedule).
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
# Guided Exercise: Choosing the Right Workload Primitive
|
||||
|
||||
Paper-and-thinking exercise before the Labs.
|
||||
|
||||
## The scenario
|
||||
|
||||
Your team needs four things running in the cluster:
|
||||
|
||||
1. A nightly database backup that must run at 2am and finish before morning.
|
||||
2. A log-shipping agent that must run on every node, including nodes added later.
|
||||
3. A one-time script to backfill a new database column, run once during this deploy.
|
||||
4. The main web application, always up, serving live traffic.
|
||||
|
||||
## Your task
|
||||
|
||||
For each of the four, name the correct primitive (Deployment, Job, CronJob, or DaemonSet) and
|
||||
answer: what would go wrong if you used a Deployment for it instead?
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. **CronJob.** A Deployment has no concept of "run on a schedule" — you'd need external
|
||||
tooling to trigger it repeatedly, reinventing what CronJob already does. Using a bare Job
|
||||
would run it once; CronJob is what recreates that Job on schedule.
|
||||
2. **DaemonSet.** A Deployment's replica count is a number, not "one per node" — nothing
|
||||
guarantees even coverage, and a newly added node wouldn't automatically get a copy the way a
|
||||
DaemonSet guarantees.
|
||||
3. **Job.** A Deployment expects its Pod to run forever; a script that's SUPPOSED to exit when
|
||||
done would be treated as a crash and restarted forever, which is wrong for a one-time task.
|
||||
4. **Deployment.** This is the actual always-on, N-replicas case the other three are explicitly
|
||||
NOT designed for.
|
||||
|
||||
## Self-check
|
||||
|
||||
If you answered "Job" for #4 or "Deployment" for #1-3, revisit the lesson's decision table
|
||||
before the Labs — getting this choice right the first time avoids building something that
|
||||
technically runs but behaves wrong in a way that's easy to miss until it causes a real problem
|
||||
(a backup CronJob accidentally deployed as a Job that only ever runs once, for example).
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Good as a quick individual exercise before revealing the answer — the "what
|
||||
would go wrong" framing forces reasoning about the CONSEQUENCE of a wrong choice, not just
|
||||
recall of a definition.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live with discussion; 8 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what would need to be true for #1 (the nightly backup) to actually
|
||||
be SAFE to run as a plain Job triggered manually once, instead of a CronJob — a good check that
|
||||
students understand CronJob's value is specifically the SCHEDULING, not something inherent to
|
||||
backups.
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
# Reflection: The Right Primitive Is a Correctness Decision, Not a Style Choice
|
||||
|
||||
You just built a one-shot Job and a scheduled CronJob for real. Take a few minutes to connect
|
||||
this to production thinking.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. If your team accidentally deployed the database backup script as a Deployment instead of a
|
||||
CronJob, what's the FIRST symptom you'd likely notice — and would it look like a bug in the
|
||||
backup script itself, or something else?
|
||||
2. A CronJob that fails silently every night for a week is a real, common incident pattern. What
|
||||
would you want monitored to catch this quickly?
|
||||
3. Why does "it ran once and worked" not prove a CronJob is configured correctly?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**Workload-primitive mistakes often look like application bugs at first.** A backup script
|
||||
"crashing" every few minutes on a Deployment isn't a bug in the script — the script did exactly
|
||||
what it was told to do (exit when finished) and the Deployment is "correctly" restarting it
|
||||
according to its own contract. Recognizing "wrong primitive," not "buggy code," as the root
|
||||
cause saves real debugging time.
|
||||
|
||||
**A schedule you never verify is a schedule you can't trust.** A CronJob failing every single
|
||||
run since deployment is easy to miss if nobody checks — unlike a Deployment's crash-looping
|
||||
Pods, which are visible in `kubectl get pods` by default, a CronJob's failed history requires
|
||||
someone to actually look at `kubectl get jobs` or the CronJob's own status. Real teams monitor
|
||||
CronJob success/failure explicitly rather than assuming silence means success.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 1 is worth pushing on — many students'
|
||||
first instinct is to debug the "crashing" script rather than question the workload type it's
|
||||
running under.
|
||||
|
||||
**Approximate timing:** 8 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to name one alerting rule they'd add for a critical nightly
|
||||
CronJob (e.g. "alert if no successful completion recorded in the last 25 hours").
|
||||
+84
@@ -0,0 +1,84 @@
|
||||
# Externalizing Configuration: ConfigMaps & Secrets
|
||||
|
||||
## The problem: hardcoded configuration doesn't scale
|
||||
|
||||
If your application's database URL, log level, or feature flags are baked directly into its
|
||||
container image, changing any of them means rebuilding and redeploying the image — for a
|
||||
one-line config change. Worse, the SAME image can't be reused across dev/staging/production if
|
||||
each environment needs different settings compiled in.
|
||||
|
||||
Kubernetes solves this by letting you inject configuration into a container at RUNTIME,
|
||||
separately from the image itself — the same image, different config per environment.
|
||||
|
||||
## ConfigMap: configuration that isn't secret
|
||||
|
||||
A **ConfigMap** holds arbitrary key/value configuration data — log levels, feature flags, URLs,
|
||||
whole config files. It's a plain Kubernetes object, readable by anyone with API access to that
|
||||
Namespace, so it's for configuration that ISN'T sensitive.
|
||||
|
||||
Two ways to consume a ConfigMap in a Pod:
|
||||
|
||||
- **As environment variables** — each key becomes an env var the application reads normally.
|
||||
- **As a mounted volume** — each key becomes a FILE inside the container, useful for
|
||||
applications that expect to read configuration from disk rather than environment variables.
|
||||
|
||||
```yaml
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: app-env
|
||||
```
|
||||
|
||||
## Secret: the same idea, for sensitive data
|
||||
|
||||
A **Secret** has the identical shape and consumption pattern as a ConfigMap (env vars or a
|
||||
mounted volume) but is meant for sensitive values — passwords, API keys, tokens. Kubernetes
|
||||
stores Secret data base64-ENCODED, not encrypted — this is an important distinction:
|
||||
base64 is trivially reversible, so a Secret's real protection comes from Kubernetes' RBAC
|
||||
(who can even read Secret objects at all, ROOT 09) and, in a properly configured cluster,
|
||||
encryption at rest — not from the encoding itself.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: app-secret
|
||||
type: Opaque
|
||||
data:
|
||||
api-key: c3VwZXJzZWNyZXQ= # base64 of "supersecret" — NOT encrypted
|
||||
```
|
||||
|
||||
## Why the same mechanism, two different objects
|
||||
|
||||
You might ask why Kubernetes doesn't just have one "Config" object with a "sensitive" flag.
|
||||
Keeping them separate makes the DISTINCTION visible and enforceable — RBAC rules can grant
|
||||
"read ConfigMaps" without granting "read Secrets," and a quick `kubectl get configmaps` never
|
||||
accidentally exposes something that should have been a Secret. The separation is a deliberate
|
||||
safety boundary, not redundant API surface.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live.
|
||||
|
||||
**Demo plan:** Before the demonstration activity, it's worth showing `kubectl get secret
|
||||
app-secret -o yaml` live and pointing out the base64 value, then decoding it on the spot
|
||||
(`echo <value> | base64 -d`) — the "this isn't actually encrypted" realization lands much better
|
||||
seen than described.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students very commonly assume Secrets are encrypted by Kubernetes automatically. Correct this
|
||||
explicitly and early — it's a genuine, consequential security misunderstanding.
|
||||
- The env-var vs. volume-mount choice is sometimes seen as arbitrary. Note one real
|
||||
consideration: env vars are visible via `kubectl describe pod` and process inspection tools
|
||||
inside the container more easily than a mounted file's contents — not a hard rule, but a
|
||||
reason some teams prefer volume mounts specifically for Secrets.
|
||||
|
||||
**Discussion question:** "If someone with `kubectl get secrets` access on your namespace can
|
||||
trivially read every Secret's real value, what is a Secret actually protecting you from?"
|
||||
(Answer: it protects against accidental exposure through normal channels like
|
||||
`kubectl get configmaps`, source control, or logs — not against a determined actor with direct
|
||||
API read access, which is exactly why RBAC restricting who has that access is the real control.)
|
||||
|
||||
**Optional extension:** ask what changes about this picture on a cluster with encryption-at-rest
|
||||
enabled for Secrets (answer: the ETCD-stored data itself becomes encrypted, protecting against
|
||||
someone who gains access to the underlying etcd storage directly — still separate from and in
|
||||
addition to RBAC controlling API-level access).
|
||||
+66
@@ -0,0 +1,66 @@
|
||||
# Demonstration: A Secret Is Encoded, Not Encrypted
|
||||
|
||||
## Creating a Secret
|
||||
|
||||
```
|
||||
$ kubectl create secret generic app-secret --from-literal=api-key=supersecret
|
||||
secret/app-secret created
|
||||
```
|
||||
|
||||
## Looking at it
|
||||
|
||||
```
|
||||
$ kubectl get secret app-secret -o yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: app-secret
|
||||
data:
|
||||
api-key: c3VwZXJzZWNyZXQ=
|
||||
type: Opaque
|
||||
```
|
||||
|
||||
## Decoding it — no special permission needed beyond reading the Secret itself
|
||||
|
||||
```
|
||||
$ echo "c3VwZXJzZWNyZXQ=" | base64 -d
|
||||
supersecret
|
||||
```
|
||||
|
||||
Anyone who can run `kubectl get secret app-secret -o yaml` (i.e. anyone RBAC allows to READ
|
||||
Secrets in this Namespace) can trivially recover the real value with a standard, built-in
|
||||
command — no cracking, no special tooling. This is exactly why the lesson's point matters: the
|
||||
protection here is WHO can read the Secret object at all (RBAC), not the base64 encoding itself.
|
||||
|
||||
## Comparing to a ConfigMap
|
||||
|
||||
```
|
||||
$ kubectl get configmap app-config -o yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: app-config
|
||||
data:
|
||||
environment: production
|
||||
log_level: info
|
||||
```
|
||||
|
||||
A ConfigMap's values are stored in PLAIN TEXT — no encoding step at all. This is consistent with
|
||||
ConfigMaps being explicitly for non-sensitive data; there's no pretense of protection to begin
|
||||
with.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Run this live, actually decoding the value in front of the class — the gap
|
||||
between "Secret" (a name implying protection) and what the command output shows is the entire
|
||||
point, and it lands far better performed than read.
|
||||
|
||||
**Approximate timing:** 6-8 minutes live; 5 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** this demonstration directly targets the single most common
|
||||
misconception about Kubernetes Secrets — do not skip it even if time is short, since a wrong
|
||||
belief here has real security consequences later.
|
||||
|
||||
**Optional extension:** ask what a `SealedSecret` or an external secrets manager (mentioned by
|
||||
name only, not built as a hands-on topic here) is trying to solve that a plain Kubernetes Secret
|
||||
does not — a good forward pointer without requiring new infrastructure in this course.
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
# Knowledge Check: Configuration & Secrets
|
||||
|
||||
---
|
||||
|
||||
**1. A teammate says "our database password is safe because it's stored as a Kubernetes
|
||||
Secret." What's missing from that statement?**
|
||||
|
||||
*Answer:* A Secret's data is base64-ENCODED, not encrypted — anyone who can read the Secret
|
||||
object at all can trivially decode it. The real protection is RBAC (who is allowed to read
|
||||
Secrets in that namespace) and, on a properly configured cluster, encryption at rest — not the
|
||||
Secret object type itself.
|
||||
|
||||
---
|
||||
|
||||
**2. Why does Kubernetes have two separate objects (ConfigMap and Secret) instead of one
|
||||
"Config" object with a sensitivity flag?**
|
||||
|
||||
*Answer:* Keeping them separate makes the distinction enforceable via RBAC — a role can grant
|
||||
"read ConfigMaps" without granting "read Secrets" — and makes it visible at a glance
|
||||
(`kubectl get configmaps` never risks accidentally listing something sensitive).
|
||||
|
||||
---
|
||||
|
||||
**3. An application needs its configuration to change without rebuilding its container image
|
||||
across dev/staging/production. Which Kubernetes mechanism solves this, and why not just bake
|
||||
different configs into three separate images?**
|
||||
|
||||
*Answer:* ConfigMaps (and Secrets, for sensitive values) injected at runtime via env vars or a
|
||||
mounted volume. Baking config into the image means maintaining three separate images for what
|
||||
should be identical application code — externalizing configuration lets the SAME image run
|
||||
correctly in every environment.
|
||||
|
||||
---
|
||||
|
||||
**4. True or false: mounting a ConfigMap as a volume and consuming it as environment variables
|
||||
are functionally identical from the application's perspective.**
|
||||
|
||||
*Answer:* False. An application reading environment variables needs code that reads env vars; an
|
||||
application expecting a config FILE on disk needs the volume-mount approach. The right choice
|
||||
depends on how the application itself is written to read configuration, not just personal
|
||||
preference.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 1 is the single most important takeaway from this module — confirm
|
||||
it's genuinely understood, not just correctly guessed, before moving to the Labs.
|
||||
|
||||
**Approximate timing:** 5-7 minutes.
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
# Guided Exercise: ConfigMap, Secret, Env Var, or Volume?
|
||||
|
||||
Paper-and-thinking exercise before the Labs.
|
||||
|
||||
## The scenario
|
||||
|
||||
Your application needs four pieces of configuration:
|
||||
|
||||
1. A log level (`debug`/`info`/`warn`) that ops sometimes needs to flip without redeploying.
|
||||
2. A database password.
|
||||
3. A large JSON config file the application reads from a specific path at startup.
|
||||
4. A feature flag read once at container startup via an environment variable, per the
|
||||
application's existing code.
|
||||
|
||||
## Your task
|
||||
|
||||
For each, decide: ConfigMap or Secret, and env var or volume mount — then justify each choice.
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. **ConfigMap, env var.** Not sensitive; the application likely already reads it as an env var
|
||||
(a common convention), and env vars are simple to change and trigger a rolling restart with.
|
||||
2. **Secret, either — but env var is common here too**, since most applications expect
|
||||
credentials via environment variables by convention. The KEY decision is Secret (sensitive),
|
||||
not the delivery mechanism.
|
||||
3. **ConfigMap, volume mount.** The application expects a FILE at a path, not an environment
|
||||
variable — volume mount is the only mechanism that presents a ConfigMap as an actual file.
|
||||
4. **ConfigMap (assuming not sensitive) or Secret (if it is), env var** — the application's
|
||||
EXISTING code dictates this; you don't get to choose a mechanism the application wasn't
|
||||
written to read.
|
||||
|
||||
## Self-check
|
||||
|
||||
If you chose "volume mount" for #4 or "env var" for #3, revisit the lesson — the deciding
|
||||
factor here isn't which mechanism seems more "correct" in the abstract, it's what the
|
||||
application actually expects. Getting this backwards means the application simply won't find
|
||||
its configuration at all.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Emphasize that #4's answer depends entirely on EXISTING application
|
||||
behavior — a good moment to note that platform engineers frequently have to adapt Kubernetes
|
||||
config delivery to match an application's assumptions, not the reverse.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live with discussion; 8 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what would happen if a ConfigMap consumed as a volume mount is
|
||||
UPDATED after the Pod is already running — a good bridge to the real (if not hands-on-tested
|
||||
here) behavior that mounted ConfigMap files update automatically after a short delay, while
|
||||
env-var-injected values do NOT update without a Pod restart.
|
||||
+39
@@ -0,0 +1,39 @@
|
||||
# Reflection: Configuration Is Infrastructure, Secrets Are a Trust Boundary
|
||||
|
||||
You just externalized configuration and handled a real Secret. Take a few minutes to connect
|
||||
this to production practice.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Before this module, would you have assumed a Kubernetes Secret was encrypted? What's the
|
||||
real-world consequence of a team believing that when it isn't true?
|
||||
2. If your team's RBAC accidentally granted a low-trust role `get secrets` access across an
|
||||
entire namespace, what's actually exposed — and would your team necessarily notice?
|
||||
3. Why does separating configuration from the container image matter beyond convenience — think
|
||||
about what it enables for testing or incident response.
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**A misunderstood Secret is a false sense of security.** Teams that believe Secrets are
|
||||
encrypted by default sometimes skip additional controls (proper RBAC scoping, external secrets
|
||||
management, encryption at rest) that they'd otherwise recognize as necessary. The gap between
|
||||
perceived and actual protection is exactly where real incidents happen.
|
||||
|
||||
**Externalized configuration is what makes rollback and incident response fast.** If a bad
|
||||
config value ships (a wrong feature flag, an incorrect log level flooding storage), fixing it
|
||||
means updating a ConfigMap/Secret and letting Pods pick up the change — not rebuilding and
|
||||
redeploying an entire image. This is a direct, practical payoff of the separation this module
|
||||
taught, not just an architectural nicety.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 2 works well as a small-group exercise —
|
||||
have students actually list what a namespace's Secrets typically contain in a real application
|
||||
(DB creds, API keys, TLS certs) to make the exposure concrete.
|
||||
|
||||
**Approximate timing:** 8 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to draft a one-line policy statement their hypothetical
|
||||
team would adopt about who is allowed `get`/`list` access to Secrets in production namespaces.
|
||||
+87
@@ -0,0 +1,87 @@
|
||||
# The Service/Endpoints/DNS Request Path
|
||||
|
||||
## The problem: Pod IPs are not stable
|
||||
|
||||
Every Pod gets its own IP address, but that IP is NOT durable — when a Pod is replaced (a
|
||||
rollout, a crash recovery, a reschedule), the new Pod gets a brand new IP. If another part of
|
||||
your application had that old IP hardcoded, it would silently start talking to nothing. This is
|
||||
the exact problem in `expose-service-basics`'s scenario: "other pods cannot reliably connect
|
||||
because pod IPs change when they restart."
|
||||
|
||||
## A Service: a stable address in front of a moving set of Pods
|
||||
|
||||
A Service is a stable virtual IP and DNS name that automatically routes to whichever Pods
|
||||
currently match its label selector — regardless of how many there are or how often they change.
|
||||
You never talk to a Pod's IP directly in normal operation; you talk to the Service, and the
|
||||
Service's Endpoints (a live, continuously-updated list of the matching Pods' IPs) determines
|
||||
where traffic actually goes.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: redis-svc
|
||||
spec:
|
||||
selector:
|
||||
app: redis # <- matches Pods by label, same mechanism from ROOT 05
|
||||
ports:
|
||||
- port: 6379
|
||||
```
|
||||
|
||||
This is the same label-selector mechanism you learned in ROOT 05 — a Service doesn't know or
|
||||
care about specific Pod names, only which Pods currently carry a matching label.
|
||||
|
||||
## The three Service types you'll use most
|
||||
|
||||
- **ClusterIP** (the default) — reachable only from inside the cluster. Use this for
|
||||
internal-only communication, like one microservice talking to another or to a database.
|
||||
- **NodePort** — additionally exposes the Service on a static port on every node's own IP,
|
||||
reachable from outside the cluster. Mostly used for development/testing or when there's no
|
||||
cloud load balancer available.
|
||||
- **LoadBalancer** — asks the cloud provider to provision a real external load balancer pointed
|
||||
at the Service. This is the standard way to expose something to the public internet on a
|
||||
cloud-hosted cluster.
|
||||
|
||||
Choosing wrong isn't just a style preference — a ClusterIP Service is architecturally
|
||||
unreachable from outside the cluster no matter how correctly everything else is configured.
|
||||
|
||||
## DNS: how Pods actually find a Service by name
|
||||
|
||||
Every Service automatically gets a DNS name inside the cluster
|
||||
(`<service-name>.<namespace>.svc.cluster.local`, or usually just `<service-name>` from within
|
||||
the same Namespace) — this is why application code can use a plain hostname like `redis-svc`
|
||||
instead of an IP address at all. Kubernetes' internal DNS resolves that name to the Service's
|
||||
stable virtual IP, which then routes to whichever Pods are currently healthy.
|
||||
|
||||
## The full request path, end to end
|
||||
|
||||
```
|
||||
Client -> DNS lookup "redis-svc" -> Service's stable virtual IP
|
||||
-> Service consults its Endpoints (live list of matching, ready Pods)
|
||||
-> traffic forwarded to one of those Pod IPs
|
||||
```
|
||||
|
||||
Every one of these steps can fail independently, which is exactly why "the app can't reach
|
||||
redis" requires checking each layer rather than guessing — the next activity shows you what
|
||||
that looks like when the Endpoints layer is empty.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read, 10-12 minutes live with the request-path diagram.
|
||||
|
||||
**Demo plan:** Before the next activity, it's worth live-running `kubectl get endpoints
|
||||
<service-name>` against a healthy Service so students see a populated Endpoints list before
|
||||
they see an empty one in the demonstration.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often think a Service "is" a load balancer process running somewhere. It's closer to
|
||||
a routing rule maintained by `kube-proxy` on every node — worth clarifying that there's no
|
||||
single "Service Pod" to go find and debug.
|
||||
- ClusterIP vs. NodePort vs. LoadBalancer is sometimes memorized without the "why" — the
|
||||
knowledge check's Service-type question is designed to test the underlying reasoning, not
|
||||
just recall.
|
||||
|
||||
**Discussion question:** "If a Service's selector matches zero Pods, does the Service itself
|
||||
fail to exist?" (Answer: no — the Service object exists fine; its Endpoints list is just empty,
|
||||
which looks like connection failure from the client's side without any error at the Service
|
||||
level itself. This is exactly the next activity's topic.)
|
||||
+82
@@ -0,0 +1,82 @@
|
||||
# Demonstration: A Service With Zero Endpoints
|
||||
|
||||
This is one of the most common "it just doesn't work" incidents in real Kubernetes operation —
|
||||
and it produces no error message at all at the Service level.
|
||||
|
||||
## The setup
|
||||
|
||||
A Service `api-svc` selects Pods labeled `app: api`. A Deployment was recently updated and its
|
||||
Pods now carry the label `app: api-v2` instead (a rename during a refactor, deliberately or
|
||||
accidentally).
|
||||
|
||||
## What you'd see from the client side
|
||||
|
||||
```
|
||||
$ curl http://api-svc:8080/health
|
||||
curl: (7) Failed to connect to api-svc port 8080: Connection refused
|
||||
```
|
||||
|
||||
Just a generic connection failure — nothing here says "your Service's selector doesn't match
|
||||
any Pods."
|
||||
|
||||
## What the Service object itself shows
|
||||
|
||||
```
|
||||
$ kubectl get service api-svc
|
||||
NAME TYPE CLUSTER-IP PORT(S) AGE
|
||||
api-svc ClusterIP 10.96.34.201 8080/TCP 2h
|
||||
```
|
||||
|
||||
Completely normal-looking. The Service exists, has a stable IP, is not in any kind of "error"
|
||||
state — because from the Service's own perspective, nothing is wrong. It's doing exactly what
|
||||
it's configured to do: route to whatever matches its selector, which currently happens to be
|
||||
nothing.
|
||||
|
||||
## Where the real evidence is: Endpoints
|
||||
|
||||
```
|
||||
$ kubectl get endpoints api-svc
|
||||
NAME ENDPOINTS AGE
|
||||
api-svc <none> 2h
|
||||
```
|
||||
|
||||
This is the actual signal. `<none>` means the Service's selector currently matches zero Pods —
|
||||
this is the single fastest check for "is this a Service-to-Pod mismatch" any time something
|
||||
reachable-in-principle looks like a connection failure.
|
||||
|
||||
## Confirming the mismatch
|
||||
|
||||
```
|
||||
$ kubectl get pods --show-labels
|
||||
NAME READY STATUS LABELS
|
||||
api-abc123 1/1 Running app=api-v2
|
||||
```
|
||||
|
||||
The Pod is healthy and running — `app=api-v2`, not `app=api`. The Service's selector
|
||||
(`app: api`) matches nothing, and nothing about the Pod itself is broken.
|
||||
|
||||
## The general diagnostic shape
|
||||
|
||||
`curl`/application-level connection failure tells you *that* something's wrong, but nothing
|
||||
about *where*. `kubectl get endpoints <service>` is the fastest way to distinguish "the Service
|
||||
has no matching Pods" from "the Pods exist and are ready but something else is wrong" —
|
||||
reach for it early whenever a Service-fronted connection fails for no obvious reason.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Live-create this exact mismatch (a Service with one selector, a Deployment
|
||||
with a slightly different label) and walk through `kubectl get service` -> `kubectl get
|
||||
endpoints` -> `kubectl get pods --show-labels` in front of the class, narrating what each
|
||||
command is and isn't telling you.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; 5-7 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students often check the Service object first and, seeing
|
||||
nothing obviously wrong, assume the problem must be elsewhere entirely (DNS, firewall, the
|
||||
application itself) — reinforce that `kubectl get endpoints` is the specific, fast check for
|
||||
this exact failure mode, before reaching for anything more complex.
|
||||
|
||||
**Optional extension:** ask what `kubectl describe service api-svc` would additionally show
|
||||
compared to `kubectl get service` (answer: the same selector the Service is using, which lets
|
||||
you directly compare it against `kubectl get pods --show-labels`'s actual Pod labels side by
|
||||
side, rather than inferring the mismatch).
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
# Knowledge Check: Services & Service Discovery
|
||||
|
||||
---
|
||||
|
||||
**1. A Deployment has three replicas but `kubectl get endpoints` shows `<none>` for its Service.
|
||||
Which part of the system should you inspect first, and why?**
|
||||
|
||||
*Answer:* Compare the Service's selector (`kubectl describe service`) against the actual Pods'
|
||||
labels (`kubectl get pods --show-labels`) — a Service with a healthy backend Deployment but zero
|
||||
Endpoints almost always means the selector doesn't match the Pods' current labels, not that the
|
||||
Pods themselves are unhealthy.
|
||||
|
||||
---
|
||||
|
||||
**2. You need a microservice reachable ONLY from other services inside the same cluster, never
|
||||
from the public internet. Which Service type, and why not LoadBalancer?**
|
||||
|
||||
*Answer:* ClusterIP — it's reachable only from inside the cluster by design, which is exactly
|
||||
the requirement. A LoadBalancer would provision a real external endpoint reachable from the
|
||||
public internet, which is more exposure than the requirement calls for, and depends on cloud
|
||||
provider support this scenario doesn't need.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: application code that connects to `redis-svc` by that plain hostname is
|
||||
relying on Kubernetes' internal DNS to resolve it.**
|
||||
|
||||
*Answer:* True. Every Service automatically gets a DNS name; that's specifically what lets code
|
||||
use a stable hostname instead of hardcoding a Pod IP that could change at any time.
|
||||
|
||||
---
|
||||
|
||||
**4. A Service's Endpoints list is non-empty, but the application still can't connect. Does
|
||||
this rule out a Service-level problem entirely?**
|
||||
|
||||
*Answer:* Not entirely — a non-empty Endpoints list means at least one Pod currently matches the
|
||||
selector, but the connection could still fail for other reasons (wrong port number on the
|
||||
Service, a NetworkPolicy blocking the traffic, the application inside the Pod not actually
|
||||
listening). A populated Endpoints list narrows the problem, it doesn't prove the whole path
|
||||
works.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 1 is the single most transferable diagnostic habit from this module —
|
||||
worth explicitly calling out as "the first thing to check" for this entire class of problem.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+49
@@ -0,0 +1,49 @@
|
||||
# Guided Exercise: Choosing the Right Service Type
|
||||
|
||||
Paper-and-thinking exercise before the Lab.
|
||||
|
||||
## The scenario
|
||||
|
||||
You're deploying three components for a class project on a cloud-hosted cluster:
|
||||
|
||||
1. A Postgres database that only your own backend application should ever talk to.
|
||||
2. A backend API that the frontend (also running in the cluster) calls internally.
|
||||
3. The frontend web application itself, which needs to be reachable by users on the public
|
||||
internet.
|
||||
|
||||
## Your task
|
||||
|
||||
For each of the three components, decide: ClusterIP, NodePort, or LoadBalancer — and justify
|
||||
each choice in one sentence referencing WHO needs to reach it.
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. **Postgres — ClusterIP.** Only your own backend (running inside the same cluster) needs to
|
||||
reach it; there is no reason for it to be reachable from outside the cluster at all, and
|
||||
exposing a database externally is a real, avoidable security risk.
|
||||
2. **Backend API — ClusterIP.** The frontend calls it from inside the cluster too — same
|
||||
reasoning as the database. External users never talk to the backend directly; they only ever
|
||||
go through the frontend.
|
||||
3. **Frontend — LoadBalancer** (on a real cloud-hosted cluster). Public internet users need to
|
||||
reach it, and a cloud provider's LoadBalancer is the standard mechanism for that. NodePort
|
||||
would technically work but is typically used for development/testing rather than a real
|
||||
public-facing service on a cloud cluster.
|
||||
|
||||
## Self-check
|
||||
|
||||
If you chose LoadBalancer for the database or the backend API, reconsider — "internet-reachable
|
||||
by default" is not the safe default for anything that doesn't need to be, and getting this
|
||||
choice right the first time avoids a real security exposure later, not just a style issue.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Push on the "why not just LoadBalancer everything, it works" instinct some
|
||||
students will have — this is a good early moment to connect Service-type choice to the security
|
||||
mindset that ROOT 09 builds on more fully (exposure should be intentional, not the path of
|
||||
least resistance).
|
||||
|
||||
**Approximate timing:** 10 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what would need to be added (not covered in this module) to let
|
||||
public users reach the frontend at a real domain name over HTTPS rather than a bare
|
||||
LoadBalancer IP — a natural forward pointer to Ingress, covered in the next module.
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
# Launch Lab: Exposing an Application
|
||||
|
||||
Time to create a real Service in your disposable Lab and connect it to a real Deployment.
|
||||
|
||||
## What you're doing
|
||||
|
||||
A `redis` Deployment is running, but other Pods can't reliably reach it because Pod IPs change
|
||||
whenever a Pod restarts — exactly the problem the lesson opened with. Your task is to create a
|
||||
stable ClusterIP Service in front of it.
|
||||
|
||||
## Before you start
|
||||
|
||||
Recall the request path from the lesson: your Service's `selector` has to match the Deployment
|
||||
Pods' actual labels, or you'll end up with the empty-Endpoints situation from the
|
||||
demonstration. Check the Deployment's Pod labels before writing your Service's selector, don't
|
||||
guess at them.
|
||||
|
||||
## Your task in this Lab
|
||||
|
||||
Create a ClusterIP Service exposing the `redis` Deployment on the correct port. Once it's
|
||||
created, verify it the same way the demonstration did: `kubectl get service`, then `kubectl get
|
||||
endpoints` to confirm it actually has a non-empty Endpoints list, not just that the Service
|
||||
object exists.
|
||||
|
||||
This Lab has its own in-context hints if you get stuck.
|
||||
|
||||
When you're done, return to this module for a reflection on why service discovery is
|
||||
infrastructure, not magic.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Classroom flow:** Encourage students to run `kubectl get endpoints` as their OWN verification
|
||||
step even though the Lab's automated validation will check this too — building the habit of
|
||||
checking Endpoints, not just "the Service exists," is the actual transferable skill here.
|
||||
|
||||
**Approximate timing:** 10-15 minutes.
|
||||
|
||||
**Common mistake to watch for:** a selector that matches on the wrong label key/value, or a
|
||||
port/targetPort mismatch (the Service's `port` is what clients connect to; `targetPort` is what
|
||||
the container actually listens on — conflating the two is a very common first-time error).
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
# Reflection: Service Discovery Is Infrastructure, Not Magic
|
||||
|
||||
You just created a Service that gives a moving set of Pods a stable address. Take a few minutes
|
||||
to connect this to real operational thinking.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Before this module, "the app can just connect to redis by name" might have seemed obvious or
|
||||
automatic. What specific mechanism actually makes that true, and what would break it?
|
||||
2. If your Service's selector had been wrong (matching zero Pods), what would the SYMPTOM have
|
||||
looked like from the application's side, versus what the actual CAUSE would have been?
|
||||
3. Why is checking `kubectl get endpoints` before assuming a networking/DNS/firewall problem a
|
||||
good default habit, given how common a selector mismatch actually is?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**"It's probably a networking issue" is often wrong, and Endpoints is how you find out fast.** A
|
||||
huge fraction of real "service X can't reach service Y" incidents in Kubernetes are Service
|
||||
selector mismatches, not actual network problems — checking `kubectl get endpoints` first, before
|
||||
escalating to firewall rules or DNS debugging, saves real time precisely because this failure
|
||||
mode produces zero errors at the layers people instinctively check first.
|
||||
|
||||
**Stable addressing is what makes rolling updates and self-healing actually usable.** Everything
|
||||
you learned in ROOT 06 about Pods being replaced during a rollout or after a crash only works
|
||||
smoothly in practice because Services give callers a stable target that survives those
|
||||
replacements — without this layer, every Pod replacement would require updating every caller's
|
||||
configuration by hand.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** use question 2 as a discussion prompt — the
|
||||
symptom/cause gap is exactly why this class of bug is often under-diagnosed in real teams.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to write the one-line runbook entry they'd want a future
|
||||
on-call engineer to have for "Service X seems unreachable" — a good exercise in turning a
|
||||
lesson into something operationally reusable.
|
||||
+90
@@ -0,0 +1,90 @@
|
||||
# Ingress and NetworkPolicy: What They Architecturally Do
|
||||
|
||||
## Ingress: routing external HTTP(S) traffic by host/path
|
||||
|
||||
You already know a LoadBalancer Service can expose something to the public internet. But a real
|
||||
application usually needs more than "one raw IP to one Service" — you might want
|
||||
`api.example.com` and `app.example.com` to route to different backend Services, all through one
|
||||
entry point, with TLS termination handled in one place instead of per-Service.
|
||||
|
||||
**Ingress** is the Kubernetes object for exactly this: a set of HTTP routing rules (by hostname
|
||||
and/or URL path) that an Ingress controller reads and turns into actual load-balancer/proxy
|
||||
configuration.
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: app-ingress
|
||||
spec:
|
||||
rules:
|
||||
- host: app.example.com
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: frontend-svc
|
||||
port: { number: 80 }
|
||||
```
|
||||
|
||||
An important distinction: Ingress is a set of ROUTING RULES, not a load balancer itself — an
|
||||
Ingress controller (a separate piece of software running in the cluster) is what actually reads
|
||||
these rules and does the routing. Without a controller installed, an Ingress object exists in
|
||||
the API but nothing acts on it.
|
||||
|
||||
## NetworkPolicy: traffic reachability as something you declare, not assume
|
||||
|
||||
By default, every Pod in a Kubernetes cluster can reach every other Pod — there is no built-in
|
||||
network isolation between them at all. This surprises people coming from a "obviously the
|
||||
network is locked down unless configured otherwise" assumption. It is exactly backwards from
|
||||
that assumption: **everything is reachable until you say otherwise.**
|
||||
|
||||
A NetworkPolicy changes this for the Pods it selects, by declaring which traffic IS allowed —
|
||||
everything not explicitly allowed to a Pod covered by at least one NetworkPolicy is then denied.
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: allow-frontend-to-backend
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels: { app: backend }
|
||||
ingress:
|
||||
- from:
|
||||
- podSelector:
|
||||
matchLabels: { app: frontend }
|
||||
```
|
||||
|
||||
This example allows traffic to `backend` Pods only from `frontend` Pods — once this policy
|
||||
exists and selects `backend`, every OTHER Pod trying to reach `backend` is now blocked, even
|
||||
though nothing was true about that before the policy existed.
|
||||
|
||||
## Why this matters architecturally, before it matters as a security control
|
||||
|
||||
Right now, think of NetworkPolicy purely as "declaring intended reachability, not assuming it."
|
||||
ROOT 09 comes back to NetworkPolicy specifically as a SECURITY control (default-deny postures,
|
||||
least-privilege network access) — this module is deliberately just the architectural mental
|
||||
model: Kubernetes doesn't isolate network traffic for you by default, and NetworkPolicy is the
|
||||
mechanism for declaring the reachability you actually intend.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often assume Ingress by itself "does" the routing. Emphasize the controller
|
||||
requirement — an Ingress object with no controller installed does nothing.
|
||||
- The "everything reachable by default" fact genuinely surprises most students the first time
|
||||
they hear it — it's worth stating plainly and letting it land rather than rushing past it.
|
||||
|
||||
**Discussion question:** "If NO NetworkPolicy exists anywhere in a Namespace, what's the actual
|
||||
reachability between Pods in that Namespace?" (Answer: completely open — every Pod can reach
|
||||
every other Pod, matching the "everything reachable by default" rule.)
|
||||
|
||||
**Optional extension:** ask what would happen if you created a NetworkPolicy that selects
|
||||
`frontend` Pods but declares no `ingress` rules at all (answer: `frontend` Pods would become
|
||||
completely unreachable by anything — an empty ingress list inside a NetworkPolicy that HAS an
|
||||
ingress policyType means "deny all incoming," not "no policy applies").
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
# Knowledge Check: Ingress & NetworkPolicy Foundations
|
||||
|
||||
---
|
||||
|
||||
**1. You create an Ingress object with correct routing rules, but no traffic is ever routed
|
||||
according to them. What's the most likely reason?**
|
||||
|
||||
*Answer:* No Ingress controller is installed/running in the cluster. Ingress objects are just
|
||||
routing rules; a separate controller has to actually read and act on them, or nothing happens.
|
||||
|
||||
---
|
||||
|
||||
**2. True or false: by default, a Pod in Namespace A cannot reach a Pod in Namespace B unless a
|
||||
NetworkPolicy explicitly allows it.**
|
||||
|
||||
*Answer:* False — the opposite is true by default. With NO NetworkPolicy in place at all,
|
||||
every Pod can reach every other Pod regardless of Namespace. NetworkPolicy is what RESTRICTS
|
||||
this default-open behavior, not what's required to enable connectivity in the first place.
|
||||
|
||||
---
|
||||
|
||||
**3. A NetworkPolicy selects Pods labeled `app: backend` and declares one ingress rule allowing
|
||||
traffic from Pods labeled `app: frontend`. What happens to traffic from a Pod labeled
|
||||
`app: admin-tool` trying to reach `backend`?**
|
||||
|
||||
*Answer:* It's blocked. Once a NetworkPolicy selects a Pod, only traffic matching one of its
|
||||
explicit rules is allowed — everything else to that Pod is denied, including traffic that would
|
||||
have worked fine before the policy existed.
|
||||
|
||||
---
|
||||
|
||||
**4. Why is Ingress the right tool for exposing an HTTP application by hostname, rather than
|
||||
creating a separate LoadBalancer Service per hostname?**
|
||||
|
||||
*Answer:* A LoadBalancer Service typically provisions its own external IP/load balancer per
|
||||
Service — one per hostname would mean paying for and managing N separate external endpoints.
|
||||
Ingress lets one entry point route to many backend Services by hostname/path, which is both
|
||||
cheaper and centralizes concerns like TLS termination.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2 catches a very common and consequential misconception — worth
|
||||
confirming out loud that everyone actually updated their mental model, not just guessed
|
||||
correctly.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+72
@@ -0,0 +1,72 @@
|
||||
# Demonstration: Tracing a Request Through Ingress
|
||||
|
||||
## The setup
|
||||
|
||||
An Ingress `web-ingress` routes `app.example.com/` to Service `web-svc`, which fronts a
|
||||
Deployment `web-app`. Nothing works yet from outside the cluster — let's see why, step by step.
|
||||
|
||||
## Step 1: is the Ingress object even doing anything?
|
||||
|
||||
```
|
||||
$ kubectl get ingress web-ingress
|
||||
NAME CLASS HOSTS ADDRESS PORTS AGE
|
||||
web-ingress traefik app.example.com 80 2m
|
||||
```
|
||||
|
||||
`ADDRESS` is empty. On a real cluster, this column fills in once the Ingress controller has
|
||||
actually processed the object and assigned it a reachable address — an empty address after a
|
||||
couple of minutes is the first sign to check the controller itself, not your YAML.
|
||||
|
||||
## Step 2: is the controller running at all?
|
||||
|
||||
```
|
||||
$ kubectl get pods -n kube-system -l app.kubernetes.io/name=traefik
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
traefik-7d8f9c6b5d-abcde 1/1 Running 0 10d
|
||||
```
|
||||
|
||||
Running fine — so the ADDRESS delay in this case is transient, not a missing controller. (If
|
||||
this returned no Pods at all, THAT would be the actual root cause — no controller means no
|
||||
Ingress object ever gets acted on, exactly like the demonstration in ROOT 07's Ingress lesson
|
||||
described.)
|
||||
|
||||
## Step 3: once ADDRESS populates, does the route actually work?
|
||||
|
||||
```
|
||||
$ kubectl get ingress web-ingress
|
||||
NAME CLASS HOSTS ADDRESS PORTS AGE
|
||||
web-ingress traefik app.example.com 10.0.1.50 80 3m
|
||||
|
||||
$ curl -H "Host: app.example.com" http://10.0.1.50/
|
||||
<html>...web-app's real response...</html>
|
||||
```
|
||||
|
||||
Success — but notice the `-H "Host: app.example.com"` header. Ingress routes by HOSTNAME, not
|
||||
just by IP — hitting the Ingress controller's address WITHOUT the matching Host header would
|
||||
route differently (or not at all), because that's literally what the Ingress rule is matching
|
||||
on.
|
||||
|
||||
## The general shape
|
||||
|
||||
Symptom (nothing reachable from outside) has at least three DIFFERENT possible layers: the
|
||||
Ingress object itself, the controller processing it, and the Service/Pod chain underneath it
|
||||
(already covered in ROOT 07's Services module). Check them in that order — Ingress ADDRESS,
|
||||
then controller health, then the underlying Service — rather than guessing at DNS or firewall
|
||||
issues first.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** If a real Ingress controller is available in the demo environment, run this
|
||||
live including the Host-header nuance — students are frequently surprised that `curl <ip>`
|
||||
alone doesn't work the way they expect.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; 5-7 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students often expect an Ingress's IP/address to work like
|
||||
a Service's ClusterIP (reachable directly, no extra header needed). Reinforce that Ingress
|
||||
routing is fundamentally HOSTNAME/PATH-based, which is exactly what makes one Ingress able to
|
||||
route many different domains through one entry point.
|
||||
|
||||
**Optional extension:** ask what would happen if TWO Ingress objects both claimed the same host
|
||||
+ path — a good forward pointer to why Ingress configuration conflicts are a real operational
|
||||
concern on a shared cluster, without needing to build a hands-on lab around it here.
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
# Guided Exercise: Planning Network Isolation
|
||||
|
||||
Paper-and-thinking exercise before the Lab.
|
||||
|
||||
## The scenario
|
||||
|
||||
A `database` Pod (label `app=database`) in the `netpol` namespace should only ever be reachable
|
||||
by Pods labeled `role=backend` — not the frontend, not anything else in the cluster.
|
||||
|
||||
## Your task
|
||||
|
||||
1. Before writing any YAML: right now, with NO NetworkPolicy in place, can the frontend reach
|
||||
the database? Why?
|
||||
2. What would a NetworkPolicy that ONLY specifies `podSelector: { matchLabels: { app: database
|
||||
} }` and no `ingress` rules at all do to traffic reaching the database — including from
|
||||
`role=backend` Pods?
|
||||
3. Sketch (in words, not full YAML) what the `ingress` rule needs to say to allow ONLY
|
||||
`role=backend` traffic in.
|
||||
4. If a new Pod is later created with `role=backend` in a DIFFERENT namespace, would your policy
|
||||
as sketched in #3 allow it? What would need to change if cross-namespace access should be
|
||||
allowed?
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. Yes — by default, every Pod can reach every other Pod regardless of labels or namespace
|
||||
(ROOT 07's Ingress & NetworkPolicy lesson). Nothing restricts this until a policy exists.
|
||||
2. It would block ALL incoming traffic to the database, including from `role=backend` — an
|
||||
empty `ingress` list under a policy that selects the Pod means "no traffic is allowed in,"
|
||||
not "no policy applies."
|
||||
3. An `ingress` rule with `from: [{ podSelector: { matchLabels: { role: backend } } }]` — this
|
||||
allows traffic specifically from Pods carrying that label, and nothing else.
|
||||
4. Not automatically — a bare `podSelector` in the `from` clause matches Pods in the SAME
|
||||
namespace as the policy by default. Allowing a different namespace requires adding a
|
||||
`namespaceSelector` alongside the `podSelector` in the same `from` entry.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2 is the one most likely to trip students up — it directly reinforces
|
||||
the "empty ingress rules means deny-all, not no-op" point from the lesson, which is worth
|
||||
confirming actually landed before the Lab.
|
||||
|
||||
**Approximate timing:** 10-12 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what a security-minded team would ALSO want to restrict about this
|
||||
database Pod's OWN outbound (egress) traffic, beyond who can reach it — a light forward pointer
|
||||
to ROOT 09's security-control framing of NetworkPolicy, without requiring students to build an
|
||||
egress rule here.
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
# Reflection: Reachability Should Be a Decision, Not a Default
|
||||
|
||||
You just configured real Ingress routing and real network isolation. Take a few minutes to
|
||||
connect this to production thinking.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Before this module, would you have assumed a fresh Kubernetes cluster isolates Pods from
|
||||
each other by default? Has that changed?
|
||||
2. If your team runs a multi-tenant cluster (several unrelated applications sharing it), what's
|
||||
the risk of NEVER applying any NetworkPolicy at all?
|
||||
3. Ingress lets many different domains share one entry point. What's the operational benefit of
|
||||
that beyond just "fewer objects to manage"?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**"No NetworkPolicy" is not neutral — it's full reachability.** Exactly like an unhardened
|
||||
Pod's SecurityContext (ROOT 09 covers this in depth), the absence of a NetworkPolicy isn't a
|
||||
safe default; it's the MOST permissive configuration available. On any cluster where more than
|
||||
one team or application shares infrastructure, that's a real, often-overlooked exposure.
|
||||
|
||||
**Centralizing external entry points through Ingress reduces attack surface, not just object
|
||||
count.** Every separate LoadBalancer Service is a separate externally-reachable endpoint to
|
||||
secure, monitor, and patch. Routing through one (or a small number of) Ingress controllers means
|
||||
security controls — TLS termination, rate limiting, WAF rules — are configured and audited in
|
||||
one place instead of scattered across every application team's own Service definitions.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 2 is worth pushing on with a concrete
|
||||
scenario — ask what a compromised low-value application (a demo app, a staging environment)
|
||||
could reach on a cluster with zero NetworkPolicies, versus one with sensible defaults.
|
||||
|
||||
**Approximate timing:** 8 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to sketch (in words) what a reasonable DEFAULT-DENY policy
|
||||
for an entire new namespace would look like, before any application-specific allow rules are
|
||||
added — previews the security-control framing ROOT 09 builds on.
|
||||
+74
@@ -0,0 +1,74 @@
|
||||
# Ephemeral vs. Persistent Storage
|
||||
|
||||
## The default: a container's filesystem is thrown away
|
||||
|
||||
By default, anything a container writes to its own filesystem disappears the moment that
|
||||
container is replaced — a crash, a rollout, a reschedule, anything. This is a deliberate design
|
||||
choice (it's part of what makes a Pod safely disposable/replaceable), but it means "just write
|
||||
files to disk" silently loses data unless you use one of Kubernetes' actual storage mechanisms.
|
||||
|
||||
## emptyDir: shared, but still temporary
|
||||
|
||||
An `emptyDir` volume gives a Pod's containers a shared directory that survives a container
|
||||
restart WITHIN the same Pod, but is deleted the moment the Pod itself is removed. It's the right
|
||||
tool for two containers in one Pod that need to share files with each other (a sidecar writing
|
||||
logs another container reads, for example) — not for anything that needs to outlive the Pod.
|
||||
|
||||
```yaml
|
||||
volumes:
|
||||
- name: shared-data
|
||||
emptyDir: {}
|
||||
```
|
||||
|
||||
## PersistentVolume & PersistentVolumeClaim: storage that outlives the Pod
|
||||
|
||||
A **PersistentVolume (PV)** represents an actual piece of storage (a cloud disk, an NFS share,
|
||||
whatever the cluster's infrastructure provides) as a Kubernetes object. A
|
||||
**PersistentVolumeClaim (PVC)** is a Pod's REQUEST for storage matching some criteria (size,
|
||||
access mode) — the Pod references the PVC, not a PV directly, and Kubernetes binds the PVC to a
|
||||
matching PV.
|
||||
|
||||
```
|
||||
Pod ──references──> PVC ("I need 5Gi, ReadWriteOnce") ──binds to──> PV (actual storage)
|
||||
```
|
||||
|
||||
This indirection matters: a Pod's manifest doesn't need to know or care exactly which physical
|
||||
storage backs its PVC — that's resolved separately, and the same PVC keeps working even if the
|
||||
Pod behind it is replaced.
|
||||
|
||||
## StorageClass & dynamic provisioning
|
||||
|
||||
Manually creating a PV for every PVC doesn't scale. A **StorageClass** describes a category of
|
||||
storage (e.g. "fast SSD" vs. "cheap archival") and, when a PVC requests that class, the cluster
|
||||
can automatically provision a matching PV on demand — this is "dynamic provisioning," and it's
|
||||
the normal way storage works on any real cloud-hosted cluster. A StorageClass also controls the
|
||||
**reclaim policy** — what happens to the underlying storage after its PVC is deleted (`Delete`
|
||||
the data, or `Retain` it for manual recovery).
|
||||
|
||||
## The decision in one pass
|
||||
|
||||
| You need... | Use |
|
||||
|---|---|
|
||||
| Temporary scratch space shared between containers in ONE Pod | `emptyDir` |
|
||||
| Storage that survives Pod replacement, manually provisioned | PV + PVC |
|
||||
| The above, provisioned automatically on demand | PVC + StorageClass (dynamic provisioning) |
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live with the PV/PVC/StorageClass diagram.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often assume `emptyDir` is "persistent" because the name doesn't obviously say
|
||||
otherwise. Be explicit: it survives a CONTAINER restart, not a POD deletion.
|
||||
- The PV/PVC indirection can seem like unnecessary ceremony at first — frame it as the same
|
||||
"declare what you want, let the system resolve it" pattern from ROOT 05's desired-state
|
||||
lesson, applied to storage instead of Pod count.
|
||||
|
||||
**Discussion question:** "If a Pod using a PVC is deleted and replaced, what happens to its
|
||||
data?" (Answer: it's preserved — the PVC and its bound PV are separate objects from the Pod,
|
||||
and a new Pod referencing the same PVC picks up right where the data was left, which is the
|
||||
entire point of using persistent storage over `emptyDir`.)
|
||||
|
||||
**Optional extension:** ask what a `Retain` reclaim policy is protecting against, compared to
|
||||
`Delete` (answer: accidental data loss from deleting a PVC by mistake — `Retain` keeps the
|
||||
underlying storage around for manual recovery instead of immediately destroying it).
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
# Knowledge Check: Storage Fundamentals
|
||||
|
||||
---
|
||||
|
||||
**1. Two containers in the same Pod need to share files with each other, but nothing needs to
|
||||
survive the Pod being replaced. What's the simplest correct choice?**
|
||||
|
||||
*Answer:* `emptyDir` — shared within the Pod, deliberately temporary, exactly matching the
|
||||
requirement. Reaching for a PV/PVC here would be more machinery than the problem needs.
|
||||
|
||||
---
|
||||
|
||||
**2. A Pod using a PVC is deleted and a new Pod is created referencing the SAME PVC. What
|
||||
happens to the data?**
|
||||
|
||||
*Answer:* It's preserved. The PVC and its bound PV exist independently of any specific Pod — the
|
||||
new Pod picks up exactly where the data was left, which is the entire reason to use persistent
|
||||
storage instead of `emptyDir` for anything that needs to outlive a single Pod's lifetime.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: a Pod's manifest must specify exactly which PersistentVolume backs its
|
||||
storage request.**
|
||||
|
||||
*Answer:* False. A Pod references a PVC (a request), not a PV directly — the binding between
|
||||
PVC and an actual PV is resolved separately, which is exactly the indirection that lets the same
|
||||
Pod manifest work regardless of which physical storage ends up behind it.
|
||||
|
||||
---
|
||||
|
||||
**4. What does a StorageClass's reclaim policy actually control, and why would a team choose
|
||||
`Retain` over `Delete` for a production database's storage?**
|
||||
|
||||
*Answer:* It controls what happens to the underlying storage after its PVC is deleted. `Retain`
|
||||
keeps the data around for manual recovery instead of immediately destroying it — a safety
|
||||
margin against an accidental `kubectl delete pvc` on something genuinely important, at the cost
|
||||
of needing manual cleanup later.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2 is worth confirming lands correctly — it's the single fact that
|
||||
justifies the entire PV/PVC indirection existing at all.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+86
@@ -0,0 +1,86 @@
|
||||
# How the Scheduler Places Pods, and How Governance Shapes It
|
||||
|
||||
## The scheduler's job
|
||||
|
||||
Every time a new Pod needs to run, the scheduler decides which node it lands on — you never
|
||||
manually assign a Pod to a node in normal operation (ROOT 05's mental model). By default, the
|
||||
scheduler picks based on available resources (does this node have enough free CPU/memory), but
|
||||
several mechanisms let you INFLUENCE or CONSTRAIN that decision when the default isn't good
|
||||
enough.
|
||||
|
||||
## Steering placement: nodeSelector, taints/tolerations, affinity
|
||||
|
||||
- **nodeSelector** — the simplest constraint: "only schedule this Pod on nodes with THIS
|
||||
label" (e.g. `disktype: ssd`). A blunt but clear tool.
|
||||
- **Taints and tolerations** — the inverse relationship. A NODE is tainted (marked "don't
|
||||
schedule here unless you specifically tolerate this taint") and a Pod TOLERATES a specific
|
||||
taint to be allowed to land there anyway. Commonly used to reserve nodes for specific
|
||||
workloads (e.g. GPU nodes tainted so only GPU-requiring Pods land there) or to keep general
|
||||
workloads off nodes reserved for something else.
|
||||
- **Affinity/anti-affinity** — more expressive rules: "prefer/require scheduling near Pods with
|
||||
label X" (affinity) or "prefer/require scheduling AWAY from Pods with label X"
|
||||
(anti-affinity). A common real use: anti-affinity between replicas of the same application, so
|
||||
a single node failure can't take down every replica at once.
|
||||
|
||||
## Priority: which Pods matter more when resources are tight
|
||||
|
||||
A **PriorityClass** lets you mark some Pods as more important than others. When the cluster is
|
||||
under resource pressure, a higher-priority pending Pod can even cause a lower-priority RUNNING
|
||||
Pod to be evicted (preempted) to make room — this is a real, consequential mechanism, not just a
|
||||
scheduling hint, and misconfiguring it can mean an unimportant batch job unexpectedly evicts
|
||||
something that actually mattered.
|
||||
|
||||
## PodDisruptionBudget: protecting against VOLUNTARY disruption
|
||||
|
||||
A **PodDisruptionBudget (PDB)** limits how many replicas of an application can be taken down
|
||||
AT ONCE during voluntary disruptions — a node drain for maintenance, a cluster upgrade — by
|
||||
declaring a minimum number (or percentage) that must stay available. This does NOT protect
|
||||
against involuntary disruption (a node crashing unexpectedly) — it specifically constrains
|
||||
operations that Kubernetes itself controls the timing of.
|
||||
|
||||
```yaml
|
||||
apiVersion: policy/v1
|
||||
kind: PodDisruptionBudget
|
||||
metadata:
|
||||
name: web-pdb
|
||||
spec:
|
||||
minAvailable: 2
|
||||
selector:
|
||||
matchLabels: { app: web }
|
||||
```
|
||||
|
||||
Without a PDB, a node drain during routine maintenance could legally take down every replica of
|
||||
`web` simultaneously if they all happened to be on nodes being drained together — a PDB is what
|
||||
prevents that from being "technically allowed."
|
||||
|
||||
## ResourceQuota & LimitRange: governance at the Namespace level
|
||||
|
||||
A **ResourceQuota** caps the TOTAL resources (CPU, memory, object counts) a Namespace can
|
||||
consume — useful for multi-team clusters where one team's runaway workload shouldn't be able to
|
||||
starve everyone else. A **LimitRange** sets DEFAULT and/or min/max resource requests/limits for
|
||||
individual containers within a Namespace, so a Pod created without explicit resource settings
|
||||
still gets sensible ones rather than none at all.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 20 minutes read; 15 minutes live given the number of concepts covered.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Taints/tolerations are frequently confused with affinity — clarify the directionality: a
|
||||
taint REPELS by default (node says no); a toleration is what lets a specific Pod through
|
||||
anyway. Affinity is the Pod expressing a preference, not the node restricting access.
|
||||
- Students sometimes assume a PDB prevents ALL disruption. Be explicit that it only constrains
|
||||
VOLUNTARY disruption (things Kubernetes/an operator controls the timing of) — a hardware
|
||||
failure ignores PDBs entirely, because nobody is "asking permission" for a crash.
|
||||
|
||||
**Discussion question:** "Why would a real team use anti-affinity between replicas of the same
|
||||
application instead of just trusting the scheduler's default spread?" (Answer: the default
|
||||
scheduler does consider spread as one factor among several, but doesn't GUARANTEE it the way an
|
||||
explicit anti-affinity rule does — for something where "don't lose all replicas to one node
|
||||
failure" is a hard requirement, not just a nice-to-have, an explicit rule is safer than an
|
||||
implicit tendency.)
|
||||
|
||||
**Optional extension:** ask what would happen to a Namespace's Pods if a ResourceQuota were
|
||||
added AFTER Pods without explicit resource requests already existed there (a good bridge to why
|
||||
LimitRange defaults matter — a Quota alone doesn't retroactively fix Pods created without
|
||||
resource requests, it just constrains what NEW Pods can consume going forward).
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
# Knowledge Check: Scheduling & Resource Governance
|
||||
|
||||
---
|
||||
|
||||
**1. A node is tainted `dedicated=gpu:NoSchedule`. A Pod with no matching toleration is
|
||||
pending and never gets scheduled anywhere else, even though other untainted nodes have free
|
||||
capacity. What's actually going on?**
|
||||
|
||||
*Answer:* Nothing is wrong — the taint only affects the tainted node specifically; it doesn't
|
||||
explain why the Pod isn't landing elsewhere. If it's stuck pending entirely, the cause is
|
||||
something else (insufficient resources cluster-wide, a nodeSelector/affinity rule that can't be
|
||||
satisfied anywhere) — the taint itself only ever repels from the ONE node it's applied to.
|
||||
|
||||
---
|
||||
|
||||
**2. During a planned node drain for maintenance, would you expect a PodDisruptionBudget to
|
||||
prevent an UNRELATED node from crashing unexpectedly at the same time?**
|
||||
|
||||
*Answer:* No. A PDB only constrains VOLUNTARY disruptions — operations Kubernetes/an operator
|
||||
controls the timing of, like a drain. An unrelated hardware failure is involuntary and ignores
|
||||
PDBs entirely; nothing is "asking permission" for a crash.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: a higher-priority pending Pod can cause a lower-priority, already-RUNNING
|
||||
Pod to be evicted to make room.**
|
||||
|
||||
*Answer:* True — this is what Pod priority/preemption actually does under resource pressure,
|
||||
not just a scheduling hint. Misconfigured priorities can have real, disruptive consequences for
|
||||
this reason.
|
||||
|
||||
---
|
||||
|
||||
**4. A team notices new Pods in their Namespace keep getting created with no CPU/memory
|
||||
requests at all, even though nobody set that intentionally. What governance mechanism would fix
|
||||
this going forward, and what would it NOT fix?**
|
||||
|
||||
*Answer:* A LimitRange with default requests/limits would apply sensible defaults to future
|
||||
Pods created without explicit values. It would NOT retroactively change Pods that already exist
|
||||
— it only affects new Pod creation from the point it's added.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 2 reinforces a genuinely important, often-missed distinction
|
||||
(voluntary vs. involuntary disruption) — worth confirming it lands, not just that the answer was
|
||||
guessed correctly.
|
||||
|
||||
**Approximate timing:** 5-7 minutes given the density of this module's lesson.
|
||||
+87
@@ -0,0 +1,87 @@
|
||||
# Defense in Depth & the Kubernetes Security Model
|
||||
|
||||
## Security is a posture, not a single setting
|
||||
|
||||
There is no single "make it secure" switch in Kubernetes. Real security comes from **defense in
|
||||
depth** — multiple independent layers, each reducing what an attacker (or a bug) can do if an
|
||||
earlier layer fails. If a container has a vulnerability that lets an attacker execute arbitrary
|
||||
code, the question that actually matters is: what can that code DO once it's running? A
|
||||
container running as root, with every Linux capability, on a writable filesystem, with no
|
||||
restrictions at all, can do far more damage than one deliberately hardened to need almost
|
||||
nothing.
|
||||
|
||||
## Least privilege: grant only what's actually needed
|
||||
|
||||
The organizing principle behind everything in this course is **least privilege**: a workload
|
||||
should have exactly the permissions and capabilities it needs to do its job, and nothing more.
|
||||
This sounds obvious stated plainly, but the DEFAULT behavior of a plain container is closer to
|
||||
"maximum privilege unless told otherwise" — root user, all standard Linux capabilities, a
|
||||
writable filesystem. Hardening a Pod means deliberately narrowing that down.
|
||||
|
||||
## SecurityContext: where you actually configure this
|
||||
|
||||
`securityContext` (settable at the Pod level and per-container) is where you declare these
|
||||
restrictions:
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
containers:
|
||||
- name: app
|
||||
securityContext:
|
||||
readOnlyRootFilesystem: true
|
||||
capabilities:
|
||||
drop: ["ALL"]
|
||||
```
|
||||
|
||||
- **`runAsNonRoot` / `runAsUser`** — the container process runs as a specific non-root user
|
||||
instead of root. If a container image was built to require root and you set this, it will
|
||||
fail to start — which is a good, loud failure, not a silent security gap. Hardening
|
||||
sometimes surfaces an image's OWN bad assumptions.
|
||||
- **`capabilities: drop: ["ALL"]`** — Linux capabilities are fine-grained permissions
|
||||
(`NET_BIND_SERVICE`, `SYS_ADMIN`, dozens more) a process can hold even without full root.
|
||||
Dropping all of them, then adding back only the specific ones actually needed, is the
|
||||
least-privilege pattern applied to capabilities specifically.
|
||||
- **`readOnlyRootFilesystem: true`** — the container's own filesystem cannot be written to at
|
||||
all (use an explicit `emptyDir` volume, ROOT 08, for anything that genuinely needs to write
|
||||
temporary files). This closes off a whole category of attack that relies on writing a
|
||||
malicious file to disk.
|
||||
|
||||
## Why this matters even when "nothing is currently wrong"
|
||||
|
||||
None of these settings fix a bug or a vulnerability — they limit the BLAST RADIUS if one exists
|
||||
that you don't yet know about. A container hardened with non-root, dropped capabilities, and a
|
||||
read-only filesystem gives a successful exploit dramatically less to work with than an
|
||||
unhardened one, even against the exact same vulnerability. This is the entire point of defense
|
||||
in depth: you don't rely on any single layer being perfect.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10-12 minutes live with the demonstration.
|
||||
|
||||
**Demo plan:** Have the demonstration activity ready to run right after this lesson — seeing a
|
||||
concrete before/after Pod definition lands the abstract "least privilege" principle far better
|
||||
than the lesson text alone.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students sometimes think hardening settings are only relevant "if you expect to be attacked."
|
||||
Reframe: defense in depth assumes SOMETHING will eventually go wrong somewhere (a
|
||||
vulnerability, a bug, a misconfiguration) and prepares for that, rather than betting
|
||||
everything on nothing ever going wrong.
|
||||
- `runAsNonRoot: true` failing a Pod's startup is sometimes read as "the security setting is
|
||||
broken." It's the opposite — it's correctly refusing to let a root-requiring image start,
|
||||
which is valuable, actionable information about the image itself.
|
||||
|
||||
**Discussion question:** "If dropping all capabilities breaks a container that turns out to
|
||||
genuinely need one specific capability, what's the right fix — removing the drop, or something
|
||||
more targeted?" (Answer: add back only that ONE specific capability rather than removing the
|
||||
blanket drop — least privilege means "exactly what's needed," not "give up and grant
|
||||
everything again.")
|
||||
|
||||
**Optional extension:** ask students to identify one setting from this lesson that would have
|
||||
made zero difference against a vulnerability that lets an attacker read environment variables
|
||||
(a good check that they understand these are layered, not universal, protections — the answer
|
||||
is that none of SecurityContext's settings here directly protect secrets stored in env vars;
|
||||
that's a different concern, covered later in this course).
|
||||
+87
@@ -0,0 +1,87 @@
|
||||
# Demonstration: A Pod Before and After Hardening
|
||||
|
||||
## Before: the default, unhardened Pod
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: app-unhardened
|
||||
spec:
|
||||
containers:
|
||||
- name: app
|
||||
image: myapp:latest
|
||||
```
|
||||
|
||||
Checking what this actually runs as:
|
||||
|
||||
```
|
||||
$ kubectl exec app-unhardened -- id
|
||||
uid=0(root) gid=0(root) groups=0(root)
|
||||
|
||||
$ kubectl exec app-unhardened -- touch /etc/anything-i-want
|
||||
$ kubectl exec app-unhardened -- ls -la /etc/anything-i-want
|
||||
-rw-r--r-- 1 root root 0 Jan 1 00:00 /etc/anything-i-want
|
||||
```
|
||||
|
||||
Running as root, with a fully writable filesystem, including system directories. This is not a
|
||||
misconfiguration — it's simply what happens when nothing restricts it.
|
||||
|
||||
## After: the same application, hardened
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: app-hardened
|
||||
spec:
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
containers:
|
||||
- name: app
|
||||
image: myapp:latest
|
||||
securityContext:
|
||||
readOnlyRootFilesystem: true
|
||||
capabilities:
|
||||
drop: ["ALL"]
|
||||
```
|
||||
|
||||
```
|
||||
$ kubectl exec app-hardened -- id
|
||||
uid=1000 gid=0 groups=0
|
||||
|
||||
$ kubectl exec app-hardened -- touch /etc/anything-i-want
|
||||
touch: /etc/anything-i-want: Read-only file system
|
||||
```
|
||||
|
||||
Same application image, same functionality (assuming the app doesn't itself require root or
|
||||
filesystem writes it wasn't told about) — but a compromise of this container now can't write
|
||||
to the filesystem, can't use the elevated capabilities that were dropped, and never had root to
|
||||
begin with.
|
||||
|
||||
## What this actually buys you
|
||||
|
||||
Nothing here fixed a bug in `myapp:latest` — the image is identical in both cases. What changed
|
||||
is what an attacker (or a bug behaving unexpectedly) could DO if something inside that container
|
||||
went wrong. The unhardened version had almost no limits; the hardened version has several
|
||||
independent ones, any of which alone would have blocked the `touch` command above.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Run both Pods live if time allows — watching `touch` succeed on one and fail
|
||||
on the other in real time is far more convincing than reading it.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; 5-7 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students sometimes read the hardened Pod's failed `touch`
|
||||
as "hardening broke something." Reinforce: the FUNCTIONALITY didn't change (the application
|
||||
itself never needed to write to `/etc`) — only an unnecessary, unused capability was removed.
|
||||
If hardening breaks something the application genuinely needs, that's real information about
|
||||
the application, not a reason to skip hardening.
|
||||
|
||||
**Optional extension:** ask what would happen if `myapp:latest` were built assuming it could
|
||||
write a cache file to `/tmp` inside the container — a good bridge to the fact that
|
||||
`readOnlyRootFilesystem` often needs an explicit `emptyDir` volume mounted at the specific path
|
||||
an application legitimately needs to write to (ROOT 08), rather than being an all-or-nothing
|
||||
choice.
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
# Knowledge Check: Security Model & SecurityContext
|
||||
|
||||
---
|
||||
|
||||
**1. A container is hardened with `capabilities: drop: ["ALL"]` and then fails because it
|
||||
genuinely needs to bind to a low-numbered network port, which requires `NET_BIND_SERVICE`.
|
||||
What's the least-privilege fix?**
|
||||
|
||||
*Answer:* Add back only `NET_BIND_SERVICE` specifically (`capabilities: add:
|
||||
["NET_BIND_SERVICE"]` alongside the drop-all), not remove the blanket drop entirely.
|
||||
Least privilege means granting exactly what's needed, not reverting to granting everything.
|
||||
|
||||
---
|
||||
|
||||
**2. True or false: setting `runAsNonRoot: true` fixes a vulnerability inside the application
|
||||
itself.**
|
||||
|
||||
*Answer:* False. It doesn't fix anything about the application's code — it limits what a
|
||||
compromise of that code could DO, by ensuring the process never has root privileges to begin
|
||||
with. This is the defense-in-depth distinction: reducing blast radius, not eliminating bugs.
|
||||
|
||||
---
|
||||
|
||||
**3. Why might `runAsNonRoot: true` cause a Pod to fail to start entirely, and is that a
|
||||
problem with the setting?**
|
||||
|
||||
*Answer:* If the container image was built assuming it runs as root (e.g. it tries to bind a
|
||||
privileged port, or write to a root-owned directory), the Pod will fail to start under
|
||||
`runAsNonRoot`. This is NOT a problem with the setting — it's valuable, actionable information
|
||||
that the image itself makes an assumption that should probably be fixed.
|
||||
|
||||
---
|
||||
|
||||
**4. A container has `readOnlyRootFilesystem: true` and needs to write temporary cache files.
|
||||
What's the correct way to allow that without abandoning the read-only filesystem entirely?**
|
||||
|
||||
*Answer:* Mount an explicit `emptyDir` volume (ROOT 08) at the specific path the application
|
||||
needs to write to. This keeps the rest of the filesystem read-only while allowing the one
|
||||
legitimately-needed writable location.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 1 and question 4 both reinforce the same "narrow the exception, don't
|
||||
remove the protection" pattern — worth calling out that they're the same principle applied
|
||||
twice.
|
||||
|
||||
**Approximate timing:** 5-7 minutes.
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
# Guided Exercise: Planning a Pod Hardening Checklist
|
||||
|
||||
Paper-and-thinking exercise before the Labs.
|
||||
|
||||
## The scenario
|
||||
|
||||
You're handed a Pod spec for a small internal API with NO `securityContext` set at all — the
|
||||
fully-default, unhardened case from the demonstration. Your team lead asks you to harden it
|
||||
before it ships.
|
||||
|
||||
## Your task
|
||||
|
||||
Without writing the actual YAML yet, answer these:
|
||||
|
||||
1. List the settings you'd add, in the order you'd verify them (hint: there are at least three
|
||||
distinct concerns from the lesson — user/root, filesystem, capabilities).
|
||||
2. For each setting, name ONE way you'd notice if you'd broken the application by adding it
|
||||
(what would the failure actually look like?).
|
||||
3. If the application genuinely needs to write a cache file to disk, does that mean you should
|
||||
skip `readOnlyRootFilesystem` entirely? What's the more precise fix?
|
||||
4. Suppose after hardening, the container still runs fine but a security scan flags that it
|
||||
still has the `NET_RAW` capability. What's the most likely reason, and what's the fix?
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. In a reasonable order: (a) `runAsNonRoot`/`runAsUser` — a fundamental identity change,
|
||||
surfaces image assumptions fastest; (b) `capabilities: drop: ["ALL"]` — narrow, usually safe
|
||||
for a typical web API; (c) `readOnlyRootFilesystem: true` — most likely to need an
|
||||
accompanying volume, so verify last once the others are confirmed working.
|
||||
2. (a) Pod fails to start entirely (`CrashLoopBackOff` or an immediate error) if the image
|
||||
assumed root. (b) The application errors on a specific operation that needed a capability
|
||||
you dropped (e.g. failing to bind a privileged port). (c) A write operation fails at runtime
|
||||
(a log line, an exception) the first time the app tries to write to disk.
|
||||
3. No — the fix is mounting an explicit `emptyDir` volume at the specific path the cache needs,
|
||||
keeping the rest of the filesystem read-only rather than abandoning the protection entirely.
|
||||
4. Most likely: `capabilities: drop: ["ALL"]` was never actually applied (a YAML indentation
|
||||
mistake, or it was added to the Pod's securityContext instead of the container's — capability
|
||||
drops are container-level, not Pod-level). The fix is verifying the drop is actually present
|
||||
on the CONTAINER's securityContext specifically.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 4 is deliberately tricky — Pod-level vs. container-level
|
||||
`securityContext` fields is a real, common source of "I set this and it didn't work" confusion
|
||||
worth surfacing here before students hit it in a Lab.
|
||||
|
||||
**Approximate timing:** 15 minutes live with discussion; 12-15 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask students to identify which of the settings in this exercise would
|
||||
be MOST valuable to apply first if they could only add one to a large, existing fleet of
|
||||
unhardened Pods under time pressure, and why.
|
||||
+42
@@ -0,0 +1,42 @@
|
||||
# Reflection: Least Privilege Is a Default, Not an Afterthought
|
||||
|
||||
You just hardened real Pods across several Labs — non-root, dropped capabilities, a read-only
|
||||
filesystem, and a full SecurityContext review. Take a few minutes to connect this to real
|
||||
practice.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Before this module, would you have described "runs as root with a fully writable
|
||||
filesystem" as insecure, or just as "the default, unremarkable configuration"? Has that
|
||||
changed?
|
||||
2. None of the settings you configured fixed a bug in any application. What did they actually
|
||||
change, precisely?
|
||||
3. If your team had to choose between hardening every new Pod by default (as a policy) versus
|
||||
hardening Pods only after an incident reveals a gap, which produces less total risk over
|
||||
time, and why?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**Unhardened is not neutral — it's the maximum-privilege default.** A container with no
|
||||
SecurityContext configured isn't "unopinionated"; it's root, every capability, fully writable —
|
||||
the MOST privileged configuration possible, not a safe middle ground. Teams that treat hardening
|
||||
as optional are choosing maximum blast radius by default, usually without realizing they're
|
||||
choosing anything at all.
|
||||
|
||||
**Defense in depth means no single mistake should be catastrophic.** The value of stacking
|
||||
non-root + dropped capabilities + read-only filesystem isn't that any ONE of them is perfect —
|
||||
it's that a real compromise would have to defeat several independent restrictions simultaneously
|
||||
instead of exploiting one gap that grants everything. This is exactly why security-minded teams
|
||||
apply all of these by default rather than picking "the one that matters most."
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 3 is worth pushing on directly — it's
|
||||
the actual business case for security-by-default, not just a technical preference.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to draft a one-paragraph "default Pod hardening policy"
|
||||
their hypothetical team would adopt, based on what they've learned in this module.
|
||||
+90
@@ -0,0 +1,90 @@
|
||||
# Identity for Workloads: ServiceAccounts & RBAC
|
||||
|
||||
## Workloads need an identity too, separate from yours
|
||||
|
||||
When YOU run `kubectl`, you're authenticated as yourself. But a Pod running INSIDE the cluster
|
||||
that needs to call the Kubernetes API (to list other Pods, read a ConfigMap, whatever) needs its
|
||||
own identity — it can't and shouldn't use your personal credentials. That identity is a
|
||||
**ServiceAccount**.
|
||||
|
||||
Every Pod runs as SOME ServiceAccount — if you don't specify one, it uses the `default`
|
||||
ServiceAccount in its Namespace, which by default has essentially no permissions. A Pod's
|
||||
ServiceAccount token is automatically mounted into the container unless you explicitly disable
|
||||
that (`automountServiceAccountToken: false`) — worth doing for any Pod that never actually needs
|
||||
to call the Kubernetes API at all, since an unused, mounted credential is still something an
|
||||
attacker could steal and use.
|
||||
|
||||
## RBAC: what that identity is actually allowed to do
|
||||
|
||||
Having an identity (a ServiceAccount) doesn't grant any permissions by itself — **RBAC
|
||||
(Role-Based Access Control)** is the separate system that defines what an identity can DO.
|
||||
|
||||
- A **Role** (namespace-scoped) or **ClusterRole** (cluster-wide) defines a set of permissions:
|
||||
which API resources, which verbs (get/list/watch/create/update/delete).
|
||||
- A **RoleBinding** (or **ClusterRoleBinding**) connects a Role/ClusterRole to a specific
|
||||
identity (a ServiceAccount, a user, or a group) — the Role alone grants nothing to anyone
|
||||
until it's bound.
|
||||
|
||||
```yaml
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: pod-reader
|
||||
namespace: default
|
||||
rules:
|
||||
- apiGroups: [""]
|
||||
resources: ["pods"]
|
||||
verbs: ["get", "list"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: read-pods
|
||||
namespace: default
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: my-app
|
||||
roleRef:
|
||||
kind: Role
|
||||
name: pod-reader
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
```
|
||||
|
||||
This grants the `my-app` ServiceAccount permission to `get`/`list` Pods in the `default`
|
||||
Namespace — and nothing else. It cannot delete Pods, cannot touch Secrets, cannot do anything
|
||||
not explicitly listed.
|
||||
|
||||
## Role vs. ClusterRole: scope matters
|
||||
|
||||
A Role only grants permissions within the ONE Namespace it's defined in. A ClusterRole can grant
|
||||
permissions across the entire cluster (or, bound via a RoleBinding instead of a
|
||||
ClusterRoleBinding, can also be reused namespace-by-namespace) — reach for a ClusterRole only
|
||||
when the workload genuinely needs cluster-wide visibility, not by default.
|
||||
|
||||
## Least privilege, applied to permissions specifically
|
||||
|
||||
The same principle from the previous module applies directly here: grant the narrowest set of
|
||||
verbs on the narrowest set of resources that the workload actually needs. A Role granting
|
||||
`get`/`list` on Pods is very different from one granting `*` on `*` — the latter is
|
||||
"root-equivalent" API access, and should be treated with exactly that level of caution.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often conflate "ServiceAccount" with "user account for logging in." A ServiceAccount
|
||||
is specifically an identity for WORKLOADS, not a human login mechanism.
|
||||
- The Role/RoleBinding two-step (define permissions, then separately grant them to someone) is
|
||||
sometimes seen as redundant. Emphasize the reuse benefit: the same Role can be bound to
|
||||
multiple identities without redefining the permission set each time.
|
||||
|
||||
**Discussion question:** "Why might you explicitly disable automounting a ServiceAccount token
|
||||
for a Pod that never calls the Kubernetes API?" (Answer: an unused but present credential is
|
||||
still something an attacker who compromises the container could steal and use elsewhere —
|
||||
removing it entirely removes that risk for a Pod that never needed it.)
|
||||
|
||||
**Optional extension:** ask what the practical difference is between granting a ClusterRole via
|
||||
a RoleBinding (scoped to one Namespace) versus a ClusterRoleBinding (cluster-wide) — a good
|
||||
check that students understand the Role/ClusterRole and Binding-type distinctions are
|
||||
independent axes, not the same choice.
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
# Knowledge Check: ServiceAccounts & RBAC
|
||||
|
||||
---
|
||||
|
||||
**1. A Pod is created with no ServiceAccount specified. What identity does it actually run as,
|
||||
and what can that identity do by default?**
|
||||
|
||||
*Answer:* The `default` ServiceAccount in its Namespace, which by default has essentially no
|
||||
permissions — it can authenticate to the API but is granted nothing until an RBAC binding says
|
||||
otherwise.
|
||||
|
||||
---
|
||||
|
||||
**2. You create a Role granting `get`/`list` on Pods, but a workload still can't read Pods.
|
||||
What's the most likely missing piece?**
|
||||
|
||||
*Answer:* A RoleBinding (or ClusterRoleBinding) connecting that Role to the workload's actual
|
||||
ServiceAccount. A Role by itself grants nothing to anyone — it has to be bound to an identity.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: a ClusterRole can only ever be applied cluster-wide.**
|
||||
|
||||
*Answer:* False. A ClusterRole can be bound via a RoleBinding to apply within just ONE
|
||||
Namespace (reusing a cluster-wide-DEFINED permission set in a namespace-SCOPED way), or via a
|
||||
ClusterRoleBinding to apply cluster-wide. The Role-type and Binding-type are independent
|
||||
choices.
|
||||
|
||||
---
|
||||
|
||||
**4. Why would a security review flag a Role granting `verbs: ["*"]` on `resources: ["*"]`
|
||||
even if the workload "seems to work fine"?**
|
||||
|
||||
*Answer:* That grant is root-equivalent API access — it violates least privilege regardless of
|
||||
whether the workload happens to only use a small subset of it today. "Works fine" doesn't mean
|
||||
"correctly scoped" — an over-broad grant is a real risk the moment that workload (or its
|
||||
credentials) is compromised, whether or not it's ever exploited in practice.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 4 is worth discussing even if everyone answers correctly — the "it
|
||||
works, so what's the problem" instinct is exactly what least-privilege review exists to counter.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+62
@@ -0,0 +1,62 @@
|
||||
# Demonstration: Watching RBAC Actually Deny Something
|
||||
|
||||
## The setup
|
||||
|
||||
A ServiceAccount `readonly-bot` has a Role granting only `get`/`list` on `pods` — nothing else.
|
||||
|
||||
## What it CAN do
|
||||
|
||||
```
|
||||
$ kubectl auth can-i list pods --as=system:serviceaccount:default:readonly-bot
|
||||
yes
|
||||
|
||||
$ kubectl get pods --as=system:serviceaccount:default:readonly-bot
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
web-abc123 1/1 Running 0 2h
|
||||
```
|
||||
|
||||
## What it CANNOT do
|
||||
|
||||
```
|
||||
$ kubectl auth can-i delete pods --as=system:serviceaccount:default:readonly-bot
|
||||
no
|
||||
|
||||
$ kubectl delete pod web-abc123 --as=system:serviceaccount:default:readonly-bot
|
||||
Error from server (Forbidden): pods "web-abc123" is forbidden:
|
||||
User "system:serviceaccount:default:readonly-bot" cannot delete resource "pods"
|
||||
in API group "" in the namespace "default"
|
||||
```
|
||||
|
||||
This is what least privilege actually looks like in practice — not a policy document, a real,
|
||||
enforced denial with a specific, readable error naming exactly which permission was missing.
|
||||
`kubectl auth can-i` is your fastest way to CHECK a permission before attempting an action that
|
||||
might fail — useful both for debugging "why can't my workload do X" and for verifying a Role is
|
||||
actually as narrow as intended.
|
||||
|
||||
## What happens when the ServiceAccount tries something it was never even considered for
|
||||
|
||||
```
|
||||
$ kubectl auth can-i get secrets --as=system:serviceaccount:default:readonly-bot
|
||||
no
|
||||
```
|
||||
|
||||
Notice this returns `no` the same way `delete pods` did — RBAC has no concept of "not yet
|
||||
decided," only "explicitly allowed" or "everything else, denied." This is the practical meaning
|
||||
of default-deny applied to permissions: nothing is grey area.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Run all three `kubectl auth can-i`/action pairs live — the FORBIDDEN error
|
||||
message's exact wording (naming the verb, resource, and namespace) is worth reading aloud, since
|
||||
that's exactly the evidence a student would use to debug a real permissions issue later.
|
||||
|
||||
**Approximate timing:** 8 minutes live; 5 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students sometimes think `kubectl auth can-i` REQUIRES the
|
||||
action to actually be attempted to know the answer. It doesn't — it's a dry-run permission
|
||||
check against the same RBAC rules, safe to run repeatedly without side effects.
|
||||
|
||||
**Optional extension:** ask what the error message would look like if `readonly-bot` tried to
|
||||
act on a DIFFERENT namespace than the one its Role/RoleBinding applies to — a good check that
|
||||
students understand Role scope is namespace-specific unless a ClusterRole+ClusterRoleBinding is
|
||||
used instead.
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
# Guided Exercise: Writing a Least-Privilege Role
|
||||
|
||||
Paper-and-thinking exercise before the Labs.
|
||||
|
||||
## The scenario
|
||||
|
||||
A monitoring agent needs to run inside your cluster. Its ONLY job is to periodically list Pods
|
||||
and read their status, across the whole cluster, to report health metrics elsewhere. It never
|
||||
needs to create, modify, or delete anything, and it never needs to touch Secrets, ConfigMaps, or
|
||||
any object type other than Pods.
|
||||
|
||||
## Your task
|
||||
|
||||
1. Should this use a Role or a ClusterRole? Justify your answer based on scope.
|
||||
2. Which verbs does it actually need? Which common verbs (create, update, delete, patch) should
|
||||
be deliberately left OUT?
|
||||
3. Does it need access to Secrets? Why is it tempting to grant broader access "just in case,"
|
||||
and why should you resist that?
|
||||
4. What would you check, using `kubectl auth can-i`, to CONFIRM your Role is exactly as narrow
|
||||
as intended before deploying it?
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. **ClusterRole** (bound via a ClusterRoleBinding) — it needs to see Pods across the WHOLE
|
||||
cluster, not just one namespace, so a namespace-scoped Role can't satisfy the requirement.
|
||||
2. **`get`, `list`, `watch`** on `pods` — read-only verbs only. Deliberately exclude `create`,
|
||||
`update`, `patch`, `delete` — this workload never legitimately needs to change anything.
|
||||
3. No — it never touches Secrets, so it should get zero permission on them. The temptation to
|
||||
"just in case" grant broader access is exactly what least privilege exists to resist: every
|
||||
unused permission is pure risk with no corresponding benefit if this workload is ever
|
||||
compromised.
|
||||
4. Run `kubectl auth can-i delete pods --as=<the ServiceAccount>` and confirm it returns `no` —
|
||||
and similarly check a Secrets verb returns `no` — proving the Role is bounded, not just
|
||||
assuming it from reading the YAML.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 3 is the core judgment call this exercise is testing — push on WHY
|
||||
"just in case" access is a real anti-pattern, not just point out that it is one.
|
||||
|
||||
**Approximate timing:** 10-12 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what would change about the answer if this monitoring agent later
|
||||
needed to also restart crash-looping Pods automatically — a good check that students can reason
|
||||
about incrementally expanding a Role's scope only as far as a genuinely new requirement demands.
|
||||
+42
@@ -0,0 +1,42 @@
|
||||
# Reflection: Permissions Should Reflect What a Workload Actually Needs
|
||||
|
||||
You just configured real ServiceAccounts and RBAC rules. Take a few minutes to connect this to
|
||||
production security practice.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Before this module, would you have thought twice about a workload's ServiceAccount having
|
||||
broad, unused permissions "just in case"? Has that changed?
|
||||
2. If a workload with cluster-admin-equivalent RBAC access is ever compromised (a vulnerable
|
||||
dependency, a supply-chain issue), what's the realistic blast radius compared to one scoped
|
||||
to exactly what it needs?
|
||||
3. How would you discover, on a real cluster you didn't build, whether any ServiceAccount has
|
||||
more access than it actually uses?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**RBAC over-permissioning is a silent, cumulative risk.** Unlike a misconfigured probe or a
|
||||
crashing Pod, an over-broad Role doesn't cause a visible symptom on a healthy day — it just sits
|
||||
there as unused risk until the one day a workload using it is compromised. This makes RBAC
|
||||
review a discipline that has to be practiced deliberately, not one that gets forced on you by
|
||||
an obvious failure.
|
||||
|
||||
**Least privilege is a default you set once, not a fix you apply after an incident.** Every
|
||||
example in this module started from "grant nothing, add exactly what's needed" rather than
|
||||
"grant broadly, restrict later" — the second approach almost never actually happens once
|
||||
something works, because there's no pressure forcing the cleanup.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 3 is a good one to leave partially open
|
||||
— acknowledge that real answers involve tooling beyond plain `kubectl` (RBAC audit tools) that
|
||||
this course doesn't build hands-on, while still reinforcing that the QUESTION itself is the
|
||||
right one to ask.
|
||||
|
||||
**Approximate timing:** 8 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to identify one ServiceAccount permission from earlier in
|
||||
ROOT_ (any Lab so far) that, in a real production system, they'd want to double-check was
|
||||
actually necessary.
|
||||
+76
@@ -0,0 +1,76 @@
|
||||
# NetworkPolicy as a Security Control & Supply Chain Trust
|
||||
|
||||
## NetworkPolicy, revisited as a security posture
|
||||
|
||||
ROOT 07 introduced NetworkPolicy architecturally: Kubernetes doesn't isolate Pod-to-Pod traffic
|
||||
by default, and NetworkPolicy is how you declare intended reachability. Here, the same
|
||||
mechanism is the point of a deliberate SECURITY practice: **default deny**.
|
||||
|
||||
A default-deny policy blocks all traffic to selected Pods, and you then explicitly allow only
|
||||
what's actually needed:
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-deny
|
||||
spec:
|
||||
podSelector: {} # selects EVERY Pod in the namespace
|
||||
policyTypes: ["Ingress"]
|
||||
```
|
||||
|
||||
An empty `podSelector: {}` with no ingress rules means "select every Pod, allow nothing in" —
|
||||
the network equivalent of the SecurityContext principle from earlier in this course: start
|
||||
from nothing, add back only what's proven necessary. This is exactly the opposite of
|
||||
Kubernetes' un-configured default (everything open), which is precisely why it has to be
|
||||
declared explicitly rather than relying on any built-in setting.
|
||||
|
||||
**Egress restriction** works the same way in reverse — restricting what a Pod can call OUT to,
|
||||
not just what can call IN. A compromised Pod that can freely make outbound connections to
|
||||
anything is far more dangerous (data exfiltration, calling out to an attacker's server) than one
|
||||
restricted to only the specific destinations it legitimately needs.
|
||||
|
||||
## Supply chain: trusting what you run before it ever starts
|
||||
|
||||
Everything covered so far in this course assumes the container image itself is trustworthy.
|
||||
Supply-chain security is about that assumption specifically:
|
||||
|
||||
- **Image-pull secrets** — when pulling from a private registry, the cluster needs credentials
|
||||
to authenticate. Storing and using these correctly (rather than, say, a publicly-readable
|
||||
registry with no access control) is a basic supply-chain control: only images from a trusted,
|
||||
access-controlled source get pulled at all.
|
||||
- **Minimizing image footprint** (referenced here, not a hands-on Lab in this module) — a
|
||||
smaller image means fewer packages, fewer known CVEs, and a smaller attack surface if the
|
||||
image is ever compromised.
|
||||
|
||||
## TLS at the Ingress layer: protecting data in transit
|
||||
|
||||
Recall ROOT 07's Ingress lesson — routing external HTTP(S) traffic by hostname/path. Configuring
|
||||
TLS termination at that same Ingress layer means traffic between users and your cluster is
|
||||
encrypted, protecting against interception on the network path. This is a security concern
|
||||
layered on top of the same architectural mechanism, the same relationship NetworkPolicy has to
|
||||
its ROOT 07 introduction.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students sometimes think "default deny" means blocking ALL traffic including what the
|
||||
application needs to function. Clarify: default deny is the STARTING point; you then add
|
||||
specific allow rules for exactly what's needed — the goal state is fully functional AND fully
|
||||
intentional, not non-functional.
|
||||
- Egress restriction is sometimes seen as less important than ingress. Emphasize the
|
||||
exfiltration angle explicitly — a compromised Pod's outbound reach is just as much a real
|
||||
risk as inbound exposure.
|
||||
|
||||
**Discussion question:** "Why is 'default deny, then explicitly allow' a better starting
|
||||
posture than 'default allow, then explicitly block known-bad traffic'?" (Answer: default-deny
|
||||
means a NEW, unanticipated attempt to reach or leave a Pod is blocked by default; default-allow
|
||||
means anything not YET identified as bad gets through — you're always one step behind an
|
||||
attacker under the second model.)
|
||||
|
||||
**Optional extension:** ask what a real team would want to verify before trusting a
|
||||
publicly-available container image from an unfamiliar source, beyond just "does it work" — a
|
||||
good bridge into image scanning/provenance concepts (not built as a hands-on Lab in this course,
|
||||
but worth naming as a real practice).
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
# Knowledge Check: Network & Supply Chain Security
|
||||
|
||||
---
|
||||
|
||||
**1. A NetworkPolicy has `podSelector: {}` with `policyTypes: ["Ingress"]` and no ingress rules
|
||||
defined. What's the practical effect on every Pod in that namespace?**
|
||||
|
||||
*Answer:* Every Pod is selected, and since no ingress rule allows anything, all incoming traffic
|
||||
to every Pod is denied — this is the "default deny" starting posture, applied namespace-wide.
|
||||
|
||||
---
|
||||
|
||||
**2. Why is restricting EGRESS (outbound) traffic from a Pod a real security control, not just
|
||||
a mirror-image nicety of ingress restriction?**
|
||||
|
||||
*Answer:* A compromised Pod with unrestricted egress can exfiltrate data to an attacker-controlled
|
||||
destination or call out to download additional malicious payloads. Restricting egress to only
|
||||
what's legitimately needed limits what a compromise can accomplish, independent of whether
|
||||
inbound access was ever an issue.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: "default deny, then explicitly allow" and "default allow, then explicitly
|
||||
block known-bad traffic" provide equivalent security given the same set of rules.**
|
||||
|
||||
*Answer:* False. Default-deny blocks anything NOT yet explicitly allowed, including new/
|
||||
unanticipated attempts. Default-allow only blocks what's already been identified as bad — an
|
||||
attacker doing something not yet on the block list gets through under the second model but not
|
||||
the first.
|
||||
|
||||
---
|
||||
|
||||
**4. Why does minimizing a container image's footprint (fewer packages) improve supply-chain
|
||||
security specifically, rather than just making the image smaller/faster to pull?**
|
||||
|
||||
*Answer:* Fewer installed packages means fewer potential known vulnerabilities (CVEs) present in
|
||||
the image at all, and a smaller attack surface if the image is ever compromised — it's a
|
||||
security property, not merely a performance optimization.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 3 is the core conceptual point of this module — worth confirming it's
|
||||
genuinely understood, not just correctly guessed.
|
||||
|
||||
**Approximate timing:** 5-7 minutes.
|
||||
+89
@@ -0,0 +1,89 @@
|
||||
# Demonstration: Default-Deny, Before and After
|
||||
|
||||
## Before: everything can reach everything
|
||||
|
||||
```
|
||||
$ kubectl run test-pod --image=busybox --rm -it -- wget -qO- --timeout=2 http://backend-svc
|
||||
<real response from backend>
|
||||
```
|
||||
|
||||
Any Pod in the namespace can reach `backend-svc`, regardless of what it is or who deployed it —
|
||||
this is the un-configured default the ROOT 07 lesson described, now made concrete from a
|
||||
security angle.
|
||||
|
||||
## Applying a default-deny policy
|
||||
|
||||
```
|
||||
$ kubectl apply -f default-deny.yaml
|
||||
networkpolicy.networking.k8s.io/default-deny created
|
||||
```
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-deny
|
||||
spec:
|
||||
podSelector: {}
|
||||
policyTypes: ["Ingress"]
|
||||
```
|
||||
|
||||
## After: the same request, now blocked
|
||||
|
||||
```
|
||||
$ kubectl run test-pod --image=busybox --rm -it -- wget -qO- --timeout=2 http://backend-svc
|
||||
wget: download timed out
|
||||
```
|
||||
|
||||
No error naming a permission (that's RBAC's job, a different layer) — just a timeout, because
|
||||
the packets are being dropped at the network layer before they ever reach the Pod. This is a
|
||||
DIFFERENT failure signature than an RBAC denial, worth being able to tell apart: RBAC denials
|
||||
are explicit and immediate; NetworkPolicy denials look like the destination is simply
|
||||
unreachable.
|
||||
|
||||
## Explicitly allowing only what's needed
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: allow-frontend-to-backend
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels: { app: backend }
|
||||
ingress:
|
||||
- from:
|
||||
- podSelector:
|
||||
matchLabels: { app: frontend }
|
||||
```
|
||||
|
||||
```
|
||||
$ kubectl run frontend-test --labels="app=frontend" --image=busybox --rm -it -- wget -qO- --timeout=2 http://backend-svc
|
||||
<real response from backend>
|
||||
|
||||
$ kubectl run other-test --labels="app=other" --image=busybox --rm -it -- wget -qO- --timeout=2 http://backend-svc
|
||||
wget: download timed out
|
||||
```
|
||||
|
||||
Same destination, same request — the ONLY difference is the caller's label. This is
|
||||
least-privilege network access made concrete: reachability now reflects an explicit decision,
|
||||
not an accident of both Pods existing in the same cluster.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Run all three states live if a real cluster is available — the timeout (not an
|
||||
error) for the denied case is worth calling out explicitly, since it looks identical to a
|
||||
genuinely broken/unreachable service from the caller's perspective.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; 5-7 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students sometimes expect a NetworkPolicy denial to
|
||||
produce a clear "forbidden" error like RBAC does. It doesn't — reinforce that a NetworkPolicy-
|
||||
caused timeout is genuinely hard to distinguish from an actual network/DNS problem without
|
||||
already suspecting NetworkPolicy, which is exactly why checking existing policies is a real
|
||||
diagnostic step worth knowing.
|
||||
|
||||
**Optional extension:** ask how a student would distinguish "blocked by NetworkPolicy" from
|
||||
"the backend Pod is actually down" purely from the caller's side (answer: check the backend
|
||||
Pod's own status/logs directly — if it's healthy and Ready, but still unreachable from one
|
||||
caller and reachable from another with a different label, NetworkPolicy is the likely cause).
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
# Guided Exercise: Planning Egress Restriction
|
||||
|
||||
Paper-and-thinking exercise before the Labs.
|
||||
|
||||
## The scenario
|
||||
|
||||
A `payment-processor` Pod handles sensitive transactions. It legitimately needs to reach:
|
||||
- an internal `payments-db` Service, and
|
||||
- one external payment gateway API.
|
||||
|
||||
It should NOT be able to reach anything else — including the public internet in general, other
|
||||
internal services, or any other namespace.
|
||||
|
||||
## Your task
|
||||
|
||||
1. Why is restricting this Pod's OUTBOUND (egress) traffic just as important as restricting who
|
||||
can reach IT (ingress), given what this Pod handles?
|
||||
2. If this Pod were compromised via a vulnerable dependency, what would UNRESTRICTED egress let
|
||||
an attacker do that restricted egress would prevent?
|
||||
3. Sketch (in words) what an egress policy for this Pod needs to allow, and — just as
|
||||
important — what it must NOT allow by omission.
|
||||
4. Why might a security review flag a Pod with `policyTypes: ["Egress"]` but an EMPTY `egress`
|
||||
list as suspicious, rather than assume it's simply "not configured yet"?
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. A Pod handling sensitive data that's compromised is far more dangerous if it can freely
|
||||
exfiltrate that data anywhere (unrestricted egress) than if its only reachable destinations
|
||||
are pre-approved and narrow.
|
||||
2. Unrestricted egress would let an attacker exfiltrate transaction data to an external server,
|
||||
or download additional malicious tooling. Restricted egress limits it to only the
|
||||
pre-approved destinations, even after compromise.
|
||||
3. Allow: egress to `payments-db` (internal, by label/namespace selector) and the specific
|
||||
external payment gateway (by IP range or, if supported, a domain-based egress rule). Must NOT
|
||||
allow: general internet access, other internal Services, or any other namespace by default.
|
||||
4. An empty `egress` list under `policyTypes: ["Egress"]` means DENY ALL outbound — exactly like
|
||||
the ingress case from the lesson. A reviewer seeing this should confirm it's intentional
|
||||
(a Pod that genuinely needs zero outbound network access) rather than assume it's an
|
||||
oversight, since the two look identical in the YAML.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 4 reinforces the same "empty rules list means deny-all, not
|
||||
unconfigured" point from earlier lessons — a good moment to confirm it generalizes correctly to
|
||||
egress, not just ingress.
|
||||
|
||||
**Approximate timing:** 10-12 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what additional control (beyond NetworkPolicy) a team might want for
|
||||
truly sensitive egress, like restricting DNS resolution itself for this Pod — a forward pointer
|
||||
to defense-in-depth without requiring a hands-on Lab here.
|
||||
+41
@@ -0,0 +1,41 @@
|
||||
# Reflection: Reachability and Trust Are Decisions You Make on Purpose
|
||||
|
||||
You just applied default-deny NetworkPolicy, egress restriction, TLS ingress, and image-pull
|
||||
authentication for real. Take a few minutes to connect these to a single underlying principle.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Four different mechanisms in this module (NetworkPolicy ingress, NetworkPolicy egress, TLS,
|
||||
image-pull secrets) all express the same underlying idea in a different layer. What is that
|
||||
idea, stated in one sentence?
|
||||
2. If you had to prioritize implementing ONLY ONE of this module's controls on a brand-new,
|
||||
unsecured cluster, which would you pick first, and why?
|
||||
3. How would you explain, to someone who thinks "our cluster is internal, so it doesn't need
|
||||
this," why internal-only still benefits from all four controls?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**Trust should be explicit, narrow, and verifiable — never assumed from proximity.** A Pod on
|
||||
the same cluster, a request from inside the corporate network, an image from "a registry we
|
||||
usually use" — none of these are actually trustworthy by default. Every control in this module
|
||||
replaces an assumption ("it's internal, it's probably fine") with an explicit, checkable
|
||||
decision (this specific Pod can reach this specific destination; this specific registry is
|
||||
authenticated; this specific connection is encrypted).
|
||||
|
||||
**Supply-chain and network security compound.** A compromised image with unrestricted egress on
|
||||
an open network is a far worse incident than the same compromised image on a hardened one — the
|
||||
controls in this module don't individually prevent every incident, but together they
|
||||
dramatically shrink what any single failure can turn into.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 1 is the module's thesis — make sure
|
||||
students can actually articulate it in their own words, not just recognize it when stated.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to rank this module's four controls by how much
|
||||
IMPLEMENTATION EFFORT each typically requires on a real team, separately from security value —
|
||||
a good exercise in real-world prioritization tradeoffs.
|
||||
+64
@@ -0,0 +1,64 @@
|
||||
# Pod Security Admission & a Survey of Runtime Hardening
|
||||
|
||||
This module is deliberately a survey, not a deep dive — the topics here are real and important,
|
||||
but lean toward cluster-operator/platform-team concerns rather than the daily-driver skills most
|
||||
ROOT_ learners will use constantly. Treat the labs in this module as good-to-know, not required
|
||||
mastery, before moving on.
|
||||
|
||||
## Pod Security Admission: enforcing a baseline across a Namespace
|
||||
|
||||
Everything you configured by hand in earlier modules (non-root, dropped capabilities, read-only
|
||||
filesystem) can also be ENFORCED automatically at the Namespace level via **Pod Security
|
||||
Admission (PSA)** — a label on the Namespace itself that rejects any Pod violating a chosen
|
||||
security standard (`privileged`, `baseline`, or `restricted`, from least to most strict):
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: production
|
||||
labels:
|
||||
pod-security.kubernetes.io/enforce: restricted
|
||||
```
|
||||
|
||||
This replaces the older PodSecurityPolicy (PSP) mechanism, which was removed from Kubernetes —
|
||||
if you encounter PSP in older documentation or an older cluster, know that PSA is its current
|
||||
replacement.
|
||||
|
||||
## Immutable Secrets: preventing a class of accidental change
|
||||
|
||||
A Secret can be marked `immutable: true`, which prevents its data from being changed after
|
||||
creation (a new Secret with a new name has to be created instead, and workloads updated to
|
||||
reference it). This trades a small amount of operational flexibility for a real safety
|
||||
property: an accidental or malicious edit to a live Secret can't silently happen.
|
||||
|
||||
## A survey of further topics (labs available, treat as optional depth)
|
||||
|
||||
- **Audit policy** — configuring what the Kubernetes API server logs about requests made against
|
||||
it (who did what, when) — genuinely important for incident investigation, but a cluster-admin
|
||||
concern, not something most workload teams configure themselves.
|
||||
- **RuntimeClass** — lets specific Pods run under a different container runtime configuration
|
||||
(e.g. an extra-sandboxed runtime like gVisor or Kata Containers) for workloads that need
|
||||
stronger isolation than the default runtime provides — relevant mainly for genuinely
|
||||
untrusted or high-risk workloads, not typical applications.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read (shorter than other lessons — deliberately survey-depth).
|
||||
|
||||
**Framing for instructors:** Be explicit with students that this module is intentionally
|
||||
lighter than the earlier ones in this course — the goal is awareness ("this exists, here's
|
||||
roughly what it's for") rather than hands-on fluency, consistent with not every CKS-style topic
|
||||
belonging in the core learning path.
|
||||
|
||||
**Discussion question:** "Why would a platform team prefer enforcing security standards via
|
||||
Pod Security Admission at the Namespace level, instead of trusting every team to configure
|
||||
SecurityContext correctly on every Pod by hand?" (Answer: PSA provides a hard backstop —
|
||||
correctly-configured SecurityContext relies on every individual Pod author doing it right every
|
||||
time; PSA rejects violations automatically regardless of whether any individual author
|
||||
remembered to.)
|
||||
|
||||
**Optional extension:** ask which of `privileged`/`baseline`/`restricted` a team should start a
|
||||
brand-new, security-conscious project with, and why not jump straight to the strictest option
|
||||
on a cluster already running older, unaudited workloads (a good practical judgment question
|
||||
about migration risk, not just definitions).
|
||||
+86
@@ -0,0 +1,86 @@
|
||||
# Why Health Checks Exist & Probe Types
|
||||
|
||||
Kubernetes cannot see inside your application. It doesn't know if your process is deadlocked,
|
||||
stuck in an infinite loop, or quietly failing every request while the process itself stays
|
||||
alive. Without a way to ask "are you actually working?", the only signal Kubernetes has is
|
||||
whether the container's process is still running — and a hung process still counts as running.
|
||||
|
||||
**A health probe is how your application tells Kubernetes the truth.** It's a small check —
|
||||
an HTTP request, a TCP connection attempt, or a command run inside the container — that
|
||||
Kubernetes runs on a schedule and uses to make real decisions: restart this container, or stop
|
||||
sending it traffic.
|
||||
|
||||
## The three probe types
|
||||
|
||||
Kubernetes supports three distinct probes, and they answer three different questions. Mixing
|
||||
them up is the single most common probe mistake in production.
|
||||
|
||||
**Liveness probe — "Is this container still alive, or should it be restarted?"**
|
||||
If a liveness probe fails repeatedly, the kubelet kills the container and it gets restarted per
|
||||
its restart policy. Use this for detecting a genuinely stuck process — a deadlock, a crashed
|
||||
event loop. Don't use liveness to mean "not ready for traffic yet" — that's what readiness is
|
||||
for, and confusing the two causes the exact incident you're about to fix.
|
||||
|
||||
**Readiness probe — "Is this container ready to receive traffic right now?"**
|
||||
If a readiness probe fails, the Pod is removed from the Service's list of endpoints —
|
||||
Kubernetes stops routing traffic to it, but does **not** restart the container. This is the
|
||||
right tool for "still starting up," "warming a cache," or "temporarily overloaded, back off for
|
||||
a bit." A Pod can be Running and healthy but still correctly Not Ready.
|
||||
|
||||
**Startup probe — "Has this slow-starting container finished starting up yet?"**
|
||||
Some applications take a long time to become ready (loading a large dataset, running
|
||||
migrations). A startup probe disables liveness/readiness checks until it succeeds once, so a
|
||||
slow-but-healthy startup doesn't get killed by an impatient liveness probe. Once it succeeds,
|
||||
liveness and readiness take over as normal.
|
||||
|
||||
## How a probe is actually defined
|
||||
|
||||
Every probe has a **mechanism** (how the check runs) and a **schedule** (how often, how long to
|
||||
wait, how many failures before acting):
|
||||
|
||||
- `httpGet` — a GET request to a path and **port** on the container. A 2xx or 3xx response
|
||||
counts as success.
|
||||
- `tcpSocket` — a TCP connection attempt to a **port**. A successful connection counts as
|
||||
success — this doesn't check application-level health, just "is something listening."
|
||||
- `exec` — runs a command inside the container. Exit code `0` counts as success.
|
||||
|
||||
Timing fields (`initialDelaySeconds`, `periodSeconds`, `timeoutSeconds`, `failureThreshold`,
|
||||
`successThreshold`) control how patient Kubernetes is before it acts — but no amount of correct
|
||||
timing fixes a probe that's pointed at the wrong port or path in the first place.
|
||||
|
||||
## Common production mistakes
|
||||
|
||||
1. **Wrong port.** The probe's `httpGet.port` (or `tcpSocket.port`) doesn't match the port the
|
||||
application actually listens on. The container is completely healthy — Kubernetes is just
|
||||
asking the wrong door to open. This is the exact failure you'll diagnose in this module's lab.
|
||||
2. **Liveness probe too aggressive on a slow-starting app**, with no startup probe — the app
|
||||
gets killed for "not being ready yet," restarts, and never gets far enough to pass.
|
||||
3. **Readiness probe that always passes** (e.g. checking a static file instead of real
|
||||
dependencies) — traffic gets routed to a Pod that can't actually serve it.
|
||||
4. **No probes configured at all** — Kubernetes has no way to know the container is stuck, so a
|
||||
deadlocked process just sits there serving nothing, forever, looking "Running" the whole time.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 12-15 minutes for a live read-through with pauses for questions;
|
||||
10 minutes for a self-paced read.
|
||||
|
||||
**Demo plan:** Before moving to the guided exercise, it's worth live-editing one Deployment's
|
||||
`readinessProbe.httpGet.port` in front of the class to an obviously wrong value and running
|
||||
`kubectl get pods` / `kubectl describe pod` together, so students see the Ready column and the
|
||||
Events section change in real time before they do it themselves.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students often assume a failing liveness probe and a failing readiness probe look the same in
|
||||
`kubectl get pods` — they don't (liveness failures show restart counts climbing; readiness
|
||||
failures show `0/1` Ready with no restarts). Worth calling out explicitly.
|
||||
- Students sometimes think `tcpSocket` checks are "less safe" than `httpGet` — reframe as
|
||||
"checks a different, shallower thing," not "worse."
|
||||
|
||||
**Discussion question:** "Why might a team deliberately choose a readiness probe that fails
|
||||
under high load, even though it takes healthy capacity out of rotation?" (Answer: shedding load
|
||||
from an overloaded Pod is often better than serving every request slowly or with errors.)
|
||||
|
||||
**Optional extension:** Ask students to sketch what would happen if liveness and readiness
|
||||
probes were swapped (readiness probe restarts on failure, liveness probe just removes from
|
||||
Service) — this clarifies the distinction by making the wrong version visibly worse.
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
# Demonstration: Probes in Action
|
||||
|
||||
Watch what a probe failure actually looks like from the outside, before you're the one
|
||||
diagnosing it.
|
||||
|
||||
Picture a Deployment called `web` running `nginx`, which listens on port 80. Someone adds a
|
||||
readiness probe but mistypes the port as `8080`:
|
||||
|
||||
```yaml
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /
|
||||
port: 8080
|
||||
initialDelaySeconds: 3
|
||||
periodSeconds: 5
|
||||
```
|
||||
|
||||
**What you'd see immediately after this rolls out:**
|
||||
|
||||
```
|
||||
$ kubectl get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
web-7d8f9c6b5d-abcde 0/1 Running 0 45s
|
||||
```
|
||||
|
||||
`STATUS` says `Running` — the container started fine, nginx is up. But `READY` says `0/1` — the
|
||||
readiness probe is failing, so this Pod has never been added to its Service's endpoints. No
|
||||
traffic reaches it. `RESTARTS` stays at `0` because this is a readiness failure, not a liveness
|
||||
failure — nothing is being killed, it's just being excluded.
|
||||
|
||||
**The trail that explains why**, from `kubectl describe pod`:
|
||||
|
||||
```
|
||||
Events:
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Normal Scheduled 50s default-scheduler Successfully assigned default/web-... to node-1
|
||||
Normal Pulled 49s kubelet Container image "nginx:1.25" already present on machine
|
||||
Normal Created 49s kubelet Created container web
|
||||
Normal Started 49s kubelet Started container web
|
||||
Warning Unhealthy 15s (x6 over 40s) kubelet Readiness probe failed: Get "http://10.244.0.7:8080/": dial tcp 10.244.0.7:8080: connect: connection refused
|
||||
```
|
||||
|
||||
That last line is the whole story: kubelet tried to connect to port `8080` and nothing was
|
||||
listening there — because nginx is listening on `80`. The container was never broken. The probe
|
||||
was asking the wrong question.
|
||||
|
||||
**The general diagnostic shape** for any probe issue: `kubectl get pods` tells you *that*
|
||||
something is wrong (Ready count, restart count). `kubectl describe pod` and its Events section
|
||||
tell you *what* is wrong, usually in plain English, including the exact port/path that was
|
||||
tried and the exact error. Reach for Events before logs — logs show you what the application
|
||||
did; Events show you what Kubernetes tried to do and how it failed.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Live-run this exact sequence against a throwaway Deployment (or the classroom
|
||||
projector) rather than only showing static output — `kubectl get pods`, then
|
||||
`kubectl describe pod <name>`, scrolling straight to Events. Ask the class to predict the
|
||||
`READY` column and `RESTARTS` count *before* you show it.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; ~5 minutes as a self-paced read.
|
||||
|
||||
**Common misconception to preempt:** students often reach for `kubectl logs` first when
|
||||
something isn't Ready. For a probe misconfiguration, logs are frequently silent or unhelpful —
|
||||
the application is healthy and has nothing to say. This is a good moment to reinforce
|
||||
"Events explain what Kubernetes tried; logs explain what the app did," and that reading Events
|
||||
first is usually faster for this class of failure.
|
||||
|
||||
**Optional extension:** show the same failure signature for a `tcpSocket` probe (a bare
|
||||
"connection refused" with no HTTP verb in the Events message) so students can tell the two
|
||||
mechanisms apart from Events alone.
|
||||
+56
@@ -0,0 +1,56 @@
|
||||
# Knowledge Check: Health Probes
|
||||
|
||||
Answer each question yourself before reading the answer that follows it. This isn't graded —
|
||||
it's a checkpoint to catch a misunderstanding before you're troubleshooting a live Pod in the
|
||||
next activity.
|
||||
|
||||
---
|
||||
|
||||
**1. A Pod shows `RESTARTS: 12` and `READY: 1/1`. What kind of probe failure, if any, is most
|
||||
consistent with this?**
|
||||
|
||||
*Answer:* A liveness probe that was failing and has now recovered (or is intermittently
|
||||
failing) — the climbing restart count is the signature of liveness failures, since a failed
|
||||
liveness check causes the kubelet to kill and restart the container. `READY: 1/1` means it's
|
||||
currently passing its readiness check right now.
|
||||
|
||||
---
|
||||
|
||||
**2. A Pod shows `RESTARTS: 0` and `READY: 0/1`, and has looked this way for several minutes.
|
||||
What kind of probe failure is most consistent with this?**
|
||||
|
||||
*Answer:* A readiness probe that is failing. The container is running fine (no restarts), but
|
||||
Kubernetes has decided it isn't ready to receive traffic and has excluded it from the Service's
|
||||
endpoints. This is exactly the symptom you'll investigate in this module's lab.
|
||||
|
||||
---
|
||||
|
||||
**3. Why does Kubernetes offer a separate startup probe instead of just making
|
||||
`initialDelaySeconds` on the liveness probe longer?**
|
||||
|
||||
*Answer:* `initialDelaySeconds` is a fixed, one-size-fits-all wait before checks even begin —
|
||||
set it too short and a normally-slow startup gets killed; set it too long and every fast
|
||||
startup pays that same delay before liveness/readiness protection is even active. A startup
|
||||
probe instead keeps checking until the application actually reports ready (bounded by its own
|
||||
failure threshold), so liveness/readiness only take over once startup is confirmed done —
|
||||
correct for both a consistently slow app and one whose startup time varies.
|
||||
|
||||
---
|
||||
|
||||
**4. True or false: a readiness probe failure will cause Kubernetes to restart the container.**
|
||||
|
||||
*Answer:* False. A readiness probe failure only removes the Pod from Service endpoints — it
|
||||
does not restart anything. Only a liveness (or startup, before it succeeds once) probe failure
|
||||
causes a restart.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** In a live classroom, ask these out loud and take verbal answers before
|
||||
revealing each one, rather than having students read this silently — the goal is to surface a
|
||||
wrong mental model while it's still cheap to correct.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
|
||||
**Likely misconception this catches:** question 2 specifically catches students who default to
|
||||
"something is broken, kubectl logs will show an error" — for a pure readiness-probe
|
||||
misconfiguration, logs are usually clean, which is disorienting the first time you hit it.
|
||||
+61
@@ -0,0 +1,61 @@
|
||||
# Guided Exercise: Configure a Readiness Probe
|
||||
|
||||
This is a paper-and-thinking exercise, not a Lab task — you won't run these commands against a
|
||||
real cluster here (that comes next). The goal is to practice reading and writing a probe
|
||||
definition correctly before you're diagnosing someone else's broken one.
|
||||
|
||||
## The scenario
|
||||
|
||||
You're deploying a small HTTP API that listens on port `3000` and exposes a health endpoint at
|
||||
`/healthz`. It takes a couple of seconds to finish connecting to its database on startup — if a
|
||||
request arrives before that finishes, it correctly returns a `503` at `/healthz` until it's
|
||||
ready.
|
||||
|
||||
## Your task
|
||||
|
||||
Work out, on paper (or in a scratch file), what you'd write for this container's readiness
|
||||
probe. Think through:
|
||||
|
||||
1. Which mechanism fits — `httpGet`, `tcpSocket`, or `exec`? (Hint: you have a real health
|
||||
endpoint that returns a meaningful status code — use it rather than a shallower check.)
|
||||
2. What `port` and `path` should the probe target?
|
||||
3. Roughly what `initialDelaySeconds` makes sense, given the database connection takes a couple
|
||||
of seconds?
|
||||
4. Should this be a `readinessProbe`, a `livenessProbe`, or both? Justify your answer using the
|
||||
distinction from the lesson — what actually goes wrong if you get this specific choice wrong
|
||||
here?
|
||||
|
||||
## A worked shape to check yourself against
|
||||
|
||||
Your answer should end up structurally similar to this (exact field names matter more than the
|
||||
precise numbers you chose):
|
||||
|
||||
```yaml
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 3000
|
||||
initialDelaySeconds: 3
|
||||
periodSeconds: 5
|
||||
```
|
||||
|
||||
If you reached for `livenessProbe` instead of `readinessProbe` for this specific scenario,
|
||||
re-read the lesson's distinction before continuing — "returns 503 while warming up" is
|
||||
precisely the readiness case, not the liveness case. A liveness probe here would restart a
|
||||
perfectly healthy, still-starting container over and over, which never lets it finish starting.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Have students write their probe block on paper or a shared doc before showing
|
||||
the worked shape — resist showing the answer early, since the value here is in students
|
||||
committing to an answer and comparing, not passively reading a correct one.
|
||||
|
||||
**Approximate timing:** 10-15 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Common mistake to watch for:** choosing `livenessProbe` because "the app isn't healthy yet"
|
||||
— this is the single most common real-world probe mistake and exactly what question 4 is
|
||||
designed to surface before the next activity's real diagnostic task.
|
||||
|
||||
**Optional extension:** ask what would need to change in this same YAML if the health endpoint
|
||||
took up to 60 seconds to become available on cold start (answer: a `startupProbe` with a
|
||||
generous `failureThreshold` × `periodSeconds`, so liveness/readiness don't engage too early).
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
# Launch Lab: Stuck Not-Ready: The Probe Port Mismatch
|
||||
|
||||
Time to apply everything from this module against a real, running incident — not a diagram, an
|
||||
actual Deployment in your own disposable Lab environment.
|
||||
|
||||
## What you're walking into
|
||||
|
||||
A deployment called `api-gateway` has been rolled out. The pods start, the container runs — but
|
||||
the deployment never reaches Ready, and traffic isn't being routed to it. Somewhere on-call, an
|
||||
alert is firing.
|
||||
|
||||
You now have everything you learned in this module to work with:
|
||||
|
||||
- **The lesson** told you what "stuck at 0/2 Ready with 0 restarts" actually means (a readiness
|
||||
problem, not a crash).
|
||||
- **The demonstration** showed you exactly where to look for the explanation — `kubectl get
|
||||
pods`, then `kubectl describe pod` and its Events section — and what a port-mismatch failure
|
||||
looks like in that output.
|
||||
- **The guided exercise** had you reason through what a correct probe definition looks like,
|
||||
which is exactly what you'll need to compare the broken one against.
|
||||
|
||||
## Your task in this Lab
|
||||
|
||||
Diagnose why the pods are failing their readiness check, correct the probe configuration, and
|
||||
restore the deployment to fully Ready. Start with pod status and events, the same way you just
|
||||
practiced — read before you patch.
|
||||
|
||||
This Lab has its own in-context hints if you get stuck, revealed one at a time — use the
|
||||
concepts from this module first; reach for a hint only if you're genuinely stuck, not as the
|
||||
first move.
|
||||
|
||||
When the Lab confirms your fix, come back to this module to close it out with a short
|
||||
reflection on what actually caused this.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Classroom flow:** if teaching this live, this is the natural point to have students launch
|
||||
their own Lab environment and work independently or in pairs, while you circulate. The
|
||||
Real-Time Classroom dashboard's help-request and progress views (see the instructor Curriculum
|
||||
view) show who's stuck without needing to walk around asking.
|
||||
|
||||
**Approximate timing:** 15-20 minutes for most students; some will resolve it in under 5 once
|
||||
the module's concepts have landed.
|
||||
|
||||
**Common mistake to watch for:** students jumping straight to `kubectl logs` here, which will
|
||||
be unhelpful (nginx has nothing to say about a probe pointed at the wrong port) — this is a good
|
||||
moment to reinforce the demonstration's point about reading Events first for this failure class.
|
||||
+61
@@ -0,0 +1,61 @@
|
||||
# Root-Cause Reflection & Real Ops Takeaway
|
||||
|
||||
You just fixed a real incident pattern, not a puzzle. Before moving on, take a few minutes to
|
||||
think through what actually happened and why it matters beyond this one Lab.
|
||||
|
||||
## What actually went wrong
|
||||
|
||||
The container was never broken. `nginx` started fine and was serving requests correctly the
|
||||
entire time. The only thing wrong was that the readiness probe was configured to check a port
|
||||
nginx wasn't listening on — so Kubernetes, working exactly as designed, correctly reported the
|
||||
Pod as Not Ready based on the information it had. The system didn't fail; it was given wrong
|
||||
information and acted rationally on it.
|
||||
|
||||
This is worth sitting with: most "Kubernetes is broken" incidents are actually "Kubernetes is
|
||||
doing exactly what its configuration tells it to do, and the configuration is wrong."
|
||||
|
||||
## Reflection questions
|
||||
|
||||
Think through your own answers to these — there's no single correct wording, but you should be
|
||||
able to explain each one out loud:
|
||||
|
||||
1. If you had jumped straight to `kubectl logs` instead of `kubectl describe pod`, how long do
|
||||
you think it would have taken you to find this? What would the logs have actually shown you?
|
||||
2. This Lab's port mismatch was introduced deliberately. What real-world change could introduce
|
||||
the exact same failure by accident? (Consider: someone changes the container's listening
|
||||
port in a new image version, but forgets the probe definition was hand-written separately.)
|
||||
3. Why does a rolling update stall — not fail outright, just hang — when new Pods can't pass
|
||||
their readiness probe? What is Kubernetes protecting you from by refusing to proceed?
|
||||
|
||||
## Real Ops takeaway: prevention, not just fixes
|
||||
|
||||
In a real production system, you'd want to prevent this class of incident from recurring, not
|
||||
just fix it once:
|
||||
|
||||
- **Keep the probe port derived from the same source as the container's actual port**, where
|
||||
your tooling allows it (a shared value in a Helm chart or Kustomize overlay), instead of two
|
||||
independently hand-typed numbers that can drift apart silently.
|
||||
- **Treat a stalled rollout as a first-class alert**, not just "the deploy is taking a while" —
|
||||
a Deployment sitting at partial readiness for an unusual length of time is exactly this
|
||||
failure mode, and catching it via rollout status is faster than waiting for a user-facing
|
||||
outage report.
|
||||
- **Practice reading Events before logs** for anything in the "Running but not Ready" state —
|
||||
you just did this under low stakes; the habit is what pays off during a real incident at 2am.
|
||||
|
||||
This module is complete. Return to the course to continue with the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** use the three reflection questions above as
|
||||
discussion prompts rather than a silent worksheet — question 2 in particular tends to produce
|
||||
good real-world examples from students who've touched CI/CD pipelines before.
|
||||
|
||||
**Approximate timing:** 10 minutes for a live discussion; 5-8 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to write the one-sentence postmortem summary they'd post
|
||||
in a team channel for this incident (real on-call teams write these) — a good answer names the
|
||||
root cause (probe port mismatch) and the fix (corrected port), not just "it's fixed now."
|
||||
|
||||
**Why this activity is a reflection, not another knowledge check:** the goal here is
|
||||
consolidation and transfer to real-world judgment, not testing recall — resist turning this
|
||||
into another right/wrong quiz.
|
||||
+92
@@ -0,0 +1,92 @@
|
||||
# The RootNode Troubleshooting Framework
|
||||
|
||||
## Why a framework, not just "get good at kubectl"
|
||||
|
||||
The gold-standard Health Probes module taught you a lot about ONE specific failure mode
|
||||
(readiness probe misconfiguration) and the specific commands to diagnose it. This module steps
|
||||
back and asks: what's the repeatable PROCESS that works for almost any Kubernetes failure, not
|
||||
just the ones you've already seen before?
|
||||
|
||||
Guessing under pressure — trying random fixes and seeing what sticks — is slow, often makes
|
||||
things worse, and doesn't build a transferable skill. A framework does.
|
||||
|
||||
## Observe → Gather Evidence → Form Hypothesis → Test → Repair → Validate → Prevent Recurrence
|
||||
|
||||
**Observe** — what's the actual, specific symptom? Not "it's broken," but "the Deployment shows
|
||||
0/3 Ready" or "requests are timing out" or "the Pod restarts every 30 seconds." Precision here
|
||||
saves time later.
|
||||
|
||||
**Gather Evidence** — before touching anything, collect what the system itself is telling you:
|
||||
`kubectl get pods`, `kubectl describe pod`, `kubectl logs`, `kubectl get events`. Resist the urge
|
||||
to start changing things before you've looked.
|
||||
|
||||
**Form Hypothesis** — based on the evidence, state a SPECIFIC guess about the cause. "The
|
||||
container is crashing because the command it's told to run doesn't exist" is testable. "Something's
|
||||
wrong with networking" is not specific enough to act on yet.
|
||||
|
||||
**Test** — check whether your hypothesis is actually true, ideally without yet making the fix
|
||||
(read the exact command being run, compare against what the image expects) — confirm before you
|
||||
commit to a repair.
|
||||
|
||||
**Repair** — make the minimal change that addresses the confirmed cause. Not a bigger change
|
||||
"while you're in there" — exactly the fix the evidence supports.
|
||||
|
||||
**Validate** — confirm the fix actually worked, using the same kind of evidence you gathered
|
||||
initially (not just "I made a change, it should be fine now").
|
||||
|
||||
**Prevent Recurrence** — the step most often skipped under time pressure. Could this class of
|
||||
mistake happen again? Is there a smaller, structural change (a default, a validation, a check)
|
||||
that would catch it earlier next time?
|
||||
|
||||
## Applying it: a CrashLoopBackOff
|
||||
|
||||
- **Observe:** `crash-app`'s Pods show `STATUS: CrashLoopBackOff`, restart count climbing.
|
||||
- **Gather Evidence:** `kubectl describe deployment crash-app` (check the configured command),
|
||||
`kubectl logs -l app=crash-app` (what did the container actually print before dying, if
|
||||
anything).
|
||||
- **Form Hypothesis:** "The command override is invalid/wrong for this image, so the container
|
||||
exits immediately every time it starts."
|
||||
- **Test:** compare the command actually configured against what the image's default/expected
|
||||
entrypoint should be.
|
||||
- **Repair:** fix the command to what the container image actually expects.
|
||||
- **Validate:** `kubectl get pods` shows `Running`, restart count stops climbing.
|
||||
- **Prevent Recurrence:** a code-review habit of never overriding a container's command without
|
||||
verifying it against the image's documentation first.
|
||||
|
||||
## Applying it: a broken Service selector
|
||||
|
||||
- **Observe:** traffic isn't reaching `my-svc`'s backend at all.
|
||||
- **Gather Evidence:** `kubectl get endpoints my-svc` (empty?), `kubectl get pods
|
||||
--show-labels` (what labels do the actual Pods have?).
|
||||
- **Form Hypothesis:** "The Service's selector doesn't match the Pods' actual labels."
|
||||
- **Test:** compare `kubectl describe service my-svc`'s selector against the Pods' real labels
|
||||
directly, side by side.
|
||||
- **Repair:** correct the selector to match.
|
||||
- **Validate:** `kubectl get endpoints my-svc` now shows the Pod IPs.
|
||||
- **Prevent Recurrence:** keeping label keys/values consistent (or generated from one source)
|
||||
between a Deployment's Pod template and any Service that targets it.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10-12 minutes live walking through both worked
|
||||
examples.
|
||||
|
||||
**Demo plan:** Use the demonstration activity right after this lesson to show the CrashLoopBackOff
|
||||
example live, narrating each framework step explicitly as you go rather than just running
|
||||
commands.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students under time pressure often skip straight to "Repair" without confirming a hypothesis
|
||||
first. Reinforce that a wrong guess implemented as a "fix" can make evidence harder to read
|
||||
afterward, not easier.
|
||||
- "Prevent Recurrence" is the step most likely to be treated as optional/skippable. Frame it as
|
||||
what separates a team that keeps hitting the same incident from one that doesn't.
|
||||
|
||||
**Discussion question:** "What's the cost of skipping 'Gather Evidence' and jumping straight to
|
||||
a guessed fix, even if the guess happens to be right?" (Answer: even a correct guess doesn't
|
||||
teach you whether your reasoning was right — you got lucky, and the same class of mistake could
|
||||
still recur since you never confirmed the actual cause.)
|
||||
|
||||
**Optional extension:** ask students to apply this exact framework, from memory, to the gold
|
||||
module's `rootnode-probe-port-mismatch` incident they already solved — a good retroactive
|
||||
exercise connecting a lab they already completed to the generalized framework.
|
||||
+91
@@ -0,0 +1,91 @@
|
||||
# Demonstration: Applying the Framework to a CrashLoopBackOff
|
||||
|
||||
Watch the full Observe → Evidence → Hypothesis → Test → Repair → Validate → Prevent sequence
|
||||
against a real symptom, narrated step by step.
|
||||
|
||||
## Observe
|
||||
|
||||
```
|
||||
$ kubectl get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
crash-app-6d9f8b7c5d-abcde 0/1 CrashLoopBackOff 5 4m
|
||||
```
|
||||
|
||||
Specific symptom: `CrashLoopBackOff`, restart count climbing (5 and counting) — the container
|
||||
is starting, exiting, and being restarted repeatedly, not just failing to start once.
|
||||
|
||||
## Gather Evidence
|
||||
|
||||
```
|
||||
$ kubectl logs crash-app-6d9f8b7c5d-abcde
|
||||
nginx: [emerg] invalid command "daemon-off": expected valid command
|
||||
```
|
||||
|
||||
```
|
||||
$ kubectl describe deployment crash-app
|
||||
...
|
||||
Command:
|
||||
nginx
|
||||
-g
|
||||
daemon-off;
|
||||
...
|
||||
```
|
||||
|
||||
The logs show nginx itself rejecting the command it was given. The Deployment's configured
|
||||
command shows exactly why: `daemon-off;` (a typo — missing space, should be `daemon off;`).
|
||||
|
||||
## Form Hypothesis
|
||||
|
||||
"The container's command override has a syntax error that nginx can't parse, so nginx exits
|
||||
immediately every time it starts — which is why it's crash-looping rather than just failing to
|
||||
start once."
|
||||
|
||||
## Test
|
||||
|
||||
Compare the broken command (`daemon-off;`) against nginx's actual expected syntax
|
||||
(`daemon off;`, two separate words) — confirmed: this is exactly the kind of typo that would
|
||||
produce exactly this error.
|
||||
|
||||
## Repair
|
||||
|
||||
```
|
||||
$ kubectl patch deployment crash-app --type='json' \
|
||||
-p='[{"op":"replace","path":"/spec/template/spec/containers/0/command","value":["nginx","-g","daemon off;"]}]'
|
||||
```
|
||||
|
||||
## Validate
|
||||
|
||||
```
|
||||
$ kubectl get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
crash-app-7f8c9d6b5d-fghij 1/1 Running 0 15s
|
||||
```
|
||||
|
||||
New Pod, `Running`, `RESTARTS: 0` — the fix is confirmed, not just assumed.
|
||||
|
||||
## Prevent Recurrence
|
||||
|
||||
A command override like this is a hand-typed string with no validation until the container
|
||||
actually tries to run it. A team that hit this once might decide: don't override a well-known
|
||||
image's default command unless there's a specific reason to, and if you must, test the exact
|
||||
command string locally (`docker run` or similar) before deploying it — catching a typo in
|
||||
seconds locally beats debugging a CrashLoopBackOff in the cluster.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Demo script:** Run this live if time allows, deliberately narrating which framework step
|
||||
you're on at each command — students retain the framework far better tied to real commands than
|
||||
as an abstract list.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live; 5-7 minutes self-paced.
|
||||
|
||||
**Common misconception to preempt:** students sometimes skip straight from "Observe" to
|
||||
"Repair" once they spot the typo, without stating a hypothesis or testing it. Reinforce that
|
||||
even an "obvious" fix benefits from the explicit hypothesis step — it's what makes the eventual
|
||||
Validate step meaningful (you're confirming a SPECIFIC claim, not just "trying something").
|
||||
|
||||
**Optional extension:** ask what evidence you'd gather differently if the logs had been
|
||||
completely empty instead of showing a clear nginx error (a good bridge to cases where the
|
||||
container dies before it can log anything at all — `kubectl describe pod`'s Events section
|
||||
becomes the primary evidence source in that case, same as the gold module's probe-mismatch
|
||||
lesson).
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
# Knowledge Check: The Troubleshooting Framework
|
||||
|
||||
---
|
||||
|
||||
**1. You spot what looks like an obvious typo causing a failure and fix it immediately without
|
||||
writing down a hypothesis first. What's the actual cost of skipping that step, even if the fix
|
||||
works?**
|
||||
|
||||
*Answer:* You don't get a confirmed, testable claim about the cause — if the fix happens to
|
||||
"work" for an unrelated reason (a coincidental restart, a different underlying issue also
|
||||
resolving), you'd have no way to distinguish that from your fix actually being correct. The
|
||||
hypothesis step is what makes Validate meaningful.
|
||||
|
||||
---
|
||||
|
||||
**2. During the "Gather Evidence" step for a CrashLoopBackOff, which two commands give you the
|
||||
most direct evidence, and what does each one tell you that the other doesn't?**
|
||||
|
||||
*Answer:* `kubectl logs` (what the container itself printed, if anything, before exiting) and
|
||||
`kubectl describe pod`/`describe deployment` (the container's actual configuration, plus recent
|
||||
Events). Logs show application-level output; describe/Events show what Kubernetes itself
|
||||
observed and configured — a crash with no logs at all often means the failure happened before
|
||||
the application could log anything, which describe/Events can still reveal.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: "Prevent Recurrence" only applies to major incidents, not small
|
||||
one-off mistakes like a typo.**
|
||||
|
||||
*Answer:* False. A small mistake that recurs repeatedly across a team becomes a real, ongoing
|
||||
cost even if no single instance is severe — the framework applies the same reasoning regardless
|
||||
of incident size: could a small, structural change catch this earlier next time?
|
||||
|
||||
---
|
||||
|
||||
**4. A broken Service selector (endpoint-fix-task's scenario) and a CrashLoopBackOff
|
||||
(crashloop-fix's scenario) are very different failures. What's the SAME about how you'd
|
||||
approach both?**
|
||||
|
||||
*Answer:* The same seven-step framework — Observe the specific symptom, Gather Evidence before
|
||||
guessing, form a specific testable Hypothesis, Test it, make the minimal Repair the evidence
|
||||
supports, Validate with the same kind of evidence you started with, and consider how to Prevent
|
||||
Recurrence. The framework doesn't change; only the specific commands and evidence do.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 4 is the whole point of this module — make sure it's discussed even
|
||||
if answered correctly, since it's the transferability claim the entire framework rests on.
|
||||
|
||||
**Approximate timing:** 5-7 minutes.
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
# Guided Exercise: Applying the Framework to a Broken Service, on Paper First
|
||||
|
||||
Before the `endpoint-fix-task` Lab, walk through the framework yourself against a described
|
||||
symptom — no cluster needed yet.
|
||||
|
||||
## The scenario
|
||||
|
||||
A teammate says: "`my-svc` isn't routing any traffic to our Pods, and I don't know why — the
|
||||
Pods look healthy in `kubectl get pods`."
|
||||
|
||||
## Your task
|
||||
|
||||
Write out, step by step, what you would actually DO for each framework stage, before looking at
|
||||
the worked answer:
|
||||
|
||||
1. **Observe** — what's the precise, specific symptom here (not "it's broken")?
|
||||
2. **Gather Evidence** — name the exact `kubectl` commands you'd run, in order, and what each
|
||||
one is checking for.
|
||||
3. **Form Hypothesis** — based on what ROOT 07's Services lesson taught you, what's the most
|
||||
likely cause of "Pods healthy, but Service routes nowhere"?
|
||||
4. **Test** — how would you CONFIRM that hypothesis before changing anything?
|
||||
5. **Repair** — what's the minimal fix, assuming your hypothesis is correct?
|
||||
6. **Validate** — what would you check afterward to PROVE the fix worked, not just assume it?
|
||||
|
||||
## A worked answer to check yourself against
|
||||
|
||||
1. Traffic sent to `my-svc` never reaches any Pod, despite the Pods themselves reporting
|
||||
`Running`/`Ready` — this rules out "the Pods are unhealthy" as the cause.
|
||||
2. `kubectl get endpoints my-svc` (does it show any Pod IPs at all?), then `kubectl describe
|
||||
service my-svc` (what selector is it actually using?), then `kubectl get pods --show-labels`
|
||||
(what labels do the real Pods actually carry?).
|
||||
3. A Service-to-Pod label selector mismatch — exactly the failure mode ROOT 07's demonstration
|
||||
walked through, where a Service's selector no longer matches the Pods' actual labels.
|
||||
4. Directly compare the Service's selector (from `describe service`) against the Pods' labels
|
||||
(from `get pods --show-labels`) side by side — confirmed if they genuinely don't match.
|
||||
5. Correct the Service's selector (or the Pods' labels, whichever is actually wrong) so they
|
||||
match again.
|
||||
6. `kubectl get endpoints my-svc` now shows the Pod IPs populated — not just "the YAML looks
|
||||
right now," but the actual Endpoints object reflecting real, matched Pods.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** This exercise deliberately reuses ROOT 07's own content — a good moment to
|
||||
point out explicitly that the framework's value is applying PRIOR knowledge systematically, not
|
||||
learning something entirely new each time.
|
||||
|
||||
**Approximate timing:** 10-12 minutes live with discussion; 10 minutes self-paced.
|
||||
|
||||
**Optional extension:** ask what would change about steps 2-4 if the symptom were instead
|
||||
"Pods healthy, but ONE specific caller can't reach the Service while others can" — a good bridge
|
||||
to considering NetworkPolicy (ROOT 09) as an alternative hypothesis once the simpler
|
||||
selector-mismatch explanation is ruled out.
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
# Reflection: A Framework You've Actually Used Is One You'll Trust Under Pressure
|
||||
|
||||
You just solved two real, different failures — a CrashLoopBackOff and a broken Service
|
||||
selector — using the same seven-step framework both times. Take a few minutes to consolidate
|
||||
this.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Which framework step did you personally find easiest to skip or rush through across both
|
||||
Labs? Why do you think that step specifically tempts shortcuts?
|
||||
2. Compare how you approached `crashloop-fix` versus `endpoint-fix-task`. What was IDENTICAL
|
||||
about your process, and what was different only because the evidence looked different?
|
||||
3. Now that you've practiced this framework on two failures you'd never seen exact instructions
|
||||
for, how confident do you feel applying it to a THIRD, completely unfamiliar Kubernetes
|
||||
failure? What would you still want to double-check?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**The framework's value shows up most on failures you've never seen before.** Anyone can
|
||||
memorize "if X symptom, do Y fix" for a handful of common problems — that knowledge doesn't
|
||||
transfer. Observe → Evidence → Hypothesis → Test → Repair → Validate → Prevent Recurrence
|
||||
transfers to failures nobody has specifically taught you, because it's a PROCESS, not a lookup
|
||||
table.
|
||||
|
||||
**"Prevent Recurrence" is what separates incident response from firefighting.** Both Labs in
|
||||
this module had a real, nameable prevention practice (verify command strings before deploying;
|
||||
keep label keys consistent between a Deployment and its Service). Teams that consistently ask
|
||||
this question after every fix accumulate fewer repeat incidents over time — not because they got
|
||||
lucky, but because they closed the actual gap each time.
|
||||
|
||||
This module is complete. Continue to the next module.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 1 tends to produce honest, useful
|
||||
answers — "Prevent Recurrence" and "Form Hypothesis" (skipping straight to a guessed fix) are
|
||||
the two most commonly admitted shortcuts, both worth normalizing as a real temptation to resist
|
||||
rather than treating as a personal failing.
|
||||
|
||||
**Approximate timing:** 8-10 minutes live discussion; 5-7 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to apply the framework, from memory and without re-reading
|
||||
this module, to a THIRD failure type they haven't practiced yet (e.g., a Pod stuck `Pending` —
|
||||
not built as a hands-on Lab in this module, just a thought exercise) to test genuine
|
||||
transferability.
|
||||
+88
@@ -0,0 +1,88 @@
|
||||
# DevOps as an Operating Model, Not a Job Title
|
||||
|
||||
## You've already been doing this — just not naming it yet
|
||||
|
||||
Every course before this one taught you a Kubernetes skill: declaring desired state, hardening
|
||||
a Pod, diagnosing a failure with a repeatable framework, running an incident with real
|
||||
discipline. This module doesn't teach a new hands-on skill — it names the OPERATING MODEL those
|
||||
skills are all part of, so you can recognize it and apply it beyond Kubernetes specifically.
|
||||
|
||||
## DevOps: a set of practices, not a team you join
|
||||
|
||||
"DevOps" is often misused as a job title ("I'm a DevOps engineer") or a team name ("throw it
|
||||
over to DevOps"). Properly understood, DevOps is a set of PRACTICES for how development and
|
||||
operations work together: shorter feedback loops, automation over manual process, and shared
|
||||
ownership of what happens after code ships — not a separate team that operations gets outsourced
|
||||
to.
|
||||
|
||||
Concretely, you already practiced core DevOps ideas without the label:
|
||||
|
||||
- **Declarative thinking** (ROOT 05) — describing desired state and letting a system enforce it,
|
||||
instead of manually executing steps, is the same principle behind Infrastructure as Code.
|
||||
- **Fast, safe rollback** (ROOT 06) — treating recovery as a rehearsed, ordinary action rather
|
||||
than a crisis is a direct DevOps practice (fast feedback, low-drama recovery).
|
||||
- **A repeatable troubleshooting framework** (ROOT 10) and **structured incident response**
|
||||
(ROOT 12) — both are operational discipline, the "Ops" half of DevOps, practiced for real.
|
||||
|
||||
## Platform Engineering: what DevOps looks like at scale
|
||||
|
||||
**Platform engineering** is the next layer: instead of every application team reinventing how
|
||||
to deploy safely, monitor, and secure their own workloads, a platform team builds SELF-SERVICE
|
||||
tooling and golden paths that let other engineers move fast without needing deep infrastructure
|
||||
expertise themselves. The Kubernetes cluster, the CI/CD pipeline, the observability stack — all
|
||||
of it becomes a PRODUCT a platform team builds for INTERNAL customers (other engineers), not
|
||||
just infrastructure that happens to exist.
|
||||
|
||||
A useful lens: platform engineering exists to **reduce cognitive load**. A developer who can
|
||||
deploy a well-configured, secure, observable service through a simple, guided path doesn't need
|
||||
to personally understand every Kubernetes concept from ROOT 05-10 to ship safely — the platform
|
||||
encodes that expertise once, for everyone.
|
||||
|
||||
## Where Kubernetes fits in the larger picture
|
||||
|
||||
Kubernetes is INFRASTRUCTURE, not the whole platform. A real platform engineering practice
|
||||
typically layers several things around it:
|
||||
|
||||
- **Declarative delivery** — describing what should be deployed and letting automation apply it
|
||||
(the GitOps model — future ROOT 11 content, requires a Git+deployment-automation environment
|
||||
this course doesn't yet provide).
|
||||
- **Continuous integration/delivery** — automatically testing and shipping code changes (CI/CD —
|
||||
future content, requires a CI runner environment).
|
||||
- **Infrastructure as Code** — declaring cloud/infrastructure resources the same way you declare
|
||||
Kubernetes objects (future content, requires a provisioning environment like Terraform).
|
||||
- **Observability** — knowing what's actually happening in production beyond "is the Pod
|
||||
Running" (future content, requires a metrics/logging/tracing stack).
|
||||
|
||||
None of these require abandoning what you learned in ROOT 05-10 — they build directly on the
|
||||
same mental models (desired vs. actual state, evidence-based diagnosis, least privilege) applied
|
||||
to a wider set of systems.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Lesson goal:** Give students a name and a framework for practices they've already exercised,
|
||||
and an honest map of what ROOT 11 will eventually cover once appropriate lab environments exist —
|
||||
without pretending those labs exist today.
|
||||
|
||||
**Approximate timing:** 20 minutes read; 15 minutes live with discussion.
|
||||
|
||||
**Preparation:** None beyond having read `docs/ROOT11_DEVOPS_PLATFORM_SCOPE.md` to speak
|
||||
accurately about what's planned versus what exists.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- "DevOps" as a job title/team is a very common prior belief — correct this directly and early,
|
||||
it's the central point of the lesson.
|
||||
- Platform engineering is sometimes conflated with "just Kubernetes." Emphasize the self-service/
|
||||
golden-path angle — the PRODUCT framing, not just the technology.
|
||||
|
||||
**Questions to ask students:** "Which ROOT_ course so far most directly practiced a DevOps
|
||||
principle, even though it was never labeled that way?" (Most will cite ROOT 06's rollback
|
||||
discipline or ROOT 12's incident framework — both are legitimate, good answers.)
|
||||
|
||||
**When to let a student struggle vs. provide a hint:** Not applicable — this is an orientation
|
||||
lesson with no hands-on task.
|
||||
|
||||
**Optional extension:** ask students to name one manual process at a company they've worked for
|
||||
(or heard about) that platform engineering-style self-service tooling could have eliminated.
|
||||
|
||||
**Expected takeaway:** DevOps and platform engineering are the organizing ideas behind almost
|
||||
everything ROOT_ has already taught — not a new, separate subject to learn from scratch.
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
# Knowledge Check: DevOps & Platform Thinking
|
||||
|
||||
---
|
||||
|
||||
**1. A colleague says "DevOps is the team that handles our servers." What's wrong with that
|
||||
statement?**
|
||||
|
||||
*Answer:* DevOps describes a set of PRACTICES (shared ownership, fast feedback loops,
|
||||
automation) for how development and operations work together — not a separate team that
|
||||
operations gets handed off to. Treating it as a team name misses the point that these practices
|
||||
are meant to be shared across engineering, not isolated in one group.
|
||||
|
||||
---
|
||||
|
||||
**2. Which ROOT_ course's central skill is most directly an example of "declarative thinking"
|
||||
applied outside Kubernetes objects specifically?**
|
||||
|
||||
*Answer:* ROOT 05's desired-state model is the direct root of declarative thinking generally —
|
||||
Infrastructure as Code applies the exact same principle (describe what should exist, let
|
||||
automation enforce it) to cloud infrastructure rather than Kubernetes objects specifically.
|
||||
|
||||
---
|
||||
|
||||
**3. True or false: platform engineering's main goal is building the most technically
|
||||
sophisticated internal tooling possible.**
|
||||
|
||||
*Answer:* False. Its main goal is reducing COGNITIVE LOAD for other engineers via self-service,
|
||||
golden-path tooling — sophistication that doesn't make another engineer's job easier isn't the
|
||||
point, even if it's technically impressive.
|
||||
|
||||
---
|
||||
|
||||
**4. Why can't this course build a hands-on GitOps or CI/CD Lab the same way ROOT 05-10 built
|
||||
Kubernetes Labs?**
|
||||
|
||||
*Answer:* Those require different infrastructure the current Lab platform doesn't provide (a
|
||||
Git repository + deployment-automation tool like ArgoCD for GitOps; a CI runner for CI/CD) — see
|
||||
`docs/ROOT11_DEVOPS_PLATFORM_SCOPE.md`. Building a fake version of either would teach an
|
||||
inaccurate mental model, which is worse than honestly deferring the hands-on practice.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Facilitation:** Question 4 is worth stating plainly in a live classroom — students should
|
||||
understand this is a genuine platform limitation being handled honestly, not an oversight.
|
||||
|
||||
**Approximate timing:** 5 minutes.
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
# Reflection: Where You Already Practiced This
|
||||
|
||||
You've now named the operating model behind most of what ROOT_ taught. Take a few minutes to
|
||||
trace it back through your own experience in this program.
|
||||
|
||||
## Reflection questions
|
||||
|
||||
1. Pick one specific moment from ROOT 05-10 or ROOT 12 where you practiced a DevOps/platform
|
||||
engineering principle without it being labeled that way. Name the principle explicitly now.
|
||||
2. If you were designing a platform team's "golden path" for deploying a new service safely,
|
||||
which THREE things from ROOT_ (health probes, RBAC, resource limits, NetworkPolicy, etc.)
|
||||
would you bake in as defaults rather than leave to each team to remember?
|
||||
3. What's one thing about GitOps, CI/CD, Infrastructure as Code, or observability you're
|
||||
genuinely curious about, now that you have the Kubernetes foundation to eventually build
|
||||
hands-on skill in it?
|
||||
|
||||
## Real Ops takeaway
|
||||
|
||||
**The skills you already have transfer further than "just Kubernetes."** Desired-state
|
||||
thinking, least privilege, evidence-based troubleshooting, and structured incident response
|
||||
aren't Kubernetes-specific facts — they're how experienced platform engineers reason about ANY
|
||||
complex system. Recognizing that transfer is what turns "I learned Kubernetes" into "I think
|
||||
like a platform engineer."
|
||||
|
||||
**Honest scope beats fake completeness.** This course didn't pretend to teach you GitOps or
|
||||
CI/CD hands-on today — it told you exactly what's missing and why, and pointed at what it would
|
||||
take to build it for real. That's the same standard the rest of ROOT_ held itself to
|
||||
throughout — a module that doesn't exist yet is more useful, honestly labeled, than one that
|
||||
exists in name only.
|
||||
|
||||
This module is complete.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Discussion questions for a live classroom:** question 2 works well as a small-group exercise —
|
||||
have groups compare their "golden path defaults" lists and discuss disagreements.
|
||||
|
||||
**Approximate timing:** 10 minutes live discussion; 8 minutes self-paced writing.
|
||||
|
||||
**Optional extension:** ask students to research (outside class) one real company's public
|
||||
engineering blog post about their platform engineering practice, and bring one concrete example
|
||||
to share next session.
|
||||
|
||||
**Expected takeaway:** ROOT_ built real, transferable engineering judgment — this module makes
|
||||
that transfer explicit rather than leaving students to discover it on their own.
|
||||
@@ -0,0 +1,88 @@
|
||||
# The RootNode Incident-Response Framework
|
||||
|
||||
## This module feels different on purpose
|
||||
|
||||
Everything before ROOT 12 asked you to complete an exercise. This module asks something
|
||||
different: **you are responsible for this environment.** The incidents in this module are
|
||||
modeled on real operational failures — not textbook exercises designed to teach one clean
|
||||
concept, but the kind of messy, "why is this actually broken" situations a real on-call engineer
|
||||
gets paged for.
|
||||
|
||||
## Detect → Triage → Diagnose → Mitigate → Recover → Verify → Learn
|
||||
|
||||
This is a lightweight incident-response framework — not enterprise incident management
|
||||
tooling, just a way of thinking that scales down to "one engineer, one broken thing" as well as
|
||||
it scales up to a real incident-response team.
|
||||
|
||||
**Detect** — something alerted you (a monitoring page, a user report, or in this Lab's case,
|
||||
the scenario telling you something is wrong). You know an incident exists before you know what
|
||||
it is.
|
||||
|
||||
**Triage** — how bad is this, right now, before you dive into root cause? Is anything actively
|
||||
getting worse? Is this affecting one thing or cascading into others? Triage decides how urgently
|
||||
you act, before you fully understand why.
|
||||
|
||||
**Diagnose** — this is where ROOT 10's troubleshooting framework (Observe, Gather Evidence, Form
|
||||
Hypothesis, Test) does its work — the specific mechanics of finding the actual cause.
|
||||
|
||||
**Mitigate** — the fastest safe action that stops things from getting worse, which is not always
|
||||
the same as the final fix. Sometimes mitigation and repair are the same step; sometimes
|
||||
mitigation buys time (e.g. temporarily reducing load) while a proper fix is prepared.
|
||||
|
||||
**Recover** — restore the system to its actual working state — not just "stopped getting worse"
|
||||
but genuinely operational again.
|
||||
|
||||
**Verify** — confirm recovery with real evidence, the same discipline as ROOT 10's Validate
|
||||
step. "I think it's fixed" is not the same as checking.
|
||||
|
||||
**Learn** — what would prevent this exact class of incident from happening again? This is where
|
||||
`rootnode-enforce-limits` picks up directly after `rootnode-runaway-monitor` — not a separate
|
||||
topic, but the deliberate "Learn" step of the SAME incident, made into its own scenario.
|
||||
|
||||
## What this module will NOT tell you
|
||||
|
||||
The specific fixes for each incident are not in this lesson, and instructor notes on the
|
||||
individual incident activities go deeper than what's shown here — this module gives you the
|
||||
FRAMEWORK for approaching an unfamiliar incident, not a cheat sheet for these specific four.
|
||||
That's deliberate: a framework you had to apply yourself is one you'll actually remember under
|
||||
real pressure.
|
||||
|
||||
## Instructor Notes
|
||||
|
||||
**Lesson goal:** Set the tone shift for this entire module — from "complete an exercise" to
|
||||
"you own this." This lesson is short by design; the real teaching happens through the incidents
|
||||
themselves.
|
||||
|
||||
**Approximate timing:** 15 minutes read; 10 minutes live framing discussion.
|
||||
|
||||
**Preparation:** Read each incident's own instructor notes (in the Lab activities below) before
|
||||
teaching this module live — several contain deeper solution guidance not repeated here,
|
||||
specifically so students still have to work for it.
|
||||
|
||||
**Demonstration plan:** Not applicable to this framing lesson — the four incidents themselves
|
||||
serve as the demonstrations.
|
||||
|
||||
**Likely misconceptions to watch for:**
|
||||
- Students sometimes treat "Mitigate" and "Recover" as the same step. They can be, but aren't
|
||||
always — worth naming a real-world example (rolling back a bad deploy mitigates immediately;
|
||||
fully recovering might still require redeploying the CORRECT version afterward).
|
||||
- "Learn" is the step most likely to get skipped once a Lab is marked complete. Explicitly
|
||||
connect it to why `rootnode-enforce-limits` exists as its own scenario.
|
||||
|
||||
**Questions to ask students:** "Which step of this framework do you think real teams are worst
|
||||
at, under real pressure, and why?" (There's no single right answer — the goal is getting
|
||||
students to reason about incident response as a discipline, not just a checklist.)
|
||||
|
||||
**When to let a student struggle vs. provide a hint:** These incidents are intentionally less
|
||||
guided than earlier modules. Let a student sit with "I don't know where to start" for a few
|
||||
minutes before nudging — the discomfort of a genuinely unfamiliar failure is part of what this
|
||||
module is teaching. Redirect toward the Diagnose step's ROOT 10 framework if truly stuck, rather
|
||||
than pointing at the specific fix.
|
||||
|
||||
**Optional extension:** ask students to write a two-sentence incident summary for
|
||||
`rootnode-runaway-monitor` as if posting it to a real team channel — Detect/Diagnose/Recover in
|
||||
plain language, no jargon.
|
||||
|
||||
**Expected takeaway:** A framework applied under pressure beats improvisation, even for
|
||||
engineers who are technically skilled enough to eventually stumble into the right answer anyway
|
||||
— the framework is what makes the RIGHT answer arrive faster and more reliably.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user