ced4568andClaude Sonnet 5
28c925069b
content/root/root-10-kubernetes-troubleshooting-observability/m01-health-probes/: lesson (probe concepts) -> demonstration (reading kubectl describe pod Events for a probe failure) -> knowledge check -> guided exercise (write a readiness probe on paper) -> Launch Lab intro (rootnode-probe-port-mismatch) -> reflection (root cause + Real Ops prevention takeaway). 5 of 6 activities carry instructor notes (demo plan, misconceptions, timing, discussion questions, extensions). This is the reference implementation docs/COURSE_AUTHORING_STANDARD.md documents for all future RootNode lesson authoring. Co-Authored-By: Claude Sonnet 5 <[email protected]>
3.2 KiB
Demonstration: Probes in Action
Watch what a probe failure actually looks like from the outside, before you're the one diagnosing it.
Picture a Deployment called web running nginx, which listens on port 80. Someone adds a
readiness probe but mistypes the port as 8080:
readinessProbe:
httpGet:
path: /
port: 8080
initialDelaySeconds: 3
periodSeconds: 5
What you'd see immediately after this rolls out:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
web-7d8f9c6b5d-abcde 0/1 Running 0 45s
STATUS says Running — the container started fine, nginx is up. But READY says 0/1 — the
readiness probe is failing, so this Pod has never been added to its Service's endpoints. No
traffic reaches it. RESTARTS stays at 0 because this is a readiness failure, not a liveness
failure — nothing is being killed, it's just being excluded.
The trail that explains why, from kubectl describe pod:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 50s default-scheduler Successfully assigned default/web-... to node-1
Normal Pulled 49s kubelet Container image "nginx:1.25" already present on machine
Normal Created 49s kubelet Created container web
Normal Started 49s kubelet Started container web
Warning Unhealthy 15s (x6 over 40s) kubelet Readiness probe failed: Get "http://10.244.0.7:8080/": dial tcp 10.244.0.7:8080: connect: connection refused
That last line is the whole story: kubelet tried to connect to port 8080 and nothing was
listening there — because nginx is listening on 80. The container was never broken. The probe
was asking the wrong question.
The general diagnostic shape for any probe issue: kubectl get pods tells you that
something is wrong (Ready count, restart count). kubectl describe pod and its Events section
tell you what is wrong, usually in plain English, including the exact port/path that was
tried and the exact error. Reach for Events before logs — logs show you what the application
did; Events show you what Kubernetes tried to do and how it failed.
Instructor Notes
Demo script: Live-run this exact sequence against a throwaway Deployment (or the classroom
projector) rather than only showing static output — kubectl get pods, then
kubectl describe pod <name>, scrolling straight to Events. Ask the class to predict the
READY column and RESTARTS count before you show it.
Approximate timing: 8-10 minutes live; ~5 minutes as a self-paced read.
Common misconception to preempt: students often reach for kubectl logs first when
something isn't Ready. For a probe misconfiguration, logs are frequently silent or unhelpful —
the application is healthy and has nothing to say. This is a good moment to reinforce
"Events explain what Kubernetes tried; logs explain what the app did," and that reading Events
first is usually faster for this class of failure.
Optional extension: show the same failure signature for a tcpSocket probe (a bare
"connection refused" with no HTTP verb in the Events message) so students can tell the two
mechanisms apart from Events alone.