Live Google Workspace delivery validation surfaced a real, self-resolving Google-side condition: freshly-authorized domain-wide delegation can take up to about a minute to propagate across Google's auth infrastructure, producing intermittent unauthorized_client on the OAuth token exchange even with correct configuration. gws-provider.js now classifies unauthorized_client as a temporary (retryable) error instead of permanent, so the existing bounded-backoff worker absorbs it automatically. Also documents the alias-vs-delegated-user env passthrough in docker-compose.dev.yml, and records both real findings (this one and the GOOGLE_DELEGATED_USER alias distinction) plus the full live-delivery validation outcome in the runbook and Phase 5C doc. Validated live: isolated gws-email-test.js send (DKIM/SPF passing, delivered) and a full real lifecycle test (create student, enrollment, real activation email, activate, login, see program, real password-reset email, reset, login with new password) via [email protected].
249 lines
16 KiB
Markdown
249 lines
16 KiB
Markdown
# Phase 5C — Enrollment Operations & Communications
|
|
|
|
**Status: PHASE 5C COMPLETE. ENROLLMENT COMMUNICATIONS COMPLETE. STUDENT ACTIVATION PIPELINE READY.**
|
|
**LIVE GOOGLE WORKSPACE DELIVERY VALIDATED** — real activation and password-reset emails
|
|
delivered end to end via `[email protected]` (a Workspace alias), DKIM/SPF passing,
|
|
inbox delivery confirmed. See §14.
|
|
|
|
Phase 5B built the enrollment/entitlement model and the website integration boundary. What was
|
|
still missing for a real student: nobody ever told them. This phase closes that gap — real
|
|
email delivery for activation, enrollment confirmation, and password reset — plus finally
|
|
completing the production password-reset delivery Phase 5A explicitly left unfinished.
|
|
|
|
## 1. Architecture: provider-neutral by design
|
|
|
|
```
|
|
Enrollment/Auth code (integration-routes.js, admin-routes.js, student-auth-routes.js)
|
|
↓ (mint token if needed, then) notifications.enqueue()
|
|
notification_events row, committed in the SAME request — no network I/O yet
|
|
↓
|
|
notification-worker.js (a setInterval poller, same pattern as sessions.js's sweeper)
|
|
↓
|
|
email/provider.js — EMAIL_PROVIDER env var picks the implementation
|
|
↓
|
|
email/dev-provider.js (default) OR email/gws-provider.js
|
|
(never a real network call) (Gmail API, service account)
|
|
```
|
|
|
|
No enrollment/auth code anywhere imports a concrete provider. Swapping Google Workspace for a
|
|
different provider later means writing one new file under `platform/api/src/email/` and
|
|
changing `EMAIL_PROVIDER` — nothing else changes.
|
|
|
|
## 2. Google Workspace delivery mechanism: Gmail API + service account domain-wide delegation
|
|
|
|
**Selected mechanism:** the Gmail API's `users.messages.send`, authenticated as a Google Cloud
|
|
service account using Workspace domain-wide delegation to impersonate one dedicated sending
|
|
mailbox.
|
|
|
|
**Why, not SMTP or a human login:**
|
|
- **No human password, ever.** Basic SMTP auth (or an "app password") means storing credentials
|
|
that log in as an actual person's Workspace account — anyone with that value could read that
|
|
mailbox's mail, change its settings, or use it to send as that person. A service account has
|
|
no login of its own and no inbox; domain-wide delegation grants it exactly one narrow OAuth
|
|
scope (`gmail.send`) against one specific mailbox, nothing more.
|
|
- **Least privilege.** `gmail.send` cannot read mail, cannot manage the mailbox, cannot act as
|
|
any address other than the one configured. Compare to SMTP relay, which typically wants either
|
|
a static allow-listed sending IP (fragile for infrastructure that isn't pinned yet) or a
|
|
Workspace login.
|
|
- **This is Google's own documented pattern** for "an application needs to send mail as a
|
|
specific Workspace address" — see `docs/RUNBOOK_GOOGLE_WORKSPACE_EMAIL.md` for the exact
|
|
setup steps and links.
|
|
|
|
**No `googleapis`/`google-auth-library` dependency.** The JWT-bearer OAuth flow (RS256-sign a
|
|
short JWT, exchange it for an access token, POST one REST call) is ~120 lines using only Node's
|
|
built-in `crypto` and `fetch` — consistent with this codebase's established "hand-roll before
|
|
adding a dependency" convention (CSRF tokens, Prometheus exposition, and CSV parsing all do the
|
|
same). See `platform/api/src/email/gws-provider.js`.
|
|
|
|
## 3. RootNode email identity
|
|
|
|
`EMAIL_FROM_NAME` / `EMAIL_FROM_ADDRESS` / `EMAIL_REPLY_TO` — all environment-configured, no
|
|
default mailbox address baked into the code (`EMAIL_FROM_NAME` defaults to the display string
|
|
"RootNode Academy"; `EMAIL_FROM_ADDRESS` has no default at all — sending fails closed with a
|
|
clean `gws_not_configured` error until it's set, rather than guessing a real mailbox that
|
|
doesn't exist in this repository's configuration). `EMAIL_FROM_ADDRESS` doubles as the Gmail API
|
|
adapter's delegated-user (`sub`) claim — it must be a real mailbox in the Workspace domain,
|
|
authorized for delegation (see the runbook).
|
|
|
|
DNS/domain requirements before any of this can send real mail in production — SPF, DKIM, DMARC,
|
|
and Workspace's own sender-authorization step — are **entirely external to this repository**
|
|
and are documented, not performed, in `docs/RUNBOOK_GOOGLE_WORKSPACE_EMAIL.md` (marked
|
|
`REQUIRES GOOGLE WORKSPACE ADMIN ACTION`).
|
|
|
|
## 4. Notification model
|
|
|
|
`notification_events` (migration `021_notification_events.sql`): `student_id` (required — every
|
|
notification this phase sends is about a specific student), `enrollment_id` (nullable —
|
|
password-reset/changed notifications have none), `notification_type`
|
|
(`activation`/`enrollment_confirmation`/`password_reset`/`password_changed`), `recipient_email`
|
|
(a snapshot, not a live join — see §10), `status`
|
|
(`queued`→`sending`→`sent`/`failed`, or `cancelled`, unused this phase but reserved),
|
|
`attempt_count`/`next_attempt_at` for bounded exponential backoff, `last_error_code` (a
|
|
classified string, never a raw stack trace), and `metadata` JSONB.
|
|
|
|
**Privacy note on `recipient_email`:** duplicates PII already stored elsewhere (`students.email`
|
|
or an enrollment's `contact_email`) purely so the worker and any later delivery audit don't need
|
|
to reconstruct "what address would this have gone to" after the student record potentially
|
|
changes. Same sensitivity class as data already in this database — never a new exposure, and
|
|
never returned unmasked through any admin API (`notifications.js`'s `maskEmail`/`toAdminView`
|
|
are the only shapes any route returns).
|
|
|
|
## 5. The transient-plaintext-token design (read this before touching token handling)
|
|
|
|
Phase 5A's invariant — only a HASH of an activation/reset token is ever persisted long-term
|
|
(`students.credential_token_hash`) — is unchanged and non-negotiable. But an activation/reset
|
|
*email* needs the real token to build a working link, and the whole point of this phase is that
|
|
the actual send happens later, asynchronously, in the worker — not in the request that
|
|
originally minted the token.
|
|
|
|
The resolution: the triggering request mints the token exactly as Phase 5A already does
|
|
(synchronously, in-request), and hands the plaintext to `notifications.enqueue()`, which stores
|
|
it in that one row's `metadata.token` **only until the row reaches a terminal state**
|
|
(`markSent`/`markFailedOrRetry` both scrub it — see their comments in `notifications.js`). This
|
|
is a deliberate, narrow, documented exception for the row that exists specifically to deliver
|
|
the plaintext once — not a second credential store, and never returned by any admin API
|
|
regardless of whether it's still present (`toAdminView` never includes `metadata` at all).
|
|
|
|
Activation notifications from a **new enrollment** are the one case where no token is minted
|
|
ahead of time and handed in — `notifyForEnrollment()` mints it itself, at enqueue time, because
|
|
there's no other synchronous caller already holding one. Everywhere else (admin create/convert/
|
|
reset-password, self-service forgot-password, admin resend), the existing Phase 5A minting call
|
|
site is reused unchanged and its plaintext is simply also passed to `enqueue()`.
|
|
|
|
## 6. Activation flow
|
|
|
|
Reuses Phase 5A's activation token mechanism entirely — no competing invitation-token system was
|
|
built. `issueCredentialToken` already unconditionally invalidates any previous outstanding token
|
|
on mint (Phase 5A), which means **"resend" and "regenerate" are mechanically the same
|
|
operation**: there is never a moment with two simultaneously valid activation links, so a second
|
|
button that did something different would have nothing different to do. `POST /api/admin/
|
|
students/:id/resend-activation` is the one action that covers both.
|
|
|
|
```
|
|
Enrollment event/admin enrollment → student matched/created
|
|
→ credentials already active? → enrollment_confirmation (§7)
|
|
→ else → issueCredentialToken('activation') → notifications.enqueue()
|
|
→ worker delivers via the configured provider
|
|
→ student opens the link → existing Set Password flow (Phase 5A, unchanged)
|
|
→ token consumed atomically with setting the password
|
|
→ account active, student can log in
|
|
```
|
|
|
|
## 7. Enrollment confirmation flow
|
|
|
|
Fires whenever a **new enrollment** is created (integration event or admin manual enroll) for a
|
|
student who **already has credentials** (`students.password_hash IS NOT NULL`). No token, no
|
|
link beyond "go sign in" — `notifyForEnrollment()` (shared by both creation paths) is the single
|
|
decision point, so a website-triggered and an admin-triggered enrollment behave identically.
|
|
|
|
## 8. Password reset flow — now actually delivered
|
|
|
|
Phase 5A's `POST /api/student/request-password-reset` already minted a token and, only outside
|
|
production, echoed it back for local/test use. Phase 5C adds exactly one thing: the same
|
|
minted token is now also handed to `notifications.enqueue()`, so a real email goes out. Every
|
|
other behavior is unchanged: the response is the same generic anti-enumeration message
|
|
regardless of whether the account exists, is inactive, or was never activated (this phase
|
|
additionally excludes never-activated accounts, a real fix — see §13); production never returns
|
|
the token, dev/test still can. `POST /api/student/reset-password` (consuming the token) now also
|
|
enqueues a `password_changed` security-notice email once the new password is set.
|
|
|
|
## 9. Password-changed / security notice
|
|
|
|
Fires after both `POST /api/student/change-password` (self-service) and `POST /api/student/
|
|
reset-password` (token consumption) succeed — no token, no link, purely "this happened, if it
|
|
wasn't you here's what to do."
|
|
|
|
## 10. Minor/no-email communication design
|
|
|
|
Phase 5A/5B's minor-safe model (a student may have `login_identifier` with no `email` at all) is
|
|
preserved exactly. `enrollments.metadata.contact_email` (an existing JSONB column, not a new
|
|
migration) holds an OPTIONAL "who to notify about THIS enrollment" address — a guardian's email
|
|
collected at checkout, for example. **It is never written to `students.email`** — that field
|
|
stays reserved for the student's own identity/login credential; a guardian is not a student.
|
|
`enrollment.js`'s `enrollmentContactEmail()` is the one accessor with that one meaning; recipient
|
|
resolution (`notifications.js`'s `resolveRecipient`) tries the student's own email first, falls
|
|
back to the enrollment's contact email only for enrollment-scoped notification types, and
|
|
correctly has no fallback at all for password-reset/changed (which require the student to
|
|
already be able to log in themselves). This is deliberately transitional — not a guardian
|
|
account, not a linked identity, no relationship table — and does not block a future real
|
|
Guardian Portal from being built on top of it later.
|
|
|
|
## 11. Website → enrollment → communication
|
|
|
|
`POST /api/integrations/enrollments` (Phase 5B, unchanged) now also decides and enqueues the
|
|
right notification, but **only when its own enrollment-creation call reports `created: true`**
|
|
(a genuinely new enrollment, not an idempotent replay). This is the entire duplicate-webhook
|
|
safety net for notifications: since Phase 5B's DB-level idempotency already guarantees at most
|
|
one caller ever sees `created: true` for a given event/race, at most one notification is ever
|
|
queued, with zero new dedup mechanism needed.
|
|
|
|
## 12. Retry/failure handling
|
|
|
|
`markFailedOrRetry` classifies every provider error as `temporary` or `permanent` (both
|
|
providers' `send()` throw an `Error` with `.errorClass`/`.errorCode` set). Permanent failures
|
|
(bad recipient, missing/invalid Google config) go straight to `failed` on the first attempt —
|
|
retrying can't fix them. Temporary failures (network/provider hiccups, rate limiting) requeue
|
|
with bounded exponential backoff (30s base, doubling, capped at 1 hour) until `MAX_ATTEMPTS` (5)
|
|
is exhausted, then `failed` too. Never an infinite loop; every attempt is persisted
|
|
(`attempt_count`, `last_error_code`).
|
|
|
|
## 13. Real bugs found and fixed
|
|
|
|
- `request-password-reset` would previously mint a `reset` token for a student who had **never
|
|
been activated** (`password_hash IS NULL`) — a "reset" for an account with no password to
|
|
reset doesn't make semantic sense, and would have sent a confusing "reset your password" email
|
|
to someone who needs activation, not reset. Fixed: that branch now also checks
|
|
`student.password_hash` and falls through to the same generic anti-enumeration response,
|
|
exactly as if the account didn't exist.
|
|
- **HTML injection in email templates.** `displayName` (which can come straight from a website
|
|
integration event's `student.display_name` — genuinely external, caller-controlled text, not
|
|
admin-typed) was interpolated directly into the HTML email body with no escaping. `templates.js`
|
|
now escapes every user-controlled value before it reaches HTML (`escapeHtml`); the plain-text
|
|
bodies are correctly left unescaped, since there's no markup there to break. Both fixes are
|
|
regression-tested in `e2e-phase5c.js`.
|
|
|
|
## 14. Testing
|
|
|
|
- `platform/test/e2e-phase5c.js` (28 checks): activation vs enrollment-confirmation decision,
|
|
duplicate/concurrent webhook idempotency, temporary/permanent/exhausted retry classification,
|
|
resend/regenerate token invalidation, activation/reset single-use and expiration, production
|
|
token-secrecy gating (a real container recreate with `COOKIE_SECURE=1`), minor/contact-email
|
|
resolution, RBAC scoping, metrics/audit hygiene, and a real Platform API restart.
|
|
- `infra/scripts/drills/test-phase5c-enrollment-activation.js` (10 checks): one continuous
|
|
story — website enrollment → activation email rendered and its URL extracted exactly like a
|
|
real inbox click → password set → real Lab launched → real scenario completed → a second
|
|
enrollment gets confirmation not activation → full password-reset round trip with session
|
|
revocation → the minor/contact-email path.
|
|
- Both against a live `docker-compose.dev.yml` stack, using the deterministic dev email provider
|
|
exclusively — **no test in this phase ever makes a real network call to Google or anyone
|
|
else.**
|
|
- Migration safety proven explicitly against a fresh Postgres taken through every prior phase's
|
|
migrations, seeded with realistic Phase 5B data (credentialed student, minor/no-email student,
|
|
cohort, enrollment with a contact-email, progress, attempts, achievements, classroom history),
|
|
then this phase's migration applied — every prior record intact; a new notification correctly
|
|
links to the pre-existing student/enrollment; re-running the migration set is a no-op.
|
|
|
|
**Live Google Workspace delivery has been validated for real**, both in isolation
|
|
(`infra/scripts/gws-email-test.js`, one message, DKIM/SPF passing, delivered) and through the
|
|
full genuine flow: a real student created, enrolled, receiving a real activation email, setting
|
|
their password from the real link, logging in, seeing their enrolled program, requesting and
|
|
receiving a real password-reset email, and completing the reset — every step against the live
|
|
Platform API, `EMAIL_PROVIDER=gws`, nothing simulated.
|
|
|
|
**One real Google Workspace limitation found and handled:** newly-authorized domain-wide
|
|
delegation does not take effect instantly across all of Google's auth infrastructure — the
|
|
identical token request succeeded, then failed with `unauthorized_client`, then succeeded again,
|
|
over the span of about a minute, before settling. This is Google-side propagation delay, not a
|
|
configuration error, and it self-resolved. `gws-provider.js` now classifies `unauthorized_client`
|
|
as a *temporary* failure (retried with the existing bounded backoff) rather than permanent —
|
|
this is exactly the failure mode that backoff exists for, and it means a real deployment doesn't
|
|
need a human to notice and manually retry during that propagation window. See
|
|
`docs/RUNBOOK_GOOGLE_WORKSPACE_EMAIL.md`'s troubleshooting section.
|
|
|
|
**A second real-world detail found during setup, not a bug:** the sending address
|
|
(`[email protected]`) is a Workspace *alias* of an existing account rather than its
|
|
own paid seat — a deliberate cost-saving choice. Domain-wide delegation authorizes a real
|
|
account, not an arbitrary alias string, so `GOOGLE_DELEGATED_USER` (the alias's real owner) was
|
|
added as a setting distinct from `EMAIL_FROM_ADDRESS` (the visible sending address) — see §2 and
|
|
the runbook §5.
|