diff --git a/docs/superpowers/specs/2026-07-14-cache-header-audit-design.md b/docs/superpowers/specs/2026-07-14-cache-header-audit-design.md new file mode 100644 index 00000000..5ac6610b --- /dev/null +++ b/docs/superpowers/specs/2026-07-14-cache-header-audit-design.md @@ -0,0 +1,426 @@ +# Technical Specification: Origin Cache-Header Audit (`ts dev audit headers`) + +**Status:** Draft +**Author:** @vasujai +**Epic:** [#834](https://github.com/IABTechLab/trusted-server/issues/834) +**Planning task:** [#835](https://github.com/IABTechLab/trusted-server/issues/835) +**Related:** [#293](https://github.com/IABTechLab/trusted-server/issues/834) (cache header refactoring), [#428](https://github.com/IABTechLab/trusted-server/issues/428) (ETag multi-value), PR #860 +**Last updated:** 2026-07-14 + +--- + +## 1. Overview + +Trusted Server serves many content types through one edge hostname -- HTML, JS bundles, creatives/images, static assets, RTB/JSON -- each with a different optimal caching posture. Today TS mostly passes origin cache directives through untouched and exposes no diagnostics, so publishers can't tell whether each content type is cached correctly. + +This spec defines `ts dev audit headers`, a CLI command that audits origin response cache directives grouped by content type and returns a per-type pass/warn/fail verdict naming the responsible header and recommending the correct value. + +**Out of scope:** Auto-fixing headers, request-side cache-key/hit-ratio analysis, and the `ts dev proxy` tool. + +--- + +## 2. Command Surface + +``` +ts dev audit headers [OPTIONS] [URLS...] +``` + +### Arguments + +| Argument | Description | +|----------|-------------| +| `URLS...` | Explicit URLs to audit (optional) | + +### Options + +| Flag | Description | Default | +|------|-------------|---------| +| `--config ` | Path to `trusted-server.toml` | `./trusted-server.toml` | +| `--origin ` | Override origin URL (skips config lookup) | from config `publisher.origin_url` | +| `--json` | Machine-readable JSON output | human table | + +### Exit codes + +| Code | Meaning | +|------|---------| +| 0 | All groups pass | +| 1 | At least one group has a FAIL verdict | +| 2 | Warnings only (no fails) | + +--- + +## 3. Content-Type Taxonomy + +Responses are classified into groups by their `Content-Type` response header: + +| Group | Matching Content-Types | +|-------|----------------------| +| `Html` | `text/html` | +| `JavaScript` | `application/javascript`, `text/javascript`, `application/x-javascript` | +| `Image` | `image/*` | +| `StaticAsset` | `text/css`, `font/*`, `application/font-*`, `application/woff*` | +| `RtbJson` | `application/json` | +| `Other` | Everything else (not evaluated, reported as info) | + +--- + +## 4. Cacheability Rules + +Each content-type group has an expected caching posture. Rules are evaluated against: `Cache-Control`, `Surrogate-Control`, `Surrogate-Key`, `Vary`, `ETag`. + +### 4.1 HTML (personalized/consent-sensitive) + +| Header | Expected | Verdict if wrong | +|--------|----------|-----------------| +| `Cache-Control` | Contains `private` AND (`no-store` OR `no-cache`) | FAIL: "HTML served without private/no-store risks sharing personalized content across users" | +| `Vary` | Should NOT contain `*` | WARN: "Vary: * disables all caching including CDN" | +| `Surrogate-Control` | If present, `no-store` or `private` | FAIL: "CDN may cache personalized HTML" | + +### 4.2 JavaScript bundles (hashed filenames, immutable) + +| Header | Expected | Verdict if wrong | +|--------|----------|-----------------| +| `Cache-Control` | Contains `public` AND `max-age>=31536000` AND `immutable` | WARN: "Hashed JS bundles should use long-lived immutable caching" | +| `ETag` | Present (strong preferred) | WARN: "Missing ETag means no conditional revalidation" | +| `Surrogate-Key` | Present | WARN: "Without Surrogate-Key, this content type can't be selectively purged from the CDN -- full-cache purges required" | + +### 4.3 Images / Creatives + +| Header | Expected | Verdict if wrong | +|--------|----------|-----------------| +| `Cache-Control` | Contains `public` AND `max-age>=86400` | WARN: "Images re-fetched too frequently" | +| `Surrogate-Control` | If present, `max-age` should be >= `Cache-Control` max-age | WARN: "CDN caching shorter than browser caching" | +| `Surrogate-Key` | Present | WARN: "Without Surrogate-Key, creatives can't be selectively purged from the CDN -- full-cache purges required" | + +### 4.4 Static Assets (CSS, fonts) + +| Header | Expected | Verdict if wrong | +|--------|----------|-----------------| +| `Cache-Control` | Contains `public` AND `max-age>=31536000` AND `immutable` | WARN: "Static assets should use long-lived immutable caching" | +| `Surrogate-Key` | Present | WARN: "Without Surrogate-Key, static assets can't be selectively purged from the CDN -- full-cache purges required" | + +### 4.5 RTB/JSON (real-time, never cached) + +| Header | Expected | Verdict if wrong | +|--------|----------|-----------------| +| `Cache-Control` | Contains `private` AND `no-store` | FAIL: "RTB responses cached = stale bids served to users" | +| `Surrogate-Control` | If present, `no-store` | FAIL: "CDN caching RTB responses" | + +### 4.6 ETag handling (ref: #428) + +When evaluating ETag presence, handle multi-value `If-None-Match` correctly: +- Accept both strong (`"abc"`) and weak (`W/"abc"`) ETags +- Multiple comma-separated values in `If-None-Match` are valid per RFC 7232 + +### 4.7 Surrogate-Key evaluation + +`Surrogate-Key` (Fastly) enables targeted cache purging by tag. It is evaluated only on +cacheable groups (JavaScript, Image, StaticAsset) -- non-cacheable groups (Html, RtbJson) +skip the check since uncached content never needs purging. Absence is a WARN, never a +FAIL: caching still works without it, but operational purges become all-or-nothing. + +--- + +## 5. URL Discovery + +When no explicit URLs are provided: + +1. Read `publisher.origin_url` from `trusted-server.toml` +2. Probe well-known paths per content type: + - HTML: `/` + - JS: discover from `