Layered EdgeZero deploy actions + Fastly staging lifecycle (design + impl, supersedes #303)#316
Draft
aram356 wants to merge 66 commits into
Draft
Layered EdgeZero deploy actions + Fastly staging lifecycle (design + impl, supersedes #303)#316aram356 wants to merge 66 commits into
aram356 wants to merge 66 commits into
Conversation
Design docs (spec + implementation plan + adoption guide) for GitHub Actions that deploy EdgeZero apps, superseding the Fastly-only monolith from #303. Architecture: - build-cli compiles the CLI package the *application* provides (a crate in the app's own workspace), from the app checkout, isolated CARGO_TARGET_DIR + --locked, self-describing tar (cli-meta.json). - deploy-core: adapter-independent shared engine scripts sourced by wrappers; provider creds/flags only via provider-env (deploy-step-scoped), provider-env-clear, deploy-flags, deploy-args. - deploy-fastly: minimal wrapper; optional stage: true. - Fastly staging lifecycle (parity with trusted-server-actions): deploy-fastly stage mode + healthcheck-fastly + rollback-fastly, scaffolded into the CLI's Fastly adapter and exposed via the app CLI; fastly-version output. Cross-cutting: Git root vs Cargo workspace root for monorepo caching; no Python (actionlint/zizmor pinned binaries); third-party actions on readable tags; explicit Fastly build-in-deploy credential caveat. Plan includes a porting map from the #303 reference scripts. Based off main; supersedes #303.
provider-env is no longer listed among the engine's globally-passed parameters. It is bound only to the deploy step's own env: and parsed only there; setup/build steps receive only non-secret parameters plus provider-env-clear. Mirrors spec §5.2/§10 so the plan no longer reintroduces the secret-blob leak.
…te target - healthcheck-fastly / rollback-fastly now pass --service-id <id> (and step- scoped FASTLY_API_TOKEN) in their app-CLI invocations; without it the CLI can't resolve staging IPs or activate/deactivate versions. - Make provider CLI install an explicit wrapper responsibility: deploy-fastly installs the pinned Fastly CLI onto PATH; the engine assumes it is present and never learns provider tools. healthcheck/rollback need no Fastly CLI (Fastly API only). - target is wrapper-provided concrete (Fastly -> wasm32-wasip1); the engine no longer maps adapter -> target, keeping it provider-neutral. - Qualify the follow-up list: additional staging/health/rollback lifecycles are 'beyond Fastly' (Fastly's is in scope).
… guide creds Gaps found in self-review + review: - §13 error handling: add rows for staged-deploy failure, missing fastly-version, unhealthy-after-retries, rollback failure. - Pin healthcheck-fastly exit semantics: exits non-zero on unhealthy so callers can gate rollback on if: failure() (the composing example relied on this implicitly). - §5.4.3: deploy-fastly stage command now shows --service-id (matches §5.4.1). - §15 testing + §17 acceptance: cover the staging lifecycle (were absent). - §15.3 / plan smoke test: fake the app CLI + Fastly API/curl for healthcheck/rollback (they call the API, not the fastly CLI), not fake fastly binaries. - Adoption guide §6.3: healthcheck/rollback steps now pass fastly-api-token + fastly-service-id (required by the CLI --service-id path).
aram356
changed the title
- build-cli: action.yml + build-cli.sh (resolve app cli-package via cargo metadata --locked, isolated CARGO_TARGET_DIR build, cli-meta.json, tar upload). - deploy-core shared scripts: common, validate-inputs (provider-neutral allowlist + JSON→NUL parsing), install-rust (wrapper-provided target), download-cli (extract tar, read cli-meta.json, PATH-scope), resolve-project (Git root vs Cargo workspace root, cache key), cleanup, write-summary. Wrappers (deploy-fastly, healthcheck/rollback), run-cli, CI, and tests follow. All scripts shellcheck-clean; validate-inputs functionally tested.
Port install-fastly.sh (official release + SHA-256 checksum, action-owned PATH dir) and versions.json (Fastly 15.1.0) into the deploy-fastly wrapper. The wrapper action.yml and the shared run-cli.sh follow once the CLI staging contract is finalized.
…back wrappers - deploy-core/run-cli.sh: provider-neutral CLI runner; typed deploy-flags before --, caller passthrough after --; build-mode clears wrapper-named aliases. - deploy-fastly/action.yml: full orchestration (validate -> download+extract CLI -> resolve -> cache -> install rust + Fastly CLI -> optional build -> deploy), credential scoping via step-level env:, stage input -> --stage, captures fastly-version from the CLI's version=<N> line. - healthcheck-fastly / rollback-fastly: thin wrappers over <cli> healthcheck / rollback (Fastly API); healthcheck exits non-zero on unhealthy while still emitting healthy/status-code outputs. All action.yml parse; deploy-core scripts shellcheck-clean.
Apply Bash best-practices structure: wrap logic in main() with explicit local parameters and single-responsibility helpers; route the progress line to stderr; portable NUL-array collection (no bash 4.3 namerefs); a small named assertion harness (assert_succeeds/assert_fails/assert_equals) in the test runner. Kept coreutils short flags for macOS/BSD portability. All shellcheck-clean; 10/10 contract tests pass.
…st-toolchain - Apply the main()/helper structure and Bash best-practices across all engine scripts (validate-inputs, resolve-project, download-cli, install-fastly, cleanup, write-summary); route diagnostics to stderr; local scoping throughout. - Replace the custom deploy-core install-rust.sh with the maintained actions-rust-lang/setup-rust-toolchain@v1 (readable tag) in deploy-fastly, feeding the resolved toolchain + wasm32-wasip1 target; cache: false so our exact-key target/ cache stays authoritative. build-cli keeps rustup for dynamic (app-resolved) toolchain install. - Add .github/workflows/deploy-action.yml: no Python — actionlint from a pinned release binary, zizmor via cargo install (no pip), shellcheck, Bash contract tests, check-action-pins.sh (flags floating @main/@master refs), docs validation, and a build-cli -> deploy-fastly composite smoke test. - Add check-action-pins.sh; all third-party actions pinned to readable tags.
…thcheck, rollback) Add the CLI capability the deploy actions drive (spec §5.4): - args.rs: --service-id / --stage on DeployArgs; new HealthcheckArgs, RollbackArgs; Healthcheck/Rollback Command variants (+ arg-parse tests). - edgezero-adapter-fastly/cli.rs: deploy_staged (compute update --autoclone + service-version stage), emit_active_version, healthcheck (staging-IP resolution via Fastly API + curl), rollback (activate previous / deactivate staged); token piped via curl --config stdin so it never hits argv (+ 30 unit tests). - adapter registry + edgezero-cli adapter/lib/main dispatch wiring; other adapters return a clear 'unsupported' error, keeping WASM builds unaffected. - downstream CLI template: Healthcheck/Rollback arms + #[command(version)]. - Version output contract: a parseable 'version=<N>' line on stdout for deploy and staged deploy; 'rolled-back-to=<N>' / 'healthy=' / 'status-code=' for the lifecycle commands. All gated behind fastly/cli features. (Implemented by subagent; tests/clippy/fmt verified.)
… into feature/edgezero-deploy-actions
… smoke fixture - cli.rs tests: suffix numeric literals (default_numeric_fallback) and rename single-char closure params (min_ident_chars); bind+assert the ignored result (let_underscore_must_use). These fire under --all-targets, which the earlier clippy run omitted. Fastly tests: 100 pass; workspace clippy: 0 errors. - deploy-action.yml: scope actionlint to this workflow (no-arg actionlint tripped on pre-existing SC2086 in other repo workflows). - Extract the inline 'Create fixture app' block into deploy-core/tests/make-smoke-fixture.sh (shellcheck-linted) and add an empty [workspace] table so the fixture is standalone (fixes 'believes it's in a workspace').
- Set the git exec bit (100755) on run-cli.sh, deploy-fastly/common.sh, and install-fastly.sh (rewritten via editor, lost +x) so the composite actions can invoke them directly (fixes 'Permission denied' exit 126 in the smoke test). - Keep the readable @v1 tag on setup-rust-toolchain (design principle #9) and add an inline 'zizmor: ignore[unpinned-uses]' with justification, instead of an opaque SHA pin. - Give the smoke fixture a minimal fastly.toml so the CLI's Fastly deploy path reaches the fake fastly binary; assert the deploy reached 'fastly compute'.
- ShellCheck: exclude SC1091 (can't follow the dynamic $SCRIPT_DIR/common.sh source from repo root — an info finding, not a defect). zizmor now passes via the inline unpinned-uses ignore. - Smoke fixture: the real Fastly CLI (installed by install-fastly) shadowed the fake and errored on a missing package. Replace it with an edgezero.toml Fastly deploy-command override (the proven #303 approach) that records the passthrough argv; assert the typed --service-id (dummy-service) threaded through.
… cmd sites - cleanup.sh remove_if_present used '[[ -n && -d ]] && rm', which returns 1 when the dir is absent; called as a bare statement under set -e it exited non-zero, failing the deploy-fastly Cleanup step (with if: always()) even though the deploy succeeded. Use if/fi so it always returns 0. - Same footgun fixed in resolve-project.sh (lockfile hash — a real correctness bug for lockfile-less apps with cache:false) and check-action-pins.sh. - Relax the smoke assertion to marker-file existence (robust regardless of how the CLI threads passthrough args into an overridden manifest command).
…se 9) User-facing VitePress guide for the layered deploy actions: three-layer model, runner support, same-repo/separate-repo/monorepo checkout examples, build-cli and deploy-fastly input/output tables, typed-credential and trusted-ref guidance, the Fastly staging lifecycle (stage -> healthcheck -> rollback), build-mode/cache behavior, and job hardening. Wired into the VitePress sidebar under Reference. prettier + eslint + vitepress build pass locally.
HIGH 1. Fail closed on invalid lifecycle values. 'stage' must be exactly true|false (validate-inputs) and 'deploy-to' exactly production|staging (healthcheck / rollback wrappers). A typo previously fell through to PRODUCTION, so it could activate a previous production version. 2. Rollback used wrong Fastly API semantics: POST -> PUT, and staging rollback now uses PUT /version/<v>/deactivate/staging (was a plain /deactivate). Verified against Fastly's version API reference + the 2024-08 staging change. 3. curl-config injection: tokens/service-ids were interpolated into a 'curl --config -' document unescaped, so a quote/newline could terminate a value and inject options (another URL/proxy). Added curl_quote escaping plus validate_service_id / validate_version / validate_domain. The token still travels via the config file (never argv). 4. Implement the specified provider-env boundary. The wrapper no longer exports FASTLY_* directly; it passes typed values as data and run-cli.sh CLEARS every provider alias (FASTLY_TOKEN/ENDPOINT/API_URL/...) before exporting only the declared, typed credentials. Inherited aliases can no longer reach a deploy. 5. Staged deploy selected the wrong manifest: it bypassed manifest commands and searched fastly.toml from the cwd, ignoring EDGEZERO_MANIFEST — unsafe in monorepos. It now resolves and threads the configured manifest path. MEDIUM 6. A successful deploy could emit an empty fastly-version (errors were demoted to warnings), breaking deploy->healthcheck->rollback threading. Version is now parsed from the deploy output (canonical version=<N>, then Fastly's native phrasing), API only as fallback, Err if both fail; the action also fails if no version is emitted. 7. Lifecycle inputs are now required in the CLI: --service-id/--version for healthcheck and rollback, --domain for healthcheck; the token is required where it is actually used. 8. Test coverage: Bash contract tests 10 -> 19 (stage validation, artifact-name traversal, provider-env boundary), and the composite smoke now asserts version threading AND that an inherited FASTLY_ENDPOINT is cleared before deploy. 9. artifact-name is validated (no separators/traversal/leading dot) and the tarball name is fixed, so caller input is never a path component. Verified: cargo fmt/clippy(-D warnings)/test --workspace --all-targets, feature + spin-wasm checks, shellcheck, actionlint, 19/19 bash tests, prettier + docs build.
… into feature/edgezero-deploy-actions
The composite smoke test only covered a production deploy. The staging
lifecycle — stage, healthcheck, rollback — had no end-to-end coverage, which
is exactly where the review found real defects (--comment forwarded to a
command that doesn't support it, a plural staging_ips misread, POST instead of
PUT). Those are argv/verb bugs, so the test has to assert argv and verbs.
- lifecycle-smoke job: builds the app-owned fixture CLI, installs fake
`fastly`/`curl` that mirror the real contracts (singular `staging_ip`,
`--config -` on stdin), then drives stage -> healthcheck -> rollback through
the real wrappers and asserts:
* `compute update` carries --autoclone/--version=active/--non-interactive
and never --comment;
* the comment is applied via `service-version update` BEFORE staging;
* the probe is rerouted to the staging IP via --connect-to;
* an unhealthy probe FAILS healthcheck-fastly (the rollback gate);
* staging rollback PUTs /deactivate/staging, production PUTs
/version/41/activate, and rolled-back-to threads out.
- test.yml: run `cargo test -p edgezero-adapter-fastly --all-targets --features
cli`. The workspace gate never enabled the `cli` feature, so 115 adapter
dispatch tests were compiled by nothing but the clippy job.
- spec §9.1: document that compute-deploy-only flags are no-ops under --stage.
The lifecycle job's inline run blocks had grown into the largest logic in the workflow, unreadable and unlinted. Each assertion is now a named script under deploy-core/tests/ that documents the defect it regression-tests, and the YAML is a list of steps again.
…_<NAME> Security / correctness (High): - cleanup.sh removed $EDGEZERO_FASTLY_HOME, a variable nothing in the action ever set — so its value could only ever be inherited, making an `rm -rf` of the checkout (or anything on a self-hosted runner) reachable from job env. Dropped it, and confined every removal to real paths beneath RUNNER_TEMP, resolving symlinks before comparing. - run-cli.sh now scrubs its private env before exec'ing the app CLI. The typed token arrived twice — as a step variable and inside the provider-env JSON — and both stayed exported, so the CLI and every subprocess it spawned (including a manifest command) inherited the raw token under names we never promised. - Production deploy never threaded --manifest-path, so in a monorepo it fell back to "closest fastly.toml" and could deploy the wrong app. It is now threaded on both paths and stripped from the Fastly argv (compute deploy has no such flag). - A manifest-command deploy (`deploy = "fastly compute deploy"`) never received --non-interactive and could block on a TTY prompt in CI. The wrapper now supplies it as an action-owned passthrough arg; the built-in path dedupes it. Correctness (Medium): - Version parsing is anchored end-to-end. `version=15.2.0` used to parse as 15 and thread a version that was never deployed into healthcheck and rollback. - healthcheck/rollback validate their required inputs. GitHub does not enforce `required: true`, so an empty service-id or version silently reached the probe. - The toolchain search stops at the app's Git root, not github.workspace — in the separate-repo layout the deployer's .tool-versions was choosing the app's Rust. All paths canonicalized: a symlinked TMPDIR made the boundary never match. - Wrapper logs are mktemp/0600 and removed by an EXIT trap; the three aliases that were declared but never blanked (FASTLY_DEBUG_MODE/CONFIG_FILE/HOME) are blanked on every step, including third-party `uses:` steps. Env-var convention: Every action-owned variable is now EDGEZERO__<SECTION>__<NAME> — `__` between sections, `_` within. This is what makes the credential boundary a SINGLE rule (unset EDGEZERO__*) instead of a hand-maintained list that a later variable could silently escape. EDGEZERO_MANIFEST (single underscore) stays outside: it is the CLI's public contract, and the one variable we deliberately pass through. Also: build-cli -> build-app-cli (it builds the APP's CLI, never EdgeZero's own). Tests: lifecycle-smoke now drives stage -> healthcheck -> rollback through the REAL wrappers with the version threaded from the deploy output (no hard-coded 42), which required install-fastly.sh to become idempotent. Contract suite 29 -> 49, covering cleanup confinement, the env scrub, the action-owned passthrough, anchored parsing, private logs, and the toolchain boundary — the last of which caught the canonicalization bug above.
…app-cli.sh
The actions compile and run the CLI package the APPLICATION provides — never
EdgeZero's own. Half the names didn't say so, and "cli-artifact" / "cli-bin" /
"EDGEZERO__CLI__BIN" read as if they might be EdgeZero's CLI. That ambiguity is
exactly the thing this design exists to rule out, so it is swept from every layer:
- env vars: EDGEZERO__APP__CLI__{BIN,VERSION,ARTIFACT_DIR},
EDGEZERO__INPUT__APP_CLI_{PACKAGE,BIN,ARTIFACT}
- inputs: app-cli-package, app-cli-bin, app-cli-artifact
- outputs: app-cli-version, app-cli-package, app-cli-bin, app-cli-artifact
- scripts: download-app-cli.sh, run-app-cli.sh (build-app-cli.sh already renamed)
- artifact: app-cli-meta.json, with app-cli-{bin,version,package} keys
- docs: guide, spec, adoption guide, and plan all updated
The contract test for the artifact metadata caught the one place the rename would
have broken the wiring (the download step's outputs), which is what it is for.
`tar -tvf … | grep -qE '^[lh]'` is unsound under `set -o pipefail`: grep -q exits on the FIRST match, tar takes SIGPIPE, and the pipeline reports tar's failure — so the `if` reads false exactly when a link WAS found, letting an unsafe archive through. Capture the listing first, then match. Not reproducible on macOS (bsdtar buffers a small archive and never sees SIGPIPE), which is why the existing test passed; the hazard is real on GNU tar. The fix removes the class rather than the symptom.
…nt, shadowing Critical — production rollback could promote a staged version. previous_version was literally current-1, but the version counter increments over every draft and staged version. With prod v5, staged v6, prod v7, rolling back v7 activated v6 — publishing staged code AND (since a staged draft is re-linked to the staging config selector) staged config. Now resolved from the service's version list: the highest version below current that is not staged and not an untouched draft. Fails when nothing qualifies rather than guessing. Verified against the exact reported case. Staging edge cases the --staging design surfaced: - stage:true was broken for apps with NO config stores (I introduced this): the relink required the staging store unconditionally. Now it is a no-op when the app has no runtime-override store at all — staging code with no config to isolate is valid — and still fails closed when the app HAS one but the staging twin is missing. - Re-provision did not reconcile: once the [setup] marker existed, provision returned before writing selector entries, so a config store added later silently fell back to its production key. Selector entries are now upserted for every declared store on every provision (idempotent), and a remotely-deleted staging store is recreated — which is what the deploy-time error tells the operator to do. - --key + --staging is now refused, not silently wrong. An explicit key would be written where the (logical-id-derived) staging selector never points, so the push would go nowhere. Rejected at clap (conflicts_with) and at the seam. - provision guidance no longer tells the operator to point the PRODUCTION store at a _staging key (which would make the live service read staged config). Also (independent of staging): - PATH shadowing: an app CLI named `fastly` resolved to the provider CLI, since install-fastly prepends its own dir. download-app-cli now emits the app CLI's ABSOLUTE path; all four invocation sites invoke that, immune to PATH order. - build-app-cli confines cargo metadata's workspace_root to the app repository — an enclosing deployer workspace could otherwise control the lockfile and deps. - Default `edgezero.toml` discovery is confined (deploy + config-push): a committed symlink named edgezero.toml could point outside the app repo. - Archive link check no longer pipes tar into grep -q under pipefail (a false negative that could let a symlink-bearing archive through on GNU tar). 84 -> 88 shell contracts; new Rust tests for the rollback target and the config-store-less staging path. All five CI gates + docs green; the fastly suite no longer touches a real account.
The deploy-fastly validate step did its input checks inline in action.yml `run:` bash — which CI's shellcheck never sees (it globs scripts/*.sh) and which no contract test can reach. All four wrappers had the same smell. Each wrapper now has scripts/validate.sh: - deploy-fastly: artifact + token presence, service-id format, then delegates to the engine's provider-neutral validate-inputs.sh. - config-push-fastly: artifact + token presence, deploy-to fail-closed. - healthcheck/rollback: artifact presence. The Fastly-specific checks stay in the WRAPPER, never in the provider-neutral engine validate-inputs.sh — putting fastly-service-id there would leak a provider name into the layer that must not know any. New `require_present` helper asserts a credential is set via the precomputed `…_PRESENT` bool without the secret ever reaching the validate step. 12 new contract cases exercise every branch — the point of moving it out of YAML. 88 -> 100 shell contracts; shellcheck + actionlint clean.
The staged relink now no-ops when the app has no `edgezero_runtime_env` selector store (an app with no config to isolate). The smoke's fake listed only the app store and the staging twin, so the relink was correctly skipped — but the assertion expects it. The smoke exists to prove the relink works, so its fake now also lists `edgezero_runtime_env`, modelling an app that does select config.
The rollback Critical fix made production rollback GET the version list to resolve its target, but the smoke's fake curl answered every GET with the domain/staging-IP array — so resolution found no versions and (correctly) refused, failing the smoke. The fake now returns a version list for `/version`, and it is built to actually test the fix: v42 and v41 are BOTH staged, v40 is active. `version - 1` (= 41) would be a staged version — the exact Critical — so only resolution lands on 40. The assertion now requires the GET, forbids activating staged-1, and expects v40. The previous assertion used staged-1 arithmetic that coincidentally matched and would not have caught a regression.
Five assert scripts still used the inline `Reads (env): A, B, C.` one-liner while every other action script (and assert-config-push.sh) uses the aligned per-var block with a required/optional column. Converted them for consistency — comment-only, no behavior change.
… one Rollback inferred the target as `version - 1` (earlier: a resolver that filtered on a `staged` field). Fastly's version object has NO such field — the real field is `staging`, documented "Unused", and `locked` only means "not editable", which a staged version is too. So the inference activated the staged version in a staged-then-rollback flow, publishing staged code and (via the staged config relink) staged config. The passing test only passed because it fabricated a `staged` field Fastly never emits. There is no field to distinguish a previously-live version from a staged one, so the target cannot be inferred — it is captured before the deploy that supersedes it and passed in: - New CLI verb `active-version` prints `version=<active>` (reads the real `active` field). deploy-fastly runs it BEFORE a production deploy and emits `previous-version`. - rollback-fastly gains a `rollback-to` input; production activates it verbatim and fails closed when it is absent. Staging is unchanged (deactivate). - The fabricated-field resolver and its test are deleted. app-demo (the reference downstream CLI) now wires the full lifecycle surface: healthcheck, rollback, and active-version. Verified against the real fixture CLI: active-version resolves the active version, production rollback activates the explicit target, and a targetless production rollback errors. Smoke asserts the explicit-target activation and that a targetless production rollback is refused. 100 shell contracts; fmt/clippy/tests green.
The app-demo workspace (excluded from the main workspace, linted separately in CI) enforces alphabetical item ordering. The new ActiveVersion/Rollback variants were out of order.
…time
The staging config twin previously held only the config `__KEY` selectors,
written at provision time. Two defects followed:
* A staged version relinked to the twin lost every OTHER production runtime
override (adapter host, logging level, `__NAME` redirects), since the twin
never carried them.
* The relink guard treated any `edgezero_runtime_env` lookup error as "no
config to isolate" and skipped — failing OPEN on a transient CLI/schema
failure, and forcing a re-provision whenever a store was added.
A staged deploy now owns the twin end to end. It reads the production selector
store's CURRENT entries, copies the non-config overrides verbatim, redirects
every declared config store's selector to `<logical>_staging`, reconciles the
twin (deleting entries production no longer has), then relinks — so the twin
always mirrors production with only config isolated. The declared config stores
are threaded from the app manifest by the CLI as `--edgezero-staging-config`,
never a remote probe. Lookup FAILURE now fails closed; a genuine NotFound (or an
app that declares no config) is a no-op. The twin is created on demand, so
provision no longer touches it (a provision-time twin would drift the moment an
override changed).
…docs The CLI derives a staging config key from the store's logical id (`<logical>_staging`) and refuses an explicit `--key` alongside `--staging`, but the config-push-fastly wrapper still exposed a `key` input usable with `deploy-to: staging` and its docs described a `<key>_staging` variant that composed with `key`. A caller who set both got a confusing CLI-level error deep in the push. The wrapper's validate.sh now rejects `key` + `deploy-to: staging` up front with a clear message, mirroring the CLI's mutual exclusion. Input descriptions, the push-script comments, and the guide/spec docs now state that the staging key is derived from the logical store id and that `key` is production-only. The docs also correct two now-stale claims: the staging twin is created and mirrored by a staged deploy (not by provision), and a staged deploy fails closed on an unreadable store listing (not on a missing twin).
The rollback docs still implied a production rollback re-activates an inferred "previous version" (`<v> - 1`), which Fastly's version metadata cannot support — a superseded production version is indistinguishable from a staged draft. The guide and spec now describe the real contract: a production `deploy-fastly` emits `previous-version` (captured via `active-version` before the deploy supersedes it), which the caller threads into `rollback-fastly`'s `rollback-to`. A production rollback without a target fails closed. Adds the missing `rollback-to` input and `previous-version` output to the tables and a production-rollback wiring example.
The lifecycle smoke test fakes the Fastly CLI, so install-fastly.sh's real download + SHA-256 verification path is never exercised there. This dedicated workflow runs it un-faked — fetching the pinned release, verifying its checksum, and confirming the installed binary reports the pinned version. It is path-filtered to the installer, its pinned versions.json, and the shared common.sh (not scheduled), so a version/URL/checksum bump is validated against the real release exactly when it changes.
`fastly version | head -n 1` under `set -o pipefail` made `fastly` exit 141 (SIGPIPE) when head closed the pipe, failing the step even though the install itself succeeded. Capture the full output and take the first line with parameter expansion instead of piping.
The verify step's inline shell was heavy enough to belong in its own file, like the other action scripts. Extract it to deploy-fastly/scripts/verify-installed-version.sh (sourcing common.sh, using the standard Reads-env header), so it is shellcheck'd by CI and testable outside the YAML. The workflow step now just invokes it.
Removes internal spec-section references (§5.4, 8.x, 9.x, 12.x), PR/round/finding markers (PR #269, Post-F4/F6, F1/F2), plan-phase labels (Stage 5/6), and test-doc severity prefixes from comments and user-facing strings in the files this branch touches. The explanatory content is preserved verbatim — only the process reference is dropped and the surrounding sentence re-flowed. Comments now describe what the code does, not which spec section or review round produced it. Two deliberate survivors: the raw-push error string points at the real design doc's §3.2.1 (a user-facing pointer, and asserted by tests), so it and its test assertions are left intact.
The workflow invokes the script directly (`run: .../verify-installed-version.sh`), which needs the execute bit — it was committed 100644, so the installer-check step failed with permission denied. Mark it 100755 like the sibling scripts.
The scaffold's `main.rs.hbs` shipped Deploy/Healthcheck/Rollback but not ActiveVersion, so a freshly generated app CLI rejected the `active-version` command that a production deploy's rollback-target capture invokes — leaving the deploy to proceed with no rollback target. The lifecycle smoke fixture added the command by hand, masking the omission. Add ActiveVersion (import, variant, dispatch) to the template so generated apps expose it out of the box.
Four fixes to the Fastly staging/rollback lifecycle:
* Per-service staging store. The staging selector twin was a single
account-wide `edgezero_runtime_env_staging` store, destructively reconciled
and relinked on every staged deploy — so two services staging on one account
would clobber each other's selectors. The twin is now named per service
(`edgezero_runtime_env_staging_<service_id>`), created on demand.
* Isolate even without a production override store. When an app declares
config but has no `edgezero_runtime_env` store, the relink returned early and
the staged version silently read production's default key. It now still
creates the per-service twin, writes the `<logical>_staging` selectors (there
is simply nothing to mirror), and relinks — staging stays isolated.
* Distinguish "no active version" from an operational failure. The active-
version emitter returned Err for both a first-ever deploy AND API/auth/parse
failures, and the capture step's `|| true` collapsed both to an empty
rollback target. It now emits an empty `version=` and succeeds only when the
API confirms no active version; a real failure returns Err, and the capture
step fails closed rather than deploy with no way back.
* Run lifecycle commands from the app directory. active-version / healthcheck /
rollback load their manifest from the current directory, so in a monorepo a
stray root `edgezero.toml` could shadow the app's. The wrappers now thread
`working-directory` and the scripts cd into the app dir before invoking the
CLI (a shared `enter_app_dir` helper).
An extensionless `rust-toolchain` may be a legacy single-line channel OR a TOML document (`[toolchain]` + `channel = "..."`) — rustup accepts both. The channel- file parser always took the first non-blank line, so a valid TOML document resolved the toolchain to the literal `[toolchain]`. Detect the TOML form and parse its channel instead, in both build-app-cli.sh and resolve-project.sh.
The adoption guide, implementation plan, and spec output table still showed a production rollback without a target and omitted the previous-version output. Document the real flow: deploy-fastly emits previous-version, threaded into rollback-fastly's rollback-to (Fastly cannot infer it). Also correct the installer-gate section: the check is path-filtered, not scheduled, per the decision to avoid downloading the CLI on unrelated runs.
The deploy path threads resolve-project.sh's RESOLVED (absolute) working- directory into the capture step, but enter_app_dir prepended github.workspace again — breaking the production smoke. Accept both an absolute path (the resolve output) and a workspace-relative path (raw wrapper inputs used by healthcheck / rollback), with a contract test covering both plus escape/missing rejection. Also run Prettier on the spec doc.
deploy-fastly now captures the rollback target before a production deploy via a real `active-version` Fastly API call, and fails closed when it cannot resolve one. The production composite-smoke drove a manifest-command deploy with no fake `curl`, so that capture hit the real API and (correctly) failed closed — a case the old fail-open capture had silently swallowed. Set up the fake Fastly env in that job so active-version resolves the fake active version (40), and assert previous-version=40 threads out, locking in the capture contract. The staged lifecycle-smoke skips capture (stage=true) and already had the fake env.
Correctness:
* resolve_active_version now rejects MALFORMED active entries (a non-boolean
`active`, or an `active: true` entry whose `number` is missing or not an
unsigned integer) as operational errors instead of reading them as "no active
version" — closing the last fail-open path in rollback-target capture.
* The production deploy's version fallback passes `--require-active`, so a
post-activation API response with no active version is an ERROR, not a silent
empty-`version=` success for direct CLI consumers.
* active-version / healthcheck / rollback are now manifest-INDEPENDENT: they are
pure Fastly-API operations keyed on explicit flags and can never be
manifest-command overrides, so they no longer load the manifest. This fixes
monorepo runs where a stray root edgezero.toml (or an explicit `manifest:`
selection) shadowed the app's, and removes the now-unnecessary enter_app_dir /
working-directory scaffolding.
Tooling:
* The rust-toolchain TOML parser accepts a trailing `# comment` after the
channel value (valid TOML).
* The installer gate also triggers on .tool-versions — install-fastly.sh
requires it to agree with versions.json, so a Fastly tool-version change must
run the real installer.
Tests:
* The production smoke now threads the deploy's `previous-version` into a real
rollback's `rollback-to` (no hardcoded version) and asserts the activated
target; the staged healthcheck asserts its `status-code`/`healthy` outputs;
and the generator test asserts a generated CLI exposes active-version /
healthcheck / rollback.
Docs: correct the credential-boundary contract (lifecycle steps take the token
directly under the adapter convention), the per-service staging store name, and
the path-filtered (not scheduled) installer gate.
Security / correctness:
* install-fastly.sh no longer adopts a pre-seeded binary by its version text —
which let an earlier step plant an executable that later ran with
FASTLY_API_TOKEN. The `fastly` binary is now ALWAYS extracted from a
SHA-256-verified archive (cached under downloads/, so still no refetch on a
second run). The lifecycle smoke delivers its fake through that same verified
path: it packages the fake as a tar.gz at the cache path and repoints its
CHECKED-OUT versions.json at it with a matching checksum, so the real
download+verify+extract is exercised rather than bypassed.
* resolve_active_version scans the ENTIRE version list instead of returning at
the first active entry: a non-boolean `active` anywhere, or more than one
active version, is now an operational error rather than a silently-accepted
or ignored result.
* fastly_api_get uses curl's `fail` directive, so an HTTP 4xx/5xx (even an
array-shaped error body) can never be mistaken for valid version data.
* find_config_store_id treats ANY malformed entry as schema drift, so a
malformed entry can't be masked by an unrelated well-formed one and read as
NotFound (which would have failed staging isolation open).
Tests / robustness:
* The production `deploy` version fallback's `--require-active` policy is now
unit-tested (active_version_or_require). The unhealthy smoke asserts
status-code=503 and an explicit healthy=false.
* A credential-free `active-version --help` preflight fails a production deploy
early (before the token-bearing call) when the app CLI lacks the command, and
the guide documents the required command surface.
CI / docs:
* The installer gate's path filter watches deploy-fastly/common.sh (which the
installer actually sources) and the workflow is now covered by
actionlint/zizmor and the pin checker.
* Credential-boundary docs note the capture step also receives the token
(adapter convention); removed the nonexistent adapter/effective-build-mode
outputs from the spec/plan; the guide names the per-service staging store.
Correctness:
* resolve_active_version now requires EVERY element to be a version object with
an unsigned-integer `number` (an omitted `active` means "not active"). A
null, a non-object, or a missing/non-numeric `number` is an operational error
rather than being skipped and read as "no active version" (fail open).
* find_config_store_id scans the WHOLE listing instead of returning on the
first match, so a malformed entry after the match, or a duplicate store name,
is caught (schema drift) rather than silently accepted.
* fastly_api_get validates a 2xx status via write-out (as the PUT helper does),
so a 3xx redirect body — which `--fail` alone would accept — can no longer be
parsed as version data. The fake curl mirrors the new body+status shape.
* The credential-free active-version preflight runs with FASTLY_API_TOKEN
explicitly unset, so the token truly reaches only the real API call.
Tests:
* The lifecycle smoke now PLANTS a decoy binary at the extract target that
reports the pinned version but errors on every real command — a standing
regression guard: if install-fastly ever re-adopts a pre-seeded binary by its
version text, the staged deploy fails. Adjacent to the reworked verified-
archive fake delivery.
* A capture-previous contract test covers first-deploy (empty previous-version,
still succeeds), an active version threading out, fail-closed on an
operational error, and the missing-command preflight.
Tooling / docs:
* The duplicated inline actionlint install is replaced by the reusable,
checksum-verifying scripts/install-actionlint.sh, now shellcheck'd and
path-triggered.
* Credential docs (guide table + spec acceptance criterion) note the capture
step; the adoption guide gains a required-command-surface checklist; the plan
lists config-push-fastly and active-version.
Blockers:
* Rollback-target capture now fails CLOSED when active-version exits 0 but emits
no `version=` contract line (e.g. an app CLI that never called
init_cli_logger). A silent success is no longer mistaken for a first deploy
and allowed to deploy with no rollback target.
* A production rollback verifies the version it is rolling back FROM
(`--version`) is STILL the active version before activating the target. A
delayed rollback that races a newer deploy is refused rather than silently
clobbering the newer version. The lifecycle smoke's fake Fastly API now tracks
the active version through a state file so this compare-and-swap is exercised
end to end; the guard logic is also unit-tested.
* The adoption guide documents the logger/output contract: hand-written CLIs
must call edgezero_cli::init_cli_logger() (or emit equivalent unprefixed
output) or the run_* handlers' machine-readable lines are swallowed.
Also:
* resolve_active_version fails closed on an EMPTY version list (a real service
always has at least an initial version).
* Test harness hardened: the capture preflight fake asserts FASTLY_API_TOKEN is
absent (guarding the scrub), and the fake fastly exits non-zero on an
unhandled command (an unexpected provider call now breaks the smoke).
* Docs: the spec's credential principle notes the lifecycle steps receive the
token directly; healthcheck `retry` is documented as total attempts; and the
guide names the app-owned CLI for `config push` (the bundled binary returns
unsupported).
Blockers:
* Rollback-target capture now rejects a MALFORMED `version=` value, not just an
absent line. `version=12abc` previously passed the presence check, then the
anchored numeric parse dropped it to empty and the deploy continued as a
first deploy. Capture accepts only `version=<digits>` or an empty `version=`;
anything else fails closed. Covered by a new contract test.
* The production rollback's staleness check is no longer described as a
compare-and-swap. It reads the active version and activates in two separate
requests, and Fastly's activate endpoint takes no precondition, so a deploy
landing in that window can still be clobbered. Code comments, the guard's
docs, and the guide now say so plainly, and the recommended concurrency group
is SERVICE-scoped (not per-ref) — that is what actually closes the window.
Coverage:
* A stale-rollback smoke test: a newer version becomes active, the rollback is
refused, and the assertion proves it activated NOTHING (refusal precedes
mutation). The fake Fastly API now records activations so an activate
genuinely changes the active version a later read sees.
Also:
* `retry` must be >= 1 — it is a total attempt count, and 0 was accepted then
silently coerced to 1.
* Spec: drop the unimplemented `--output` file version contract (only the
anchored `version=<N>` line is parsed); align the goal and test-strategy
credential-scope wording with the principle (all provider-calling steps).
* Adoption guide: staging config push writes the derived
`<logical-store-id>_staging` key, and an explicit `key` is rejected with it.
…and rollback tests
Security:
* build-app-cli's build step runs APP-CONTROLLED code (cargo build, and the
built CLI's --help) — it now blanks every inherited provider alias, so a
job-level FASTLY_API_TOKEN can never reach a compile or a --help of untrusted
code. The upload step and the healthcheck/rollback validate steps blank them
too (only the actual provider step receives the token).
* A production healthcheck no longer requires or receives the token; the adapter
reads it only for staging-IP resolution. `fastly-api-token` is optional and
passed to the probe only for deploy-to: staging.
Fail-closed:
* Rollback-target capture now requires EXACTLY ONE `version=` contract line, so
a malformed line hidden behind a later well-formed one (`version=12abc`
followed by `version=`) can no longer read as a first deploy. Covered by
order-dependence and conflicting-line tests.
Tests / docs:
* The stale-rollback smoke now snapshots the call log and asserts the DELTA:
the active-version GET happened (the guard actually ran, not an earlier
startup failure), no activate PUT was issued, and the active version is still
99. The fake API records activations so a read reflects them.
* Fixed a stray folded `echo` that turned into an argument to the stale-rollback
assertion (dead line).
* The hand-written-CLI contract in the guide now names init_cli_logger, the
typed config-push path, and the pushed-store= output.
* The rollback serialization guidance is honest: a repo-scoped concurrency group
closes the race only when every mutation shares one deployment authority.
* Spec: credential criterion covers all provider-calling steps; retries note the
healthcheck probe; and the stale "no --version" claims are dropped (the
template sets clap version).
This was referenced Jul 20, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Layered, adapter-independent GitHub Actions for deploying EdgeZero apps —
design + implementation — superseding the Fastly-only monolith in #303.
Based off
main.Actions:
build-cli— compiles the CLI package the application provides (a cratein the app's own workspace), from the app checkout, isolated
CARGO_TARGET_DIR--locked; publishes a self-describing tar (cli-meta.json).deploy-core— adapter-independent shared engine scripts sourced bywrappers; provider creds/flags only via
provider-env(deploy-step-scoped),provider-env-clear,deploy-flags,deploy-args.deploy-fastly— minimal wrapper; installs the pinned Fastly CLI; optionalstage: true; outputsfastly-version.healthcheck-fastly/rollback-fastly— Fastly staging lifecycle (paritywith
stackpop/trusted-server-actions), driven by the app CLI over the FastlyAPI.
CLI scaffolding (
edgezero-adapter-fastly+ downstream template):--stage,healthcheck/rollbacksubcommands, deployment-version output,--version.Design docs
docs/specs/edgezero-deploy-github-action.md— specdocs/specs/edgezero-deploy-action-implementation-plan.md— plan (+ Add Fastly deploy action with config push #303 port map)docs/specs/edgezero-deploy-adoption-guide.md— adoption guide (any app repo)Cross-cutting
(
actionlint/zizmorpinned binaries); third-party actions on readable tags;explicit Fastly build-in-deploy credential caveat;
provider-envscoped to thedeploy step only.
Notes
Supersedes #303 (and the earlier stacked docs PR #315). #303's unrelated changes
(KV timing logs, dep bumps) are not carried here.
🚧 Implementation in progress — phases tracked in the plan doc.