Skip to content

Add macOS CI and mandatory real external-contract release validation #16

Description

@douglasjarquin

Problem

Made’s normal GitHub Actions workflow is Ubuntu-only. Platform-sensitive socket, filesystem, process-group, and reviewer-containment behavior on macOS is not continuously validated. Real GitHub CLI and Codex tests are opt-in and can be skipped, so local fakes can drift from the actual installed tools.

The previous remediation merge explicitly did not run a real project gate/pipeline, ask-user response, or complete external flow.

Required implementation

  1. Add a macOS CI lane for platform-sensitive packages and integration tests.
  2. Keep the existing pinned Ubuntu build/test/race/vet/lint lane.
  3. Exercise on macOS:
    • Unix socket preparation and permissions
    • singleton and shutdown behavior
    • process-group cancellation
    • sandbox-exec reviewer containment
    • filesystem/symlink behavior
    • shell/tool portability
  4. Exercise on Linux:
    • Bubblewrap containment in the same mode used in production
    • process cleanup and socket behavior
  5. Add a release-grade real GitHub CLI smoke test against a disposable repository that verifies the production PR/check/run/log/rerun command surface.
  6. Add a release-grade real Codex structured-review smoke test against a disposable repository with a known introduced defect.
  7. Real external tests may be separated from ordinary PR CI when credentials/cost require it, but they must be mandatory for release/default enablement and must report clearly when not run.
  8. Pin all actions, Go versions, linters, and external test dependencies.
  9. Record exact tool versions and output as test evidence.
  10. Add a release checklist or automated gate that refuses to call the integration validated when required real tests were skipped.

Acceptance criteria

  • Ubuntu and macOS CI are green.
  • Real production gh invocations are exercised against GitHub, not only a fake.
  • Real installed Codex produces a structured review result through Made’s production adapter.
  • Containment behavior is tested on both supported operating systems.
  • Skipped external tests are visible and cannot be mistaken for passing release validation.
  • Tool and action versions are reproducible and pinned.

Required tests

  • duplicate daemon start and socket safety on macOS/Linux
  • graceful process-group cleanup on macOS/Linux
  • Linux Bubblewrap sentinel containment
  • macOS sandbox sentinel containment
  • real GitHub pending-to-pass flow
  • real GitHub failed workflow/run-ID handling
  • real Codex known-defect review
  • release gate behavior when a required live test is missing

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions