diff --git a/INVARIANTS.md b/INVARIANTS.md index af142da1..23e88cb2 100644 --- a/INVARIANTS.md +++ b/INVARIANTS.md @@ -20,5 +20,6 @@ materialization, messaging, DING, or presence must preserve them. | **Retirement health** | A retired declaration is healthy only after every declared task ID is absent. Any live or dead declared task record reports incomplete retirement; retired declarations do not require presence. Live declarations retain their existing task and presence checks. | `tests/doctor.rs::retired_declaration_is_healthy_when_tasks_and_presence_are_absent`; `tests/doctor.rs::retired_declaration_is_unhealthy_while_a_declared_task_is_alive`; `tests/doctor.rs::retired_declaration_is_unhealthy_while_a_dead_task_record_remains` | | **Crash loops surface** | A task parked by a fail-mode restart policy notifies its supervisor once over the bus. | `tests/run.rs::surface_crash_loop_notifies_the_supervisor_over_the_bus` | | **Tracked workspaces fail closed** | Materialization simulates content operations before writing and refuses a real change to any Git-tracked target. Byte-identical tracked, untracked, and non-Git targets retain useful behavior. | `tests/materialize.rs::every_content_directive_refuses_to_change_a_tracked_target_before_any_write`; `tests/materialize.rs::byte_identical_tracked_target_is_allowed_without_modification`; `tests/materialize.rs::untracked_and_non_git_targets_remain_materializable` | +| **Catalog control state stays host-local** | Git-backed catalog creation and initialization add `.st2/` to the repository-local exclusion. Any copied incomplete marker remains authoritative and blocks destination reads. | `tests/catalog_apply.rs::bootstrap_adds_the_control_directory_to_the_containing_git_exclusion`; `tests/catalog_selection.rs::catalog_initialization_adds_the_control_directory_to_the_local_git_exclusion`; `tests/catalog_selection.rs::a_copied_incomplete_marker_blocks_destination_catalog_reads` | | **Native flat root** | Without an authored override, catalog tasks, eval messaging, shell helpers, and DING all use the catalog itself as `ST_ROOT`; no nested bus directory is synthesized. | `src/eval_run.rs::bus_root_expands_st_root_else_defaults`; `tests/eval_run_e2e.rs::st2_eval_runs_a_benign_folder_to_a_pass_verdict`; `tests/pty.rs` | | **Proof references resolve** | Every qualified test named in this table exists in its named source file, so stale invariant claims fail the suite instead of silently surviving a refactor. | `tests/invariants.rs::qualified_proof_references_resolve` | diff --git a/README.md b/README.md index 9020a37d..ae1f08e3 100644 --- a/README.md +++ b/README.md @@ -132,6 +132,15 @@ st2 catalog bootstrap --catalog "$NEW_CATALOG" --prepared ./prepared \ --input-sha256 --json ``` +A Git-backed catalog must exclude `.st2/`. st2 adds this line to the +repository-local `.git/info/exclude` when it creates or initializes the catalog +control directory. For an existing catalog, add the exact `.st2/` line before +the next sync. This directory is host-local. A copied incomplete marker blocks +catalog reads at the destination. If Git already tracks `.st2`, an exclusion +cannot untrack it. After adding the exclusion, run +`git -C "$CATALOG" rm -r --cached -- .st2` and commit that index removal. This +keeps the local control files. + `catalog apply` is policy-free. It rejects state/control content, symlinks, unprojected workspace facts, catalog-local/default PTY roots, and effective PTY-root changes. Bootstrap is a separate create-only declaration transaction, diff --git a/docs/vrs/spec.md b/docs/vrs/spec.md index 01b07401..863d8089 100644 --- a/docs/vrs/spec.md +++ b/docs/vrs/spec.md @@ -191,8 +191,9 @@ the contract is monotonic change detection, not an exactly-once counter. The lock file is a persistent real inode: replacing or removing it would split the lock domain for a process that already has it open. Consequently, the first -coherent declaration reader may initialize exactly `.st2` and this lock even -when its requested operation later refuses. Refusal still performs no +coherent declaration reader may initialize `.st2`, this lock, and the +repository-local Git exclusion. A non-Git catalog does not require Git. The +requested operation may still refuse after initialization. Refusal performs no declaration, workspace, or state mutation. The publisher derives the destination from the captured declaration, replaces @@ -332,6 +333,10 @@ files remain live while a declaration is admitted. `/.st2/catalog-apply-incomplete` is the durable whole-catalog transaction fence. Any presence is authoritative, including malformed content. +The `.st2/` directory is host-local and must not enter catalog transport. A +copied incomplete marker blocks declaration reads at the destination. When the +catalog belongs to a Git worktree, st2 adds `.st2/` to the repository-local Git +exclusion during control initialization and bootstrap. The reserved canonical record is: ```json @@ -370,11 +375,12 @@ be one absent final component below an existing canonical real parent. st2 captures `DIR` through retained no-follow capabilities, verifies its declaration root against `HEX`, admits the complete projection against logical `ROOT`, and requires one explicit external PTY root. It materializes a 0700 sibling stage, -creates the persistent authoring lock and generation `1` inside it, takes EX on -that lock, fsyncs the complete tree, and publishes it with a capability-relative +ensures the repository-local Git exclusion when applicable, creates the +persistent authoring lock and generation `1` inside it, takes EX on that lock, +fsyncs the complete tree, and publishes it with a capability-relative no-replace directory rename followed by a parent fsync. Readers therefore see -absence or a complete catalog and cannot cross the already-published lock before -the parent entry is durable. +absence or a complete catalog and cannot cross the already-published lock +before the parent entry is durable. There is no bootstrap marker or resume mode: interruption before the rename leaves `ROOT` absent, while interruption after it leaves the complete target. A diff --git a/src/catalog_lock.rs b/src/catalog_lock.rs index 5eea2370..9446143e 100644 --- a/src/catalog_lock.rs +++ b/src/catalog_lock.rs @@ -16,6 +16,7 @@ pub const LOCK_FILE: &str = "catalog-authoring.lock"; pub const APPLY_MARKER: &str = "catalog-apply-incomplete"; pub const GENERATION_FILE: &str = "catalog-generation"; pub const GENERATION_INTENT_FILE: &str = "catalog-generation-incomplete"; +const CONTROL_EXCLUDE: &str = ".st2/"; #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct CatalogReadFence(Option); @@ -205,6 +206,7 @@ impl CatalogLock { "catalog control directory is absent: {}", control.display() ); + ensure_git_control_exclusion(&catalog)?; test_control_creation_checkpoint(); let branch = match fs::create_dir(&control) { Ok(()) => { @@ -467,3 +469,102 @@ pub fn lock_path(catalog: &Path) -> PathBuf { pub fn apply_marker_path(catalog: &Path) -> PathBuf { catalog.join(CONTROL_DIR).join(APPLY_MARKER) } + +/// Add the host-local catalog control directory to the repository-local Git exclusion when the +/// catalog belongs to a Git worktree. Non-Git catalogs do not require Git. +pub(crate) fn ensure_git_control_exclusion(catalog: &Path) -> Result<()> { + use std::io::{Read as _, Seek as _, Write as _}; + use std::process::Command; + + let output = match Command::new("git") + .args(["-C"]) + .arg(catalog) + .args([ + "rev-parse", + "--is-inside-work-tree", + "--git-path", + "info/exclude", + ]) + .env_remove("GIT_DIR") + .env_remove("GIT_WORK_TREE") + .env_remove("GIT_COMMON_DIR") + .env_remove("GIT_INDEX_FILE") + .output() + { + Ok(output) => output, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + anyhow::ensure!( + !has_git_metadata_ancestor(catalog), + "catalog belongs to a Git worktree, but git is unavailable" + ); + return Ok(()); + } + Err(error) => return Err(error).context("locate catalog Git exclusion"), + }; + if !output.status.success() { + anyhow::ensure!( + !has_git_metadata_ancestor(catalog), + "cannot locate the catalog Git exclusion: {}", + String::from_utf8_lossy(&output.stderr).trim() + ); + return Ok(()); + } + + let raw = String::from_utf8(output.stdout).context("git returned a non-UTF-8 exclude path")?; + let mut lines = raw.lines(); + let inside_worktree = lines.next().context("git omitted its worktree result")?; + if inside_worktree != "true" { + return Ok(()); + } + let raw_path = lines.next().context("git omitted its exclude path")?; + anyhow::ensure!( + !raw_path.is_empty() && lines.next().is_none(), + "git returned an invalid exclude path" + ); + let path = if Path::new(raw_path).is_absolute() { + PathBuf::from(raw_path) + } else { + catalog.join(raw_path) + }; + let parent = path.parent().context("Git exclude path has no parent")?; + fs::create_dir_all(parent) + .with_context(|| format!("create Git exclude parent {}", parent.display()))?; + + let mut file = OpenOptions::new() + .create(true) + .read(true) + .write(true) + .mode(0o600) + .custom_flags(libc::O_CLOEXEC | libc::O_NOFOLLOW) + .open(&path) + .with_context(|| format!("open catalog Git exclusion {}", path.display()))?; + // SAFETY: `file` owns a valid descriptor for the duration of this function. + let result = unsafe { libc::flock(file.as_raw_fd(), libc::LOCK_EX) }; + if result != 0 { + return Err(std::io::Error::last_os_error()) + .with_context(|| format!("lock catalog Git exclusion {}", path.display())); + } + + let mut current = Vec::new(); + file.read_to_end(&mut current)?; + let already_present = current + .split(|byte| *byte == b'\n') + .any(|line| line.strip_suffix(b"\r").unwrap_or(line) == CONTROL_EXCLUDE.as_bytes()); + if already_present { + return Ok(()); + } + file.seek(std::io::SeekFrom::End(0))?; + if !current.is_empty() && !current.ends_with(b"\n") { + file.write_all(b"\n")?; + } + file.write_all(CONTROL_EXCLUDE.as_bytes())?; + file.write_all(b"\n")?; + file.sync_all() + .with_context(|| format!("sync catalog Git exclusion {}", path.display()))?; + Ok(()) +} + +fn has_git_metadata_ancestor(path: &Path) -> bool { + path.ancestors() + .any(|ancestor| fs::symlink_metadata(ancestor.join(".git")).is_ok()) +} diff --git a/src/catalog_transaction.rs b/src/catalog_transaction.rs index 0cb6afdd..8dc1ca61 100644 --- a/src/catalog_transaction.rs +++ b/src/catalog_transaction.rs @@ -1123,6 +1123,7 @@ pub fn bootstrap(request: BootstrapRequest) -> Result { ); validate_full_catalog(&stage)?; let lock = initialize_bootstrap_control(&stage)?; + crate::catalog_lock::ensure_git_control_exclusion(&parent)?; sync_tree_dirs(&stage)?; Ok(lock) })() { diff --git a/tests/catalog_apply.rs b/tests/catalog_apply.rs index c63254fe..345ee7d4 100644 --- a/tests/catalog_apply.rs +++ b/tests/catalog_apply.rs @@ -231,6 +231,38 @@ fn bootstrap_atomically_publishes_an_absent_catalog_and_replays_exactly() { ); } +#[test] +fn bootstrap_adds_the_control_directory_to_the_containing_git_exclusion() { + let temp = tempfile::tempdir().unwrap(); + let source = temp.path().join("source"); + write_agent(&source, "worker", false); + let prepared = temp.path().join("prepared"); + let captured = snapshot(&source, &prepared); + let initialized = Command::new("git") + .args(["init", "-q"]) + .arg(temp.path()) + .status() + .unwrap(); + assert!(initialized.success()); + let target = temp.path().join("target"); + + let created = bootstrap(&target, &prepared, captured["rootSha256"].as_str().unwrap()); + assert!( + created.status.success(), + "{}", + String::from_utf8_lossy(&created.stderr) + ); + let exclude = fs::read_to_string(temp.path().join(".git/info/exclude")).unwrap(); + assert_eq!(exclude.lines().filter(|line| *line == ".st2/").count(), 1); + let ignored = Command::new("git") + .args(["-C"]) + .arg(temp.path()) + .args(["check-ignore", "-q", "target/.st2/catalog-generation"]) + .status() + .unwrap(); + assert!(ignored.success()); +} + #[test] fn bootstrap_rejects_a_different_existing_catalog_without_mutation() { let temp = tempfile::tempdir().unwrap(); diff --git a/tests/catalog_selection.rs b/tests/catalog_selection.rs index 6ea66fda..b7ff61a6 100644 --- a/tests/catalog_selection.rs +++ b/tests/catalog_selection.rs @@ -170,3 +170,63 @@ fn catalog_aliases_share_the_canonical_reader_lock_domain() { assert!(catalog.join(".st2/catalog-authoring.lock").is_file()); assert!(!alias.join(".st2/catalog-authoring.lock").is_symlink()); } + +#[test] +fn catalog_initialization_adds_the_control_directory_to_the_local_git_exclusion() { + let tmp = tempfile::tempdir().unwrap(); + let catalog = tmp.path().join("catalog"); + write_agent(&catalog, "h", "worker"); + let initialized = Command::new("git") + .args(["init", "-q"]) + .arg(&catalog) + .status() + .unwrap(); + assert!(initialized.success()); + let exclude = catalog.join(".git/info/exclude"); + fs::write(&exclude, "existing-pattern").unwrap(); + + let catalog_arg = catalog.to_str().unwrap(); + for _ in 0..2 { + let output = agents(&["--catalog", catalog_arg], None, &tmp.path().join("state")); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + } + + assert_eq!( + fs::read_to_string(exclude).unwrap(), + "existing-pattern\n.st2/\n" + ); + let ignored = Command::new("git") + .args(["-C"]) + .arg(&catalog) + .args(["check-ignore", "-q", ".st2/catalog-authoring.lock"]) + .status() + .unwrap(); + assert!(ignored.success()); +} + +#[test] +fn a_copied_incomplete_marker_blocks_destination_catalog_reads() { + let tmp = tempfile::tempdir().unwrap(); + let catalog = tmp.path().join("catalog"); + write_agent(&catalog, "h", "worker"); + let catalog_arg = catalog.to_str().unwrap(); + let initialized = agents(&["--catalog", catalog_arg], None, &tmp.path().join("state")); + assert!(initialized.status.success()); + fs::write( + catalog.join(".st2/catalog-apply-incomplete"), + "copied from another host\n", + ) + .unwrap(); + + let blocked = agents(&["--catalog", catalog_arg], None, &tmp.path().join("state")); + assert!(!blocked.status.success()); + assert!( + String::from_utf8_lossy(&blocked.stderr).contains("catalog apply is incomplete"), + "{}", + String::from_utf8_lossy(&blocked.stderr) + ); +}