Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions INVARIANTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,6 @@ materialization, messaging, DING, or presence must preserve them.
| **Retirement health** | A retired declaration is healthy only after every declared task ID is absent. Any live or dead declared task record reports incomplete retirement; retired declarations do not require presence. Live declarations retain their existing task and presence checks. | `tests/doctor.rs::retired_declaration_is_healthy_when_tasks_and_presence_are_absent`; `tests/doctor.rs::retired_declaration_is_unhealthy_while_a_declared_task_is_alive`; `tests/doctor.rs::retired_declaration_is_unhealthy_while_a_dead_task_record_remains` |
| **Crash loops surface** | A task parked by a fail-mode restart policy notifies its supervisor once over the bus. | `tests/run.rs::surface_crash_loop_notifies_the_supervisor_over_the_bus` |
| **Tracked workspaces fail closed** | Materialization simulates content operations before writing and refuses a real change to any Git-tracked target. Byte-identical tracked, untracked, and non-Git targets retain useful behavior. | `tests/materialize.rs::every_content_directive_refuses_to_change_a_tracked_target_before_any_write`; `tests/materialize.rs::byte_identical_tracked_target_is_allowed_without_modification`; `tests/materialize.rs::untracked_and_non_git_targets_remain_materializable` |
| **Catalog control state stays host-local** | Git-backed catalog creation and initialization add `.st2/` to the repository-local exclusion. Any copied incomplete marker remains authoritative and blocks destination reads. | `tests/catalog_apply.rs::bootstrap_adds_the_control_directory_to_the_containing_git_exclusion`; `tests/catalog_selection.rs::catalog_initialization_adds_the_control_directory_to_the_local_git_exclusion`; `tests/catalog_selection.rs::a_copied_incomplete_marker_blocks_destination_catalog_reads` |
| **Native flat root** | Without an authored override, catalog tasks, eval messaging, shell helpers, and DING all use the catalog itself as `ST_ROOT`; no nested bus directory is synthesized. | `src/eval_run.rs::bus_root_expands_st_root_else_defaults`; `tests/eval_run_e2e.rs::st2_eval_runs_a_benign_folder_to_a_pass_verdict`; `tests/pty.rs` |
| **Proof references resolve** | Every qualified test named in this table exists in its named source file, so stale invariant claims fail the suite instead of silently surviving a refactor. | `tests/invariants.rs::qualified_proof_references_resolve` |
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,15 @@ st2 catalog bootstrap --catalog "$NEW_CATALOG" --prepared ./prepared \
--input-sha256 <rootSha256> --json
```

A Git-backed catalog must exclude `.st2/`. st2 adds this line to the
repository-local `.git/info/exclude` when it creates or initializes the catalog
control directory. For an existing catalog, add the exact `.st2/` line before
the next sync. This directory is host-local. A copied incomplete marker blocks
catalog reads at the destination. If Git already tracks `.st2`, an exclusion
cannot untrack it. After adding the exclusion, run
`git -C "$CATALOG" rm -r --cached -- .st2` and commit that index removal. This
keeps the local control files.

`catalog apply` is policy-free. It rejects state/control content, symlinks,
unprojected workspace facts, catalog-local/default PTY roots, and effective
PTY-root changes. Bootstrap is a separate create-only declaration transaction,
Expand Down
18 changes: 12 additions & 6 deletions docs/vrs/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,8 +191,9 @@ the contract is monotonic change detection, not an exactly-once counter.

The lock file is a persistent real inode: replacing or removing it would split
the lock domain for a process that already has it open. Consequently, the first
coherent declaration reader may initialize exactly `.st2` and this lock even
when its requested operation later refuses. Refusal still performs no
coherent declaration reader may initialize `.st2`, this lock, and the
repository-local Git exclusion. A non-Git catalog does not require Git. The
requested operation may still refuse after initialization. Refusal performs no
declaration, workspace, or state mutation.

The publisher derives the destination from the captured declaration, replaces
Expand Down Expand Up @@ -332,6 +333,10 @@ files remain live while a declaration is admitted.

`<catalog>/.st2/catalog-apply-incomplete` is the durable whole-catalog
transaction fence. Any presence is authoritative, including malformed content.
The `.st2/` directory is host-local and must not enter catalog transport. A
copied incomplete marker blocks declaration reads at the destination. When the
catalog belongs to a Git worktree, st2 adds `.st2/` to the repository-local Git
exclusion during control initialization and bootstrap.
The reserved canonical record is:

```json
Expand Down Expand Up @@ -370,11 +375,12 @@ be one absent final component below an existing canonical real parent. st2
captures `DIR` through retained no-follow capabilities, verifies its declaration
root against `HEX`, admits the complete projection against logical `ROOT`, and
requires one explicit external PTY root. It materializes a 0700 sibling stage,
creates the persistent authoring lock and generation `1` inside it, takes EX on
that lock, fsyncs the complete tree, and publishes it with a capability-relative
ensures the repository-local Git exclusion when applicable, creates the
persistent authoring lock and generation `1` inside it, takes EX on that lock,
fsyncs the complete tree, and publishes it with a capability-relative
no-replace directory rename followed by a parent fsync. Readers therefore see
absence or a complete catalog and cannot cross the already-published lock before
the parent entry is durable.
absence or a complete catalog and cannot cross the already-published lock
before the parent entry is durable.

There is no bootstrap marker or resume mode: interruption before the rename
leaves `ROOT` absent, while interruption after it leaves the complete target. A
Expand Down
101 changes: 101 additions & 0 deletions src/catalog_lock.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ pub const LOCK_FILE: &str = "catalog-authoring.lock";
pub const APPLY_MARKER: &str = "catalog-apply-incomplete";
pub const GENERATION_FILE: &str = "catalog-generation";
pub const GENERATION_INTENT_FILE: &str = "catalog-generation-incomplete";
const CONTROL_EXCLUDE: &str = ".st2/";

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct CatalogReadFence(Option<u64>);
Expand Down Expand Up @@ -205,6 +206,7 @@ impl CatalogLock {
"catalog control directory is absent: {}",
control.display()
);
ensure_git_control_exclusion(&catalog)?;
test_control_creation_checkpoint();
let branch = match fs::create_dir(&control) {
Ok(()) => {
Expand Down Expand Up @@ -467,3 +469,102 @@ pub fn lock_path(catalog: &Path) -> PathBuf {
pub fn apply_marker_path(catalog: &Path) -> PathBuf {
catalog.join(CONTROL_DIR).join(APPLY_MARKER)
}

/// Add the host-local catalog control directory to the repository-local Git exclusion when the
/// catalog belongs to a Git worktree. Non-Git catalogs do not require Git.
pub(crate) fn ensure_git_control_exclusion(catalog: &Path) -> Result<()> {
use std::io::{Read as _, Seek as _, Write as _};
use std::process::Command;

let output = match Command::new("git")
.args(["-C"])
.arg(catalog)
.args([
"rev-parse",
"--is-inside-work-tree",
"--git-path",
"info/exclude",
])
.env_remove("GIT_DIR")
.env_remove("GIT_WORK_TREE")
.env_remove("GIT_COMMON_DIR")
.env_remove("GIT_INDEX_FILE")
.output()
{
Ok(output) => output,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
anyhow::ensure!(
!has_git_metadata_ancestor(catalog),
"catalog belongs to a Git worktree, but git is unavailable"
);
return Ok(());
}
Err(error) => return Err(error).context("locate catalog Git exclusion"),
};
if !output.status.success() {
anyhow::ensure!(
!has_git_metadata_ancestor(catalog),
"cannot locate the catalog Git exclusion: {}",
String::from_utf8_lossy(&output.stderr).trim()
);
return Ok(());
}

let raw = String::from_utf8(output.stdout).context("git returned a non-UTF-8 exclude path")?;
let mut lines = raw.lines();
let inside_worktree = lines.next().context("git omitted its worktree result")?;
if inside_worktree != "true" {
return Ok(());
}
let raw_path = lines.next().context("git omitted its exclude path")?;
anyhow::ensure!(
!raw_path.is_empty() && lines.next().is_none(),
"git returned an invalid exclude path"
);
let path = if Path::new(raw_path).is_absolute() {
PathBuf::from(raw_path)
} else {
catalog.join(raw_path)
};
let parent = path.parent().context("Git exclude path has no parent")?;
fs::create_dir_all(parent)
.with_context(|| format!("create Git exclude parent {}", parent.display()))?;

let mut file = OpenOptions::new()
.create(true)
.read(true)
.write(true)
.mode(0o600)
.custom_flags(libc::O_CLOEXEC | libc::O_NOFOLLOW)
.open(&path)
.with_context(|| format!("open catalog Git exclusion {}", path.display()))?;
// SAFETY: `file` owns a valid descriptor for the duration of this function.
let result = unsafe { libc::flock(file.as_raw_fd(), libc::LOCK_EX) };
if result != 0 {
return Err(std::io::Error::last_os_error())
.with_context(|| format!("lock catalog Git exclusion {}", path.display()));
}

let mut current = Vec::new();
file.read_to_end(&mut current)?;
let already_present = current
.split(|byte| *byte == b'\n')
.any(|line| line.strip_suffix(b"\r").unwrap_or(line) == CONTROL_EXCLUDE.as_bytes());
if already_present {
return Ok(());
}
file.seek(std::io::SeekFrom::End(0))?;
if !current.is_empty() && !current.ends_with(b"\n") {
file.write_all(b"\n")?;
}
file.write_all(CONTROL_EXCLUDE.as_bytes())?;
file.write_all(b"\n")?;
file.sync_all()
.with_context(|| format!("sync catalog Git exclusion {}", path.display()))?;
Ok(())
}

fn has_git_metadata_ancestor(path: &Path) -> bool {
path.ancestors()
.any(|ancestor| fs::symlink_metadata(ancestor.join(".git")).is_ok())
}
1 change: 1 addition & 0 deletions src/catalog_transaction.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1123,6 +1123,7 @@ pub fn bootstrap(request: BootstrapRequest) -> Result<BootstrapResult> {
);
validate_full_catalog(&stage)?;
let lock = initialize_bootstrap_control(&stage)?;
crate::catalog_lock::ensure_git_control_exclusion(&parent)?;
sync_tree_dirs(&stage)?;
Ok(lock)
})() {
Expand Down
32 changes: 32 additions & 0 deletions tests/catalog_apply.rs
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,38 @@ fn bootstrap_atomically_publishes_an_absent_catalog_and_replays_exactly() {
);
}

#[test]
fn bootstrap_adds_the_control_directory_to_the_containing_git_exclusion() {
let temp = tempfile::tempdir().unwrap();
let source = temp.path().join("source");
write_agent(&source, "worker", false);
let prepared = temp.path().join("prepared");
let captured = snapshot(&source, &prepared);
let initialized = Command::new("git")
.args(["init", "-q"])
.arg(temp.path())
.status()
.unwrap();
assert!(initialized.success());
let target = temp.path().join("target");

let created = bootstrap(&target, &prepared, captured["rootSha256"].as_str().unwrap());
assert!(
created.status.success(),
"{}",
String::from_utf8_lossy(&created.stderr)
);
let exclude = fs::read_to_string(temp.path().join(".git/info/exclude")).unwrap();
assert_eq!(exclude.lines().filter(|line| *line == ".st2/").count(), 1);
let ignored = Command::new("git")
.args(["-C"])
.arg(temp.path())
.args(["check-ignore", "-q", "target/.st2/catalog-generation"])
.status()
.unwrap();
assert!(ignored.success());
}

#[test]
fn bootstrap_rejects_a_different_existing_catalog_without_mutation() {
let temp = tempfile::tempdir().unwrap();
Expand Down
60 changes: 60 additions & 0 deletions tests/catalog_selection.rs
Original file line number Diff line number Diff line change
Expand Up @@ -170,3 +170,63 @@ fn catalog_aliases_share_the_canonical_reader_lock_domain() {
assert!(catalog.join(".st2/catalog-authoring.lock").is_file());
assert!(!alias.join(".st2/catalog-authoring.lock").is_symlink());
}

#[test]
fn catalog_initialization_adds_the_control_directory_to_the_local_git_exclusion() {
let tmp = tempfile::tempdir().unwrap();
let catalog = tmp.path().join("catalog");
write_agent(&catalog, "h", "worker");
let initialized = Command::new("git")
.args(["init", "-q"])
.arg(&catalog)
.status()
.unwrap();
assert!(initialized.success());
let exclude = catalog.join(".git/info/exclude");
fs::write(&exclude, "existing-pattern").unwrap();

let catalog_arg = catalog.to_str().unwrap();
for _ in 0..2 {
let output = agents(&["--catalog", catalog_arg], None, &tmp.path().join("state"));
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
}

assert_eq!(
fs::read_to_string(exclude).unwrap(),
"existing-pattern\n.st2/\n"
);
let ignored = Command::new("git")
.args(["-C"])
.arg(&catalog)
.args(["check-ignore", "-q", ".st2/catalog-authoring.lock"])
.status()
.unwrap();
assert!(ignored.success());
}

#[test]
fn a_copied_incomplete_marker_blocks_destination_catalog_reads() {
let tmp = tempfile::tempdir().unwrap();
let catalog = tmp.path().join("catalog");
write_agent(&catalog, "h", "worker");
let catalog_arg = catalog.to_str().unwrap();
let initialized = agents(&["--catalog", catalog_arg], None, &tmp.path().join("state"));
assert!(initialized.status.success());
fs::write(
catalog.join(".st2/catalog-apply-incomplete"),
"copied from another host\n",
)
.unwrap();

let blocked = agents(&["--catalog", catalog_arg], None, &tmp.path().join("state"));
assert!(!blocked.status.success());
assert!(
String::from_utf8_lossy(&blocked.stderr).contains("catalog apply is incomplete"),
"{}",
String::from_utf8_lossy(&blocked.stderr)
);
}
Loading