Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ compatibility guard is pinned by `compat_global_config_schema_future`.
| sparse-view | intentionally-different | Read-only sparse VIEW filter (lore.md 2.2, Libra-only extension) — the NON-declined complement of git sparse-checkout (the materializing `sparse-checkout` command and `clone --sparse` stay declined, D10). `sparse-view set|add|list|enable|disable|clear|status` stores an allowlist of gitignore-syntax include patterns (`!pat` carves holes; allowlist last-match-wins, no ancestor-dominance) that scope what `ls-files` and the WORKING-TREE `diff` DISPLAY. It NEVER mutates the working tree, writes no skip-worktree bits, and NEVER filters the changes-to-be-committed set: `status` content is unfiltered (only a one-line advisory) and `diff --staged` / `diff A..B` are unfiltered, so nothing ever hides what a commit will record. Unmerged/conflicted entries are always shown. Patterns in the `sparse_view` SQLite table (owner `internal::sparse`), toggle in the per-worktree `sparse_view_meta` projection (the scope-less config_kv `sparse.enabled` key was retired by migration 2026072304); worktree-scoped since W1 (§C.4.1.1 — patterns and the toggle are per-worktree facts, legacy state adopts to main only absent linked worktrees, else the migration fails closed); disabled/empty = no-op (byte-identical output). `--json` on list/status. Deferred: cone mode, materialization |
| alternates | intentionally-different | Object alternates (lore.md 2.3, Libra extension) — borrow objects from a shared/parent object store instead of copying. `alternates add|list|remove|prune` registers an object dir in the git-standard `objects/info/alternates` file (`prune` is the base-side retirement of borrower registrations whose repository is gone — never automatic, because an absent path is indistinguishable from an unmounted one); reads resolve through the chain (transitive, cycle-safe, depth-capped) on a LOCAL miss, FULL-BYTE OID-verified before a borrowed hit is returned (a tampered alternate can't poison a read). `exist` consults alternates so a borrowed-but-present object is not treated as missing. DELETION SAFETY (airtight): registering a base ALSO records this repo as a BORROWER (`objects/info/borrowers`); while any live borrower exists the base's `gc` and `cache evict` REFUSE to prune loose objects — a shared base can never delete a borrowed object. Guards: refuses a self-reference, a base with a different `core.objectformat`, and a TIERED (s3/r2) base (a local alternate can't reach its remote tier). `file obliterate` refuses a borrow-only object (never reaches into a parent's store); `fsck` reports a dangling alternate. Deferred: `clone --reference` copy-avoidance (needs fetch have-negotiation — the flags stay accepted no-ops), `--dissociate`, and 2.11 default shared-store |
| deps | intentionally-different | File dependency graph (lore.md 3.1, Libra extension — Git has no file-dependency concept). `deps add|rm|list|why|tree` declares/queries typed, VERSIONED per-file edges `(from -> to, kind)`. Edges are stored per-commit under the reserved notes ref `refs/notes/deps` (owner `internal::deps::DependencyStore`, the sole reader/writer — mirrors the `refs/notes/metadata` pattern; no new SQLite table). Queries: direct deps / dependents (`--reverse`), transitive closure (`tree`, cycle-safe iterative BFS with a `--depth-limit`), and `why` (shortest dependency path). Paths are repo-relative, normalized (`./` stripped, `\`→`/`), and reject absolute / `..`-escape / empty. `--revision` targets a commit (default HEAD); `--json` on reads; add is idempotent; a fresh/empty graph reads empty (absence-tolerant). The `transitive_closure` API is the reusable seam for 3.2 (dependency-filtered clone/sync) and 3.3 (hydrating VFS). Cross-machine edge travel is delivered by 3.2: `fetch`/`pull --notes` imports `refs/notes/deps` from a LOCAL Libra source over a dedicated side-channel (a deps note is a loose blob + a SQLite `notes` row, not a commit-reachable object, so it does not ride the pack; import union-merges and re-validates every endpoint, and is per-note fault-tolerant), default OFF (Git parity). Deferred: network / foreign-Git / push-side notes travel (D17, see [_compatibility.md](docs/development/commands/_compatibility.md)), carry-forward onto new commits, rename-following, and automatic dependency inference (v1 edges are author-declared) |
| media | intentionally-different | FastCDC LFS media chunking CLIENT (lore.md §6, **feature-gated `fastcdc`, default OFF** — Libra media extension, Git has no equivalent; absent from the default binary). The honest v1 client substrate with NO live media server: `media chunk` content-defines chunks of a file (in-tree deterministic gear-hash FastCDC, frozen `fastcdc-v1` params) and builds a versioned manifest; `--store` persists chunks + manifest to a private `.libra/media/` store that is a physical SIBLING of `objects/` (a chunk is NEVER a Git object ID, §6.2). `media inspect` validates a manifest; `media verify` reassembles from the store and verifies the full `media_oid` before publishing (never a corrupt file). `media probe` capability-probes a remote (`libra/media/v1/capabilities`, §0.2 backoff) and reports the §6.4 transfer decision — chunked vs a SAFE standard-LFS fallback (no endpoint / disabled / incompatible / unknown-higher-version / 5xx-after-backoff → standard LFS; server-refuses-fallback + no-local-fallback → BLOCK, never a chunk-only half-write). `media_oid` is ALWAYS SHA-256 of the full content (independent of `core.objectformat`), byte-identical to a standard LFS pointer OID. `--json` on all. DEFERRED / FROZEN (the Libra-aware media SERVER, lore.md §6.5–6.8): real cross-machine chunked upload/download, capability + chunk + manifest-finalize endpoints, the Pending→Finalized lifecycle + GC/fsck/heal, every §6.7 anti-side-channel guarantee, and chunk-only repo policy — so against every reachable remote today the probe falls back to standard Git LFS |
| media | intentionally-different | Feature-gated FastCDC extension (`fastcdc`, default OFF): chunk, inspect, verify, probe; versioned manifests and raw SHA-256 chunks outside the Git object graph. Feature-enabled LFS transfers use Mega's authenticated user/repository-scoped media protocol for missing-chunk uploads, verified finalize with full standard-LFS fallback, and cached-chunk downloads with atomic destination replacement. Frozen in-tree `fastcdc-v1` parameters; `media_oid` is always full-content SHA-256. Other remotes use standard LFS; `lfs.fastcdc=false` disables the extension. Shared repository ACLs, server orphan GC/quota/fsck/heal/obliteration and range hydration remain deferred. See `docs/commands/media.md`. |
| hydrate | intentionally-different | On-demand whole-object hydration (lore.md 3.3, Libra extension — the honest v1 of Lore's "hydrating VFS": an EXPLICIT `libra hydrate <path>...` command, NOT a transparent FUSE-on-access filesystem, which stays a `worktree-fuse` follow-up). Materializes a path's content (and, by default, its transitive forward deps via 3.1's closure — `--no-deps`/`--depth-limit` bound it) into the working tree, resolving each blob local → alternate (2.3) → remote durable tier; the read policy is honored (`--offline`/`--local` refuse a remote fetch). FAILURE-RECOVERY contract: each blob is OID-verified on a borrowed/remote hit (`--verify` also re-hashes the local path, healing on mismatch) and published via an atomic temp-write + rename, so a hydration that fails for ANY reason (missing object, unreachable remote, verify mismatch, interruption) leaves the pre-existing file UNTOUCHED — never a truncated/half-written file. An active sparse view (2.2) gates the FULL set (roots AND deps); out-of-view paths are refused unless `--ignore-sparse`. Already-present (byte-identical) content is a no-op skip; `--dry-run` writes nothing; `--fail-fast`/`--json`. Whole-object only (NO FastCDC byte-range). Cross-machine dep expansion now works once the graph is fetched via 3.2's `fetch`/`pull --notes` (local Libra source; network deferred, D17). Deferred: LFS-pointer blobs (their download is not yet atomic — skipped cleanly), symlink/gitlink entries, and transparent FUSE on-access hydration |
| logfile | intentionally-different | Diagnostic for the tracing log-file sink (`logfile info`): reports the env-resolved path, rolling strategy (`LIBRA_LOG_ROTATION`), filter, and size; also enables `tracing-appender` time-rolled logs. Git has no equivalent (mirrors Lore's `logfile`); needs no repository. `--json`/`--machine` → `{ enabled, file, rotation, filter, size_bytes, file_count }`. Rotation splits logs by time (each file bounded); it does not prune old files, so total disk use needs external retention. Exit 0 |
| completions | intentionally-different | Shell completion generator (`bash`/`zsh`/`fish`/`powershell`/`elvish`) built from the live clap command tree; Git ships completions via `contrib/completion` rather than a `git completions` subcommand. Prints the script to stdout (`--json`/`--machine` wraps it as `{ shell, script }`); needs no repository. Exit 0 / 129 (unknown or missing shell, Git-style usage error) |
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ test-live-cloud = [] # L3: tests hitting real D1/R2 endpoints
test-provider = [] # deterministic hidden provider for local Code UI automation tests
test-live-agent = [] # plan-20260713 live agent gate: real local claude/codex/opencode CLI data (dev machine acceptance; env LIBRA_RUN_LIVE_AGENT_GATE=1)
test-upgrade = [] # plan-20260714 §A.11: auto-upgrade test hooks (trust-root/endpoint injection; requires LIBRA_TEST=1 at runtime; release workflow must NEVER enable this feature)
fastcdc = [] # lore.md §6: default-OFF FastCDC LFS media chunking client substrate (no new deps; reuses ring sha256; server protocol frozen)
fastcdc = [] # default-OFF FastCDC LFS transport (in-tree; reuses ring SHA-256)
subagent-scaffold = [] # CEX-S2-10 schema-only scaffold (Step 2 sub-agent contracts; gated on CP-4 in production — see docs/development/tracing/agent.md "Step 2 audit closure")
otlp = ["dep:opentelemetry", "dep:opentelemetry_sdk", "dep:opentelemetry-otlp", "dep:tracing-opentelemetry"] # lore.md 1.7: OTLP trace export (default binary unaffected)
keyring = ["dep:keyring"] # lore.md 2.7: OS-keyring auth backend (release builds enable it; default dev builds unaffected)
Expand Down
19 changes: 19 additions & 0 deletions docs/commands/lfs.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,25 @@ lock API.

Unlike Git, which requires a separate `git-lfs` extension installed as a smudge/clean filter, Libra integrates LFS natively. The LFS client, pointer file parsing, and attribute management are built into the `libra` binary. No additional installation or filter configuration is needed.

## Optional FastCDC transfers

Build both Libra and Mega (`mono`) with `--features fastcdc` to enable the optional
media transport, and save a host-scoped Mono Bearer access token with
`libra auth login`. Both builds default to feature OFF. The LFS URL retains the
repository path (`<repo>.git/info/lfs`); capability discovery uses
`libra/media/v1/capabilities` below that URL. Set `libra config lfs.fastcdc false`
to disable the extension in a repository.

Host-only HTTP remotes retain the legacy root LFS endpoints (for example,
`http://localhost:8000/locks`). Use a repository path for scoped FastCDC transfers.

Normal LFS uploads send missing chunks and finalize a verified full object;
downloads reuse verified local chunks. Standard LFS pointers remain unchanged,
and unsupported remotes or missing manifests use full-object LFS. Mega isolates
the extension by authenticated user and repository path; this is not a complete
repository ACL implementation. See [`libra media`](media.md) for the transfer
flow and current retention, quota, and maintenance limitations.

## Options

`libra lfs` has no top-level options. All functionality is accessed through subcommands documented below.
Expand Down
84 changes: 71 additions & 13 deletions docs/commands/media.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,22 +26,80 @@ OID.

## Safe fallback

`media probe` reports one of: `chunked (fastcdc-v1)` (a fully compatible
Libra-aware media server), `standard-lfs (fallback)` with a reason (no capability
endpoint, disabled by server, incompatible algorithm, disabled by repo policy,
unknown higher version, or a server error after backoff), or `blocked` (the
server keeps no standard fallback object AND no local complete object exists — a
chunk-only upload is refused rather than silently produced). Against every
reachable remote today — none of which run the (frozen) Libra media server — the
decision is a standard Git LFS fallback.
`media probe` reports the remote's capabilities: `chunked (fastcdc-v1)` or
`standard-lfs (fallback)` with a reason such as no capability endpoint, disabled
server support, incompatible algorithm, insufficient required capabilities,
unknown protocol version, or a server error after backoff. It assumes the
repository permits chunking and a complete local fallback object is available;
it does **not** read `lfs.fastcdc` and does not report `blocked` under these
assumptions. A `chunked` probe result therefore does not prove that transfers are
enabled in this repository.

Actual LFS transfers also apply `lfs.fastcdc` and require the server to retain a
complete standard-LFS fallback and accept manifests. Chunk-only advertisements
use basic LFS instead. Mega built with `--features fastcdc` implements the
authenticated extension; other remotes retain the standard Git LFS fallback.

## Live LFS transfers with Mega

Build Libra with `cargo build --features fastcdc` and, in the Mega repository,
build/start the HTTP server with
`cargo run -p mono --features fastcdc -- service http` using its normal server
configuration. Both builds default to feature OFF. The `libra` commands below
must use the feature-built binary (`target/debug/libra`, or `libra.exe` on
Windows); compiling does not replace a separately installed binary on PATH.

Obtain a **Mono-issued access token** through Mega's existing authenticated
token-creation flow (`POST /api/v1/user/token/generate`). `libra auth login`
only stores that token locally; it does not issue a Mega token. A GitHub PAT or
browser session cookie is not a substitute for the Mono access token.

For a local Mega HTTP server on port 8000, run in the Libra repository:

```bash
libra config remote.origin.url http://localhost:8000/project/demo.git
libra auth login --host http://localhost:8000
# Paste the Mono access token at the hidden prompt.
libra auth status --host http://localhost:8000
libra config lfs.fastcdc true
libra media probe --remote origin
```

After compiling the feature, an unset `lfs.fastcdc` permits automatic negotiation;
`true` explicitly enables it and `false` disables it in that repository. The
stored token must match the remote's **host and port**. Use HTTPS for non-loopback
servers (for example `--host https://mega.example.com:8443`); HTTP token attachment
is allowed only for loopback. Pass only the origin to `--host`, without the
repository path, and do not put tokens in URLs. For scripts, feed the token on
stdin with `--with-token`; see [`libra auth`](auth.md).

Keep the repository URL in `origin`. The LFS client preserves
`<repo>.git/info/lfs`; capability discovery appends `libra/media/v1/capabilities`
to that LFS URL. The Bearer header is attached automatically from the stored token.

Normal LFS push/upload now prepares a versioned manifest, uploads only missing
chunks, and finalizes. Mega verifies chunk hashes, full SHA-256 and the frozen
FastCDC boundaries, writes the complete standard-LFS object, then publishes the
manifest. Repeating push resumes from missing chunks. Downloads use finalized
manifests, reuse verified local chunks, and atomically publish only verified full
content. Invalid manifests or corrupted remote chunks are errors and preserve the
existing destination. No manifest, unsupported capabilities or disabled feature
means standard full-object LFS. Objects exceeding the negotiated manifest/chunk
count limits also use basic upload before any manifest is sent. Chunk-only uploads
are not supported. Outside a Libra repository, the public LFS download client uses
basic LFS instead of creating a repository cache.

Mega's initial extension isolates chunks by authenticated user and repository;
another user's data is fetched through the standard full-object fallback. It
requires Bearer access tokens and does not introduce a public chunk-hash API.
Manifests are limited to 10 MiB / 8192 chunks and chunks to 8 MiB. This is an
opt-in transport; deployments need explicit retention and quota planning.

## Deferred

The Libra-aware media **server** (real cross-machine chunked upload/download,
capability + chunk + manifest-finalize endpoints, the manifest lifecycle,
GC/fsck/heal, and every anti-side-channel guarantee) is frozen in lore.md §6.5–6.8
and not part of this client v1. Chunk-only repo policy (dropping the standard LFS
fallback object) and range-based hydration are also deferred.
Shared repository ACLs, automatic orphan GC, quota accounting, server fsck/heal,
obliteration, chunk-only policy and byte-range hydration remain deferred. The
current implementation does not claim completion of all Lore §6.5–6.8 guarantees.

## Examples

Expand Down
Loading