Fuzzy file finding and indexed content search for pi — fast when a native index is available, and working when it is not.
Part of the Pify suite. Install with pify install search or pi install npm:@pify/search.
pi's find and grep spawn a process and read the tree on every call. That is the right design for a one-shot command and the wrong one for an agent, which asks over and over inside a single session. An index built once and kept current answers the fiftieth question as fast as the first.
The other half is the shape of the question. find wants a glob; people want "the auth route file — you know the one". Fuzzy, typo-tolerant matching with results ranked by what you have actually been working on answers that; a glob does not.
| Parameter | Type | Notes |
|---|---|---|
query |
string | Part of a name or path; typos tolerated |
limit |
number, optional | Per page, default 20 |
cursor |
string, optional | From a previous call |
worktre entr finds worktree/src/enter.ts. Results are ranked: an exact filename beats a matching stem, which beats a prefix, which beats a directory that merely contains the query — and recently edited or git-modified files rise, because that is what you are probably looking for.
| Parameter | Type | Notes |
|---|---|---|
pattern |
string | What to search for |
mode |
literal | regex | fuzzy |
Default literal |
caseSensitive |
boolean, optional | Default false |
limit / cursor |
Paging, default 20 per page |
Three modes because three different questions get asked: the exact string, a shape, and "something like this" for when you do not know how it is spelled.
native — this package's own Rust core, native/, built with napi. It indexes this suite (417 files) in about 40ms cold, and a literal search then reads the 5 files the trigram index says could match rather than all 417. Searches come back in about a millisecond. Its index is stored between sessions, so the second start does not pay for the first.
fff — @ff-labs/fff-node, used when it is installed and the native core is not. A mature engine with its own watcher and git integration.
builtin — pure TypeScript, no dependencies, no binary. A trigram index for content, a fuzzy scorer for paths, an fs.watch subscription to stay current.
The fallback is the point. A native binary is a promise you cannot always keep: an unsupported platform, a locked-down install, a blocked postinstall — any of those, and a binary-only search extension is one that silently does nothing.
All three are checked against each other on a real tree (test/live/engines.mjs, 33/33): the same files found, the same literal matches, the same refusal to search node_modules, cursors that advance rather than repeat — and native and builtin rank identically, because they share their scoring constants on purpose. Losing the binary should change how fast a search is, never how it is ordered. /search says which engine is live.
npm run build:native # cargo build --release --manifest-path native/Cargo.tomlThe result is picked up automatically from native/target/release/. CI builds and smoke-tests six targets — win32 x64/arm64, darwin x64/arm64, linux x64/arm64 — on every tag.
Honest status: only win32-x64 has been built and verified by hand; the other five are proven by CI and nothing more. Per-platform npm packages (@pify/search-<triple>) are not published yet, so an installed copy of this package uses fff if you have it and the TypeScript engine otherwise. The loader already looks for them, so publishing is additive.
Every overlapping three-byte window of every text file is a trigram, packed into one number and mapped to the files containing it. A search extracts the trigrams its pattern must contain and intersects those posting lists, so only files that could match are ever read. (The design is tgrep's, which reports up to 52× over ripgrep on very large trees.)
The index only ever narrows; every surviving candidate is still matched for real, so a wrong candidate costs time and never correctness. The rule that makes it safe: a pattern with nothing indexable — \d+, a fuzzy query, one branch of an alternation that could match anywhere — reports "no candidate set is safe" and everything is read. Confusing that with "nothing matched" is how an index starts silently hiding results, so the two are different values throughout.
An index rebuilt at every start is one you pay for at every start. The native engine writes its index to disk and, next time, reloads it and reconciles instead of re-reading the tree.
Measured on a synthetic 20,000-file tree (node native/bench.mjs 20000):
| build | files read | grep | |
|---|---|---|---|
| cold (no stored index) | 1320ms | 20,000 | <1ms |
| warm (unchanged tree) | 188ms | 0 | <1ms |
7×, and the cost that remains is the directory walk, not the files. Editing one file re-reads one file.
Correctness rests on a single rule: a stored entry is trusted only while its size and mtime still match what is on disk. Anything changed, new, or vanished is re-read before a query can see it. A cache that answers confidently for a file that has moved on is worse than no cache. The reload path is checked against the rebuild path on every supported platform (native/persist.mjs, run in CI) — same totals, same lines, same order — including that a corrupt or truncated index is discarded rather than half-trusted.
Trigram lists are stored as varint deltas, which is what makes this worth doing at all: 881KB for this suite, ~2.2KB per file, about a third of the naive encoding. Reading a cache that is larger than the sources it summarises costs more than the rebuild it was meant to avoid.
The index lives in the platform cache directory — %LOCALAPPDATA%, ~/Library/Caches, $XDG_CACHE_HOME — keyed by a hash of the absolute root, never inside your working tree.
| variable | effect |
|---|---|
PIFY_SEARCH_NO_CACHE=1 |
never store an index; rebuild every start |
PIFY_SEARCH_CACHE_DIR |
store indexes somewhere else |
PIFY_SEARCH_TIMING=1 |
print how long the walk, the reload and the inversion each took |
PIFY_SEARCH_ENGINE |
builtin or fff to force an engine |
Frecency decays on a three-day half-life — an agent session is shorter and more concentrated than a human's week, so yesterday's file should not outrank today's. Every read, edit or write in the session counts as an access. History is capped at seven days and 128 timestamps per file, so the store cannot grow without bound.
This package adds two tools; it does not replace find, grep or multi_grep. Replacing them would put every search in the session behind whichever engine happened to load, and a fallback that is slower than the thing it replaced is not an improvement anyone asked for. Use fffind/ffgrep when a search is worth an index; the built-ins are still there when it is not.
/search — which engine is running, the indexed root, how many files it holds, and how much of the index came back from the stored copy rather than from disk.
MIT © Pify maintainers