Skip to content

jcode-works/jcode-ragmir

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

173 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Ragmir

npm version npm downloads CI Node.js AGPL-3.0

Confidential local RAG for coding agents, scripts, and Node.js applications.

Ragmir turns specifications, Word files, PDFs, spreadsheets, code, and local exports into cited evidence indexed and retrieved on your machine. Core works offline by default, never uploads your corpus, and calls no model. Connect the agent or automation you already use through CLI, MCP, or a typed TypeScript API, or keep the complete workflow local with the optional Chat package.

Website · npm · Documentation · CLI · API · Releases

Start in 60 seconds

The fastest path is to let your coding agent inspect the repository and tailor the setup.

Option 1: paste this into your coding agent
Set up Ragmir in this repository. Work interactively: inspect first, infer safe defaults, present a proposal, wait for approval, then execute. Never assume consent for dependency changes, model downloads, replacing skills, or sharing data.

Outcome: Core installed with the repository's package manager; useful sources selected; secrets and generated noise excluded; tools connected; cited retrieval verified. Semantic retrieval, team features, Chat, and TTS are optional.

1. Inspect without changes:
- Find the repository or monorepo root. Read package.json packageManager, lockfiles, workspace and Node/version-manager files, .gitignore, existing .ragmir state, README, AGENTS/CLAUDE/CODEX guidance, docs, specs/ADRs, apps/packages, important config, source, and tests.
- Detect Node 22+ and pnpm, npm, Yarn, or Bun. Prefer packageManager, then the lockfile. Respect workspace-root flags and mise/asdf/Volta. Never create a second lockfile. If signals conflict, ask.
- If Ragmir exists, inspect its version, config, status, sources, and rgr upgrade --check before changing it.

2. Propose one setup summary, then ask once:
- Infer the owning base and useful clients from the repository. State any nested bases you propose.
- Default to offline local-hash and Core only, or optional Chat only when requested. Optional TTS stays off unless requested. Semantic, Chat, and TTS downloads require explicit approval; Edge text transfer requires separate approval.
- Default to solo unless the repository or request shows a team workflow. For a Git-backed team, propose the current upstream as authority and safe automatic pulls; offer --no-pull when Git updates must stay manual.
- List selected source globs, exclusions, any external/private folder, and the exact package, config, skill, and download actions you would perform.
- Ask only about unresolved choices that materially change source authority, data exposure, downloads, or external execution. Wait for one approval covering the proposal.

3. Implement after approval:
- Install @jcode.labs/ragmir as a dev dependency with the detected manager. Install Chat/TTS only if selected, at a compatible version.
- Run the matching rgr setup --no-ingest --agents <selected> command. Keep project scope. If a same-name skill is unmanaged, show the diff and ask before --force.
- Build a narrow .ragmir/config.json. Prefer stable relative globs for root guidance, docs/specs/ADRs, package READMEs/manifests, useful app config, and source/tests that explain behavior. Include locales only when useful.
- Exclude .env*, credentials, keys, unapproved dumps/customer data, dependencies, generated/build/cache/coverage/log folders, vendored code, binaries/media, and .ragmir storage/models. In monorepos, keep nested bases scoped and shared knowledge at root.
- Run preview and audit --unsupported before ingest. Review redactions, unsupported/oversized files, duplicates, chunks, and sensitive paths. Fix config first, then ingest.
- For an existing install, use rgr upgrade and doctor --fix as indicated. Never delete the active index first. Rebuild only for incompatible embedding, chunk, or index-policy changes.
- Enable semantic retrieval, preload Chat, or preload TTS only after consent. Use non-sensitive TTS preload text.
- For Git teams, run rgr team sync. It safely pulls and ingests; --no-pull keeps Git manual. Snapshots are advanced diagnostics.

4. Prove the result:
- Run rgr doctor --deep, rgr audit --unsupported, and rgr security-audit.
- Run representative searches with citations and --explain. Create a small local golden suite for project questions and run rgr evaluate; do not weaken gates to pass.
- Report detected tools, answers, packages, downloads, config/sources/exclusions, changed files, readiness, retrieval results, team status, and exact remaining actions.

Never commit .ragmir, corpus files, models, snapshots, logs, or secrets. Never claim offline, semantic, team synchronization, or retrieval quality without evidence.

Prefer manual setup? Ragmir requires Node.js 22 or later. Install it in the project that owns the files to search:

pnpm add -D @jcode.labs/ragmir
pnpm exec rgr setup --agents codex,claude,kimi,opencode,cline
pnpm exec rgr sources add "README.md" "docs/**/*.md"
pnpm exec rgr ingest
pnpm exec rgr search "Which decision changed the rollout?"

Using npm? Replace pnpm add -D with npm install --save-dev and pnpm exec with npx. At a pnpm workspace root, use pnpm add -Dw.

rgr setup creates ignored local state under .ragmir/ and connects the selected agents. Ingestion is incremental and resumable, so running the same command again continues from durable progress. Use rgr doctor to confirm readiness, then ask an agent:

Use Ragmir to find which decision changed the rollout. Cite every claim and expand the strongest
citation before proposing an edit.

Why Ragmir

Strength What it gives you
Local by default Corpus, index, reports, and metadata-only logs stay under ignored .ragmir/ state
Verifiable evidence File and chunk references, source lines, PDF pages, PPTX slides, XLSX cells, and EPUB positions when the format supports them
Model-agnostic Core Cited retrieval for any compatible agent, script, CLI, MCP client, or TypeScript application
Production-grade ingestion Bounded memory and concurrency, per-file durable progress, atomic rebuild activation, and local writer serialization
Inspectable retrieval Hybrid ranking explanations, explicit evidence thresholds, stable tie-breaks, and exact vector diagnostics
Open source AGPL-3.0-only packages, commercial licensing for proprietary use, and no hosted Ragmir account

The default local-hash provider needs no model download and does not load Transformers.js, ONNX Runtime, or Sharp. Semantic Transformers.js embeddings are an explicit option. Below 100,000 rows, vector search remains exhaustive; larger tables use a quality-gated IVF-PQ policy with complete coverage. Separate bounded queues protect search, embedding, and ingestion from unbounded work.

The repository includes deterministic citation, retrieval-quality, scale, storage, parser, observability, startup, and runtime benchmarks. See the benchmark guide for claim-eligible runs and comparison rules.

How it works

flowchart LR
    A["Project files you choose"] --> B["Extract and redact"]
    B --> C["Chunk and index locally"]
    C --> D["Retrieve bounded evidence"]
    D --> E["CLI"]
    D --> F["TypeScript"]
    D --> G["MCP agent"]
    D -. optional .-> H["Local Chat or audio"]
    B -. blank PDF pages .-> I["Configured local OCR"]
Loading

Project paths resolve from the caller's working directory or explicit configuration, never from the installed package. Ragmir opens no HTTP port. A network-facing application owns transport security, authentication, authorization, and rate limits.

Choose an interface

Interface Best for Result
rgr CLI Setup, ingest, search, audit, and maintenance Human-readable or JSON output
TypeScript API Scripts and long-running Node.js workers Typed results and explicit lifecycle
Local MCP server Coding agents and compatible clients Up to three compact citations by default, with targeted expansion
Ragmir Chat Fully local answer generation Answers grounded in verified cited passages
Ragmir TTS Reviewed text briefs Local WAV or explicit online MP3

Core remains retrieval-first: ask() returns cited context, not a generated answer. Chat and TTS are optional packages loaded only when their commands are used.

Common workflows

Agents and monorepos

Generated MCP helpers pin the project root so an agent queries the intended index. In a monorepo, Ragmir selects the nearest .ragmir/config.json; root and package bases stay isolated. MCP search, ask, and research are compact by default. Agents expand one selected citation instead of loading every full passage; research may add up to three code matches, and compact: false remains available for an explicit full response.

pnpm exec rgr bases
pnpm exec rgr --project-root apps/web search "checkout contract"

Read the agent integration guide for native helpers, MCP tools, resource budgets, and monorepo routing.

Portable knowledge-base folders

Export one frozen, relocatable folder when another machine, agent, or automation needs the same cited evidence without the original source tree:

pnpm exec rgr portable export
pnpm exec rgr portable export --output ../operations-knowledge --name "Operations knowledge"
pnpm exec rgr portable export --output ../operations-knowledge --replace

The folder contains the active index, any required local embedding model, an embedded read-only runtime, two retrieval and decision-evidence skills, MCP templates, and a SHA-256 inventory. It excludes raw source files and access logs, but the indexed passages remain sensitive. Move the folder to a Node.js 22 host with the platform recorded in manifest.json, run node bin/rgr.cjs portable verify . --json, then generate a destination-specific configuration with node bin/configure.cjs generic or a native Claude, Codex, Kimi, OpenCode, or Cline target. No package-manager install or registry access is needed after transfer. The inventory detects changes but does not authenticate who published the folder.

For a stable destination, --replace builds and verifies the new export before switching the path. The previous bundle is renamed to a timestamped sibling and returned as previousOutputDir; restart long-running consumers, verify the new bundle, then retire that backup under the operator's retention policy. An unrelated existing directory is never replaced.

OpenClaw, Hermes, n8n, and custom hosts use the same MCP stdio or argument-array CLI contract when their runtime supports it. Ragmir supplies cited evidence, not permission to perform an external action. The host keeps authentication, tool permissions, approval rules, and network security. See the portable knowledge-base guide.

Teams

flowchart LR
    A["Merge into the declared upstream"] --> B["rgr team sync"]
    B --> C["Safe Git fast-forward"]
    C --> D["Incremental local ingest"]
    D --> E["Ready private index"]
Loading

Git-backed teams build one private index per developer. The current branch upstream is the only declared authority: Git carries the reviewed change, Ragmir refreshes the local evidence.

pnpm exec rgr team sync
  1. Open and review a pull request (or merge request).
  2. Merge it into the declared upstream.
  3. Run rgr team sync on each workstation that needs fresh evidence.

Ragmir fast-forwards only a clean non-divergent branch and reindexes changed sources incrementally. Otherwise it leaves history and the valid index alone, then returns one action. --no-pull keeps branch updates manual and --check previews. Snapshots are an advanced fallback for non-Git sources or exact drift analysis, including existing v2.19 snapshots. Detailed safeguards live in the team guide and configuration reference.

Audit and explain retrieval

pnpm exec rgr preview --path docs --max-chunks 3
pnpm exec rgr audit --unsupported
pnpm exec rgr security-audit
pnpm exec rgr search "release approval" --explain
pnpm exec rgr research "release obligations" --compact --timeout-ms 10000

preview inspects redacted chunks without writing an index. audit compares sources with indexed state. research combines bounded query variants with deterministic cross-query ranking. Use rgr doctor --deep only when you need a live O(corpus) inventory; normal status and doctor checks read the compact activation manifest.

Safe upgrades

pnpm exec rgr upgrade --check
pnpm exec rgr upgrade

Run this after updating the package and before the first retrieval with the new runtime. upgrade --check only explains whether the current index can be reused. upgrade refreshes local agent helpers and stages any required rebuild in a separate generation. It never deletes the active index first: only a validated replacement activates, and interrupted or failed rebuilds remain resumable. A long-running host can keep its already loaded runtime serving the previous generation, then restart or cut over after status=current and ready=true. Privacy or extractor warnings are reported separately as advisory lines and do not mislabel an operational index as needing repair; resolve them with rgr security-audit without discarding the working index.

Semantic retrieval and scanned PDFs

pnpm exec rgr setup --semantic
pnpm exec rgr ingest --rebuild

pnpm exec rgr ocr setup --engine auto
pnpm exec rgr ingest --rebuild

Semantic mode uses explicitly prepared local model weights. OCR prefers embedded PDF text and runs only for blank pages through a configured local executable, with bounded batches and a private resumable cache. Ragmir has no cloud OCR integration.

TypeScript API

import { createRagmirClient, isRagmirError } from "@jcode.labs/ragmir"

const ragmir = await createRagmirClient({ cwd: process.cwd() })
try {
  await ragmir.ingest({ timeoutMs: 120_000 })
  const results = await ragmir.search("Which decision changed the rollout?", {
    topK: 5,
    explain: true,
    timeoutMs: 10_000,
  })

  for (const result of results) console.log(result.citation, result.text)
} catch (error) {
  if (isRagmirError(error)) console.error(error.code, error.retryable)
  else throw error
} finally {
  await ragmir.close()
}

Reuse one client per project root in a long-running process and close it during shutdown. One-shot ingest, search, ask, and research functions remain available for short scripts. The API reference documents setup, preview, evaluation, privacy, OCR, MCP, source routing, lifecycle, cancellation, and timeout options.

Supported content

Ragmir handles source code, text, Markdown, configuration, logs, CSV, JSON, JSONL, YAML, PDF, DOCX, PPTX, XLSX, OpenDocument, EPUB, HTML, RTF, email, notebooks, and project-configured text extensions. Run rgr audit --unsupported for the exact files skipped and why. Ragmir does not claim universal binary support.

Privacy boundaries

Capability Default Explicit network boundary
Core retrieval Local files, index, and local-hash retrieval None required
Preferred agent or automation Receives only passages selected by the integration The consumer's data policy applies
Semantic embeddings Disabled Model download is explicit; inference can then stay local
OCR Disabled Local executable only
Ragmir Chat Optional local GGUF inference Selected profile is downloaded and verified during setup
Ragmir TTS Optional local WAV Edge MP3 sends narration text only when explicitly selected

Redaction reduces accidental exposure but is not a compliance certification. Review security hardening before indexing sensitive material.

Packages

Package Purpose
@jcode.labs/ragmir Core CLI, TypeScript API, MCP server, OCR, and agent helpers
@jcode.labs/ragmir-chat Optional cited local answer generation
@jcode.labs/ragmir-tts Optional local audio and explicit online speech

Installing Core does not install Chat or TTS.

Licensing

Ragmir v3.0.0 and later are available under GNU AGPL v3.0 only. Organizations that want to integrate, redistribute, modify, or operate Ragmir under proprietary terms can request a separate commercial license from JCode Works.

Releases before v3.0.0 keep the license published with those releases. Third-party dependencies and model files keep their own licenses.

Examples and documentation

Resource Use it for
Quick start Agent-guided or manual repository setup
Confidential local RAG demo Complete fictional CLI workflow
Library API demo Persistent TypeScript client pattern
Document evidence benchmark Exact path, line, chunk, and PDF-page expectations
CLI reference Commands, options, and JSON output
Configuration Sources, privacy, workloads, embeddings, and extractors
Agent integration Native helpers, MCP, monorepos, and teams
Portable knowledge bases Frozen folders for agents, automations, and servers
Troubleshooting Readiness, weak search, OCR, and add-on diagnostics
Release history Curated highlights, verification, artifacts, and upgrade links

Every committed example uses fictional data. Keep private corpora and generated reports outside Git or under ignored local state.

Quality and compatibility

Releases are gated on Node.js 22 for Linux x64 and macOS ARM64. pnpm validate runs formatting and lint, dependency audit, type checks, coverage, builds, public API and MCP smoke tests, package validation, semantic-release checks, and signed release-artifact generation. CI adds the portable offline installation matrix and CodeQL analysis.

See CONTRIBUTING.md, RELEASING.md, the AGPL-3.0-only license, and the commercial licensing option.