A structure to compound experience will make the file parser an old archaeology practice.
We are not sure that sentence is finished yet. OpenContext is a small prototype on Databricks — a workshop where we try to find what that structure might look like.
For you, the intent is simple: grow a private mind while you chat and upload; when something is worth passing on, publish a path others can walk and build on — without handing them your diary, and without erasing what you published first.
We do not have a finished answer.
What does a shareable knowledge structure look like — and what lets the next person build on it without replacing what came first?
A foundation model does not keep what you learned in the last conversation. The useful part is often momentary: where you looked, what you skipped, what finally clicked. That lives in the session and then disappears.
Sharing a PDF or a table is hard enough. Sharing that — the shape of attention, not the file — is harder still.
Most knowledge apps still start the same way: upload, chunk, embed, hope. That recovers text from artifacts. It does not carry forward how someone learned to walk the problem. We wonder if experience can compound as structure, so the next person does not re-parse the past so much as walk a path someone already learned how to walk.
Knowledge sharing ≠ data sharing.
Data sharing moves tables, volumes, credentials. Knowledge is messier: judgment, attention, “look here first,” “don’t go there,” “this edge matters more than that chunk.”
Roughly, in order:
- Discover structure — chat, uploads, and background jobs help you grow a private mind. You learn what matters by using it.
- Publish a path — export a thin pack: how attention should walk (
walk_rules), a small shared spine of concepts, and consent for others to resolve it when they ask. - Subscribe and build on — keep your private mind. Optionally let someone else’s path fire alongside yours. Create over their structure and publish your own — a fork in judgment, not a rewrite of their notebook.
Create → Publish path → Subscribe → Create over → Publish again → …
A pack is not the end of the story. It is the handoff that lets the next person walk, grow their own mind, and mint a new path — compounding judgment across people while each private mind stays private.
The first published path does not have to change when someone builds on it. A pack is not a ZIP of chats, private memory, or PDF bytes. It is closer to a constitution of attention over shared speech — bound to ground, revisable, independent of which model answers next.
| We try to share | We try not to share |
|---|---|
How attention should walk (walk_rules) |
Raw chat turns |
| A small legend / spine of concepts | Private memory and preferences |
| Consent to resolve a path at answer time | File dumps and table clones |
Subscribing is permission to walk a path when you ask — not a copy of someone else’s lake. Full protocol: docs/knowledge-sharing-protocol.md.
OpenContext is a personal RAG chat that can optionally publish and subscribe to paths:
| Tab | What it is for |
|---|---|
| Ask | Talk with your private mind; optionally union paths you subscribed to |
| Input | Upload, sync, learn — grow the mind in the background |
| Publish | Name a path and share the id (not your diary) |
| Share view | See what a pack exposes before you trust it |
| Learning | A slower tour of how the pieces fit together |
Two tempos keep chat snappy:
- Clock A (reply) — answer now; never wait on learning jobs
- Clock B (learn) — parse, chunk, promote, publish — in the background
Your private mind stays yours. Publishing is opt-in.
This is development software (catalog default kg_agentic_rag): chat, uploads, MindGraph, publish and subscribe, clean and unpublish, early walk_rules and clearing signals. Enough to poke at the thesis; we are still finding the structure.
More detail: docs/architecture.md · setup: docs/setup.md · security: docs/security.md
One place for fast chat (Lakebase), governed tables (Unity Catalog), background jobs, and models — without stitching five services. The App answers; Jobs learn in the background; your identity is bound on the server, not in the browser.
git clone <this-repo>
cd knowledge-graph-agentic-rag
uv sync
cp .env.example .env
# Names only in .env — never tokens or SP UUIDs in git.
databricks auth login
chmod +x scripts/kg
./scripts/kg check
./scripts/kg build
databricks bundle deploy -t dev
databricks bundle run kg_agentic_rag -t devSecrets stay in ~/.databrickscfg. First-time UC, Lakebase, Vector Search, and grants: ./scripts/kg and platform/uc/.
app/ FastAPI + static UI
frontend/ React · Vite
jobs/ Clock B work
resources/ Bundle jobs and app bindings
platform/ UC DDL and grants
scripts/kg check · build · grant helpers
docs/ Knowledge-sharing protocol
Not a bigger PDF parser — though file ingest stays useful when you need the artifact.
We hope to find a structure where experience becomes architecture (paths, walk rules, clearing), judgment compounds without copying everyone’s files, and the hardest part — sharing what the model never retained from the moment — becomes at least possible to express.
If you have ideas, critiques, or a path worth walking, issues and pull requests are welcome. We are building toward the structure; we do not claim to own it yet.
OpenContext application code is MIT.
The vendored runtime helper vendor/databricks-tools-core/ is from Databricks AI Dev Kit and is subject to the Databricks License (use with Databricks Services only). See THIRD_PARTY_NOTICES.md.