A flexible framework for collaborative AI world model analysis and tuning.
Composable, shareable latent-space analysis mutually intelligible to humans and agents.
Status: pre-MVP, working toward an initial alpha release — active development, APIs subject to change. This README is written especially for upstream maintainers and prospective collaborators: it summarizes what interpretune is, the constructs it's built on, and the roadmap from here to alpha.
Interpretune is a (pre-MVP) AI world model analysis framework. It strives to provide composable, shareable latent-space analysis mutually intelligible to humans and agents.
By granting a wide range of interpretability methods and packages access to composable, shareable analysis operations and state it accelerates novel, collaborative, world model analysis and tuning with PyTorch.
Both humans and agents can inspect and refine the mechanistic and causal faithfulness of model reasoning at mutually intelligible levels of abstraction. Core framework goals:
- transparent, causally faithful reasoning
- augmented model self-reflection
- world-model-guided collaborative tuning
Interpretune composes adapters at multiple levels of abstraction over a shared session/protocol layer:
- the framework level (core PyTorch, Lightning)
- the interpretability latent-model level (TransformerLens, NNsight)
- the analysis level (circuit-tracer, SAELens)
This composition pattern is what lets researchers collaborate across interpretability frameworks:
analysis operations (e.g. extract_top_features, gradient_attribution, ablation_attribution,
feature_intervention, graph_prune, concept_direction, compute_attribution_graph) are
written once and executed across backends, with results captured as shareable datasets.
A note on terminology: we use "world model" in the epistemic/semantic sense — the internal representations, concepts, and beliefs a model encodes about the world, as studied in AI interpretability — related to but distinct from the predictive visual world models of embodied-agent/model-based-RL research. The initial MVP focuses on LLMs; fuller multimodal support is planned.
| Construct | What it is |
|---|---|
ITSession |
Composes a datamodule + module with an ordered adapter context (e.g. (core, transformer_lens, circuit_tracer) or (core, nnsight, circuit_tracer)); the result is an "interpretunable" module whose capabilities are the composition of the selected adapters. |
| Protocols | Structural contracts (interpretune.protocol) that make the same analysis code executable across adapter compositions. |
| Latent-model abstractions | Analysis targets are latent models at an appropriate abstraction level — SAEs/transcoders today, but the protocol language (LatentAnalysisTargets, latent-model handles, ops like model_fwd_w_cache_latent_models) deliberately generalizes beyond SAEs. |
| Analysis ops | Composable, schema-declared operations (it.concept_direction, it.compute_attribution_graph, it.intervention_from_concept, …) compiled into execution plans and dispatched over the active backends. |
AnalysisStore |
The key sharing abstraction: serialized, schema-described datasets capturing analysis artifacts (activations, attributions, graphs, intervention results) — making world-model analyses exchangeable datasets rather than one-off notebook state. |
| Extensions | Cross-cutting capabilities: memory profiling, debug generation, Neuronpedia dashboard/explanation integration. |
A minimal flavor of the composition pattern (see the
docs and src/it_examples/notebooks/ demos for
runnable versions):
import interpretune as it
from it_examples.example_module_registry import MODULE_EXAMPLE_REGISTRY
dm_cfg, m_cfg, dm_cls, m_cls = MODULE_EXAMPLE_REGISTRY.get("gemma2.rte_demo.circuit_tracer")
session = it.ITSession(it.ITSessionConfig(
adapter_ctx=(it.Adapter.core, it.Adapter.nnsight, it.Adapter.circuit_tracer),
datamodule_cfg=dm_cfg, module_cfg=m_cfg, datamodule_cls=dm_cls, module_cls=m_cls,
))
it.it_init(**session)
# the same op runs unchanged under the transformer_lens adapter context
result = it.intervention_from_concept(session.module, ...)Interpretune uses uv for fast, reliable dependency management.
# Install uv (one-time setup)
curl -LsSf https://astral.sh/uv/install.sh | sh
# Clone the repository
git clone https://github.com/speediedan/interpretune.git && cd interpretune
# Set preferred venv base directory and name and create out-of-tree venv (recommended to enable multiple parallel envs)
export VENV_BASE=${HOME}/.venvs
export IT_TARGET_VENV=it_latest
uv venv ${VENV_BASE}/${IT_TARGET_VENV} --python 3.13 && source ${VENV_BASE}/${IT_TARGET_VENV}/bin/activate
# Install the package with all necessary development dependencies
# Note: the git-deps group is optional once circuit-tracer is published on PyPI
uv pip install -e ".[test,examples,lightning,profiling]" --group git-deps dev
# Optional: only if you will run the Neuronpedia dashboard local-DB import/benchmark lanes —
# installs the pinned neuronpedia-utils (--no-deps: its unused autointerp/cloud dependency
# chain conflicts with the integrated env; the runtime deps it needs here are already included)
uv pip install --no-deps -r requirements/ci/nodeps_git_requirements.txtFor advanced development builds (locked CI requirements, multi-repo from-source composition), use the build script:
./scripts/build_it_env.sh --repo_home=${PWD} --target_env_name=it_latest
# from-source example:
./scripts/build_it_env.sh --repo_home=${PWD} --target_env_name=it_latest \
--from-source="circuit_tracer:${HOME}/repos/circuit-tracer"A coordinated multi-repo PR set upstreaming scalable dashboard generation validated end-to-end with Interpretune: peak-memory-controlled generation with per-stage CUDA accounting and pretokenization records (SAEDashboard), supporting surfaces in SAELens, the circuit-tracer attribution-target abstraction, and Neuronpedia import/coverage support — with the dashboard benchmark suite in this repo providing quantified evidence. (Coordination PR link forthcoming.)
On ownership: the upstream repos — principally Neuronpedia, together with SAEDashboard — own the core dashboard APIs and generation protocol and will continue to own them; Interpretune provides one example generation/import orchestration pipeline demonstrating the improved scalability, example-aligned customizability, and (soon) direct shareability/streamability.
The MVP milestone centers on making Interpretune artifacts shareable on the Hub — most importantly AnalysisStore upload/download (analyses as exchangeable, reproducible, composable datasets), adapter/session shareability (an analysis is runnable, not just readable), and cross-backend support hardening (#201, #224). This is the same hub-artifact pattern as the streamable-dashboards effort — one consistent story for dashboards, explanations, and analysis results.
Dashboards and locally-generated feature explanations become Hugging Face Hub artifacts that viewers stream on demand, optionally disintermediating the Neuronpedia DB for user-generated dashboards (on neuronpedia.org and local dev stacks alike) — same upstream-ownership framing as above.
RTE cross-backend research (#220), Jacobian-space (J-lens) analysis support (#225), self-interpretability (interpretability that accelerates model advancement via self-reflection, including internal reflection heuristics for more sample-efficient RL exploration), epistemic-coherence objectives, reflective cognition for counterfactuals via latent-state evaluation, meta-latent world-model interfaces, and multimodal world-model support — see the roadmap docs.
- Latest docs — concepts, usage guides, design notes, API reference
- Demo notebooks:
src/it_examples/notebooks/(circuit-tracer analysis + concept-steering demos, SAE-Lens and NNsight adapter examples, with Colab badges on published copies)
# Standard suite
python -m pytest src/interpretune tests -v
# Special phases (standalone GPU, profiling)
./tests/special_tests.sh --mark_type=standalone
./tests/special_tests.sh --mark_type=profile_ciCI runs the full matrix (Linux/macOS/Windows CPU + a self-hosted GPU pipeline with
CUDA/standalone/profiling phases); coverage is tracked on
codecov. See tests/README.md for the testing
guide and requirements/utils/lock_ci_requirements.sh for CI-locked requirements regeneration.
Contributors are enthusiastically welcomed — Interpretune is built to be a
community effort, and the transition from pre-MVP to alpha is the perfect moment to get involved.
The IT-MVP milestone is the best place
to find priority items and good first issues, the
issue tracker tags documentation and
help wanted items, and the roadmap above orients where things are headed. Thank you
in advance for contributing to the open-source interpretability ecosystem — every issue, doc fix,
and PR genuinely helps.