Skip to content

Add RSSSM: regime-switching sequential sampling models - #1000

Closed
fmuia wants to merge 587 commits into
mainfrom
implement_hmm_class
Closed

Add RSSSM: regime-switching sequential sampling models#1000
fmuia wants to merge 587 commits into
mainfrom
implement_hmm_class

Conversation

@fmuia

@fmuia fmuia commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds hssm.RSSSM, a new top-level class for fitting regime-switching sequential sampling models (issue #957). Each trial belongs to one of K hidden cognitive regimes that evolve as a Markov chain; within a regime the (rt, response) emission is a standard SSM (e.g. DDM) with regime-specific switching parameters. The discrete regimes are marginalised out by the forward algorithm (one batched pytensor.scan, contributed as a scalar pm.Potential), leaving only continuous parameters for NUTS — the same approach as the hand-built docs/tutorials/hmm_ddm_regime_switching.ipynb, but as a high-level HSSM(...)/RLSSM(...)-style class.

Implements the design on the issue_957/plan_hssm_hmm_class branch (docs/design/hssm_hmm.md), Phases 2–5.

What's included

  • Core class (src/hssm/hmm/): subpackage with rsssm.py, config.py, specs.py, ordering.py, utils.py, and the layered likelihood (likelihoods/{forward,emissions,builder}.py). Builds the PyMC model directly (bambi deferred, decision 10.1.8).
  • Generic in K and in N (participants); unbalanced panels via end-padding + emission mask.
  • Both emission backends: analytical and LAN (approx_differentiable, jax & pytensor).
  • Label-switching: ordered-transform anchor — AutoOrdering (default), OrderByParam, NoOrdering.
  • Three-mode parameter rule: scalar = shared, length-K list = fixed-per-regime, prior/switching_params = inferred-per-regime.
  • Pooling: full (default) and none (pooling="none"); P/pi0 stay global. Estimable global pi0.
  • Per-regime p_outlier lapse mixture ((1 - p_k)·SSM_k + p_k·lapse).
  • Post-hoc helpers: infer_regimes (FFBS), compute_log_likelihood (per-trial logp for arviz.loo/waic), plot_regime_recovery.
  • Docs: two executed tutorials (single- and multi-participant), API reference page, changelog entry.

Deliberate deviations from the design doc (please review)

Two change documented behavior and warrant explicit sign-off:

  1. Forward algorithm is the scaled (normalised) recursion, not the textbook un-normalised one in §5.2. The un-normalised version produces NaN gradients past ~400 trials per participant (both PyTensor and JAX), while the value stays finite — masked entirely by numpyro's init jitter. The scaled recursion is value-identical (matched to machine precision) and gradient-stable.
  2. log_likelihood group is a flat (chain, draw, __obs__) over real trials only, not (chain, draw, n_participants, n_trials) (§5.6). The rectangular shape fed padded trials to arviz.loo as logp = 0 ("perfectly predicted") observations, biasing loo/waic on unbalanced panels.

Minor/cosmetic: vi() raises NotImplementedError (deferred, not implemented as §6.3 suggested); bambi-specific inherited kwargs (link_settings, process_initvals, …) are not accepted; the tutorial regression is asserted as deterministic likelihood-level equivalence rather than draw-for-draw posterior bit-for-bit (the two parameterisations can't match draw-for-draw).

Testing

  • 76 fast tests + a slow recovery suite (K=2/K=3/angle-LAN, per-regime p_outlier, end-to-end FFBS + loo). 88% line coverage on hmm/.
  • Forward correctness vs brute-force enumeration, an independent numpy forward, and long-sequence gradient finiteness (T=400/500/1000, both backends).
  • FFBS vs exact forward-backward smoothing; sum(delta) == marginal == model Potential; unbalanced loo counts real trials only.
  • A 26-config build/gradient sweep; per-regime p_outlier recovery; LAN backends agree to 1e-15.
  • Edge cases: unoccupied/imbalanced regime (no NaN), descending anchor, LAN-backend FFBS, no-pooling fixed params, plot_regime_recovery, vi/log_likelihood/graph.

Known untested (intentional): the full (K, n_trials, n_participants) recovery grid (representative points tested instead); end-to-end sampling on an unbalanced panel (the exact masked-marginal property is tested deterministically).

Scope

v1 only. Deferred to later PRs (architecture leaves hooks): partial/hierarchical pooling, per-participant pi0, covariate-driven P, per-regime regressions, cross-emission/semi-Markov, the posterior-predictive family.

Closes #957.

🤖 Generated with Claude Code

@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 80dadd6f-28ed-4293-8ea5-3ec5e2a0b1ba

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch implement_hmm_class

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

digicosmos86 and others added 28 commits July 6, 2026 15:57
centered_vs_noncentered_basic_logic.ipynb had a markdown cell with
stray execution_count/outputs fields (invalid per notebook schema),
which broke ruff format parsing. Removed the stray fields and
reformatted with ruff format.
AlexanderFengler and others added 27 commits August 8, 2026 21:37
The mypy hook has no file filter, so `prek run --all-files` type-checks
tests/ — a surface neither CI (`mypy src/hssm`) nor pyrefly
(project-includes = ["src/hssm"]) checks. It has failed since it was
restored in 8bb7c3e, which brought back the pre-pyrefly block verbatim.

Also note pyrefly in the CI workflow table; it has run in
linting_and_type_checking.yml since 5dc465c.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* test: point ddm.onnx fixture lookup at tests/fixtures

test_lapse_distribution_cav built its ONNX path from
Path(__file__).parent / "fixtures", which stopped resolving when
dda0c52 moved the file into tests/unit/likelihoods/. Add a shared
fixture_path fixture in tests/conftest.py returning the existing
FIXTURES constant and consume it, so a future test move cannot
rebreak the path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* ci: restore the BLAS/LAPACK install in setup-env

c8cf627 dropped the apt step that installs libblas-dev and
liblapack-dev, but run_tests.yml still sets
PYTENSOR_FLAGS=blas__ldflags=... -lblas -llapack. The runner image
ships only the runtime .so.3 objects, so the linker cannot resolve
those flags and PyTensor's C backend fails to compile any BLAS-using
op -- which is what turns the slow suite red on
test_simple_models_deadline.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: drop references to the nonexistent --runslow flag

No pytest_addoption defines --runslow, so the documented command
errors out. Slow tests run by default; -m "not slow" is what
deselects them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…ainst fresh resolves (#1143)

* ci: scheduled drift detection workflow (drift.yml)

- new drift.yml: weekly lint/fast/slow + monthly notebooks re-run the
  existing gates via workflow_call against a fresh dependency resolve of
  unchanged main (no lockfile is committed, so this is the upstream/
  toolchain drift detector); failures file ONE deduped drift-labeled
  issue, green runs close it; force_fail dispatch input rehearses the
  issue path
- run_slow_tests.yml + check_notebooks.yml gain workflow_call triggers
- check_notebooks.yml: SKIP_NOTEBOOKS entries fixed to full find paths
  (the two workshop notebooks executed for months while listed as
  skipped: bare filenames never matched) and the run is split into a
  4-way shard matrix with fail-fast off, so no shard can hit the 6-hour
  job limit and a drift run reports the full blast radius
- drift fast job clears the PR-oriented pytest addopts (--exitfirst,
  --reruns) via test_args for honest triage output
- setup-env composite actions: remove the dead cache-hit guards (no
  step with id 'cache' exists; the condition always evaluated true)

* drift.yml: don't treat a cancelled run as green; scope issues:write to report

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…fail (#1153)

* cartoon: honour keep_boundary in auto height; make two tests able to fail

Follow-up to #1134. Two reviewer findings were never acted on before merge and
one was marked addressed but was not, so all three are live in main.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: require every drift band to have area, not just one

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…rects, link check (#1155)

* docs: phase-0 safety net — strict build green, CI deploy, redirects, link check (#1154)

- fix griffe docstring drift (make_distribution documented removed
  model_name/choices params; noncentered folded into the **kwargs entry
  it actually arrives through)
- move design/hssm_hmm* out of the built docs tree (internal design
  docs with broken links shipped as reachable pages)
- mkdocs.yml: site_url, validation block, mkdocs-redirects (empty map),
  not_in_nav declaration for the four phase-2 orphan targets; index.md
  LICENSE link now targets the license page
- build_docs.yml -> Docs workflow: strict build on docs/src PRs, deploy
  on release/main-push/dispatch (was manual-dispatch only), weekly
  lychee link check with .lycheeignore

* ci: read-only default token, docs-affecting path filters, narrow lycheeignore (#1154)

CodeRabbit review: workflow-level contents: read (deploy keeps its write
override); push filter now matches the PR filter incl. src/hssm/** and
pyproject.toml so docstring-only merges still redeploy; lycheeignore
narrowed to doi.org only (no twitter/x links exist in the docs).
* docs: adopt the shared ecosystem brand bundle (phase 1)

Vendored from HSSMSpine docs-brand (lnccbrown/HSSMSpine#42): unified
extra.css (notebook-output CSS moved out of the template), repo-agnostic
version-free announcement banner (was: hardcoded v0.4.0 link that went
stale each release). mkdocs.yml: favicon (was Material default),
navigation.footer + navigation.indexes for wayfinding, ecosystem footer
strip linking the sibling doc sites, social links (GitHub + the shared
Discussions forum).

* docs+ci: bundle v1.1; release flow no longer edits the version-free banner

CodeRabbit on #1156: prepare-release (workflow + skill) instructed an
edit of a version string that no longer exists in the vendored banner
template — banner steps removed, changelog remains the release's only
docs edit. Bundle v1.1: kebab-case keyframe, dead header selector
dropped, vendored-copy headers point at the spine repo. Discussions
link stays deliberately shared (one ecosystem forum, per bundle README).

* docs: finish the banner-free release scope (skill + workflow metadata)

CodeRabbit follow-up on #1156: the skill's Step-5 banner block is
removed (steps renumbered), its frontmatter and report drop the banner,
and the workflow's job/step names, commit subject, and PR checklist no
longer claim a banner edit (the prompt keeps one explanatory note that
the banner needs no release edits). Job id renamed prepare-changelog
with needs: references updated.
…nment, lapse idempotency, degeneracy warning, FFBS brute-force test
…idation, fixed-value bounds checks, registry-first choices and emission resolution, inherited-method guards, pyrefly and ruff format fixes
…ite fixed values rejected, per-regime p_outlier and LAN backend default on the config path, extra_fields rejection, notebook lint
@fmuia

fmuia commented Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #1162 — identical content on a clean history (this branch's history could not be rewritten due to repository rules blocking force-pushes).

@fmuia fmuia closed this Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HSSM-HMM Class

7 participants