Skip to content

Custom boundary is accepted and silently ignored by 13 models whose compiled engine has no boundary_fun argument #310

Description

@AlexanderFengler

Simulator(model, boundary=fn, boundary_params=[...]) builds a boundary dict and splats it into the compiled engine unconditionally. Eight cssm engines have no boundary_fun parameter, so the boundary is swallowed by **kwargs and discarded; the run completes normally using the engine's built-in boundary. 11 of the 13 affected models emit no warning at all, and for ddm_legacy/addm the returned metadata["boundary_fun_type"] reports the built-in boundary name. 97 of 113 built-in models honour a custom boundary correctly, and default usage (no boundary override) is unaffected.

Environment: ssm-simulators 0.13.2, Python 3.12, macOS (Darwin 25.4.0), n_threads=1.

Reproducer

"""ssms 0.13.2 -- custom boundary silently ignored by several compiled engines."""

import inspect

import numpy as np

from ssms.basic_simulators.simulator_class import Simulator
from ssms.config import get_model_registry


# Collapses to the 0.01 floor within ~100 ms. Any engine that honours it
# must return near-instant RTs.
def hard_collapse(t, a=1.0):
    return np.maximum(a * np.exp(-50.0 * np.asarray(t)), 0.01)


def mean_rt(model, theta, **kw):
    out = Simulator(model, **kw).simulate(theta=theta, n_samples=2000, max_t=10.0)
    rts = np.asarray(out["rts"]).ravel()
    return float(rts[rts != -999.0].mean())


BND = dict(boundary=hard_collapse, boundary_params=["a"])
ddm_th = {"v": 0.5, "a": 1.5, "z": 0.5, "t": 0.0}
lba_th = {"v0": 0.5, "v1": 0.3, "v2": 0.2, "a": 1.5, "z": 0.2, "theta": 0.0}

print("ddm         default %.3f -> hard_collapse %.3f   (honoured)"
      % (mean_rt("ddm", ddm_th), mean_rt("ddm", ddm_th, **BND)))
print("ddm_legacy  default %.3f -> hard_collapse %.3f   (IGNORED)"
      % (mean_rt("ddm_legacy", ddm_th), mean_rt("ddm_legacy", ddm_th, **BND)))
print("lba_angle_3 default %.3f -> hard_collapse %.3f   (IGNORED)"
      % (mean_rt("lba_angle_3", lba_th), mean_rt("lba_angle_3", lba_th, **BND)))
print("lba_angle_3 control: same model, a=1.5 -> a=0.3 gives %.3f "
      "(engine does react to 'a', just not to boundary_fun)"
      % mean_rt("lba_angle_3", {**lba_th, "a": 0.3}))

# ---- sweep every built-in model -------------------------------------------
ignored = []
for m in sorted(get_model_registry().list_models()):
    cfg = get_model_registry().get(m)
    th = dict(zip(cfg["params"], map(float, cfg["default_params"])))
    base, coll = mean_rt(m, th), mean_rt(m, th, **BND)
    if not (base - coll) / base > 0.10:
        ignored.append(m)
print("\nboundary silently ignored by %d/%d built-in models:" % (
    len(ignored), len(get_model_registry().list_models())))
print(ignored)

print("\nengine signatures:")
for f in ["ddm_flexbound", "ddm", "addm", "lba_vanilla", "lba_angle",
          "rlwm_lba_pw_v1", "rlwm_lba_race", "poisson_race",
          "racing_diffusion_model"]:
    import cssm
    print("  cssm.%-24s boundary_fun in signature: %s"
          % (f, "boundary_fun" in inspect.signature(getattr(cssm, f)).parameters))

cfg = get_model_registry().get("lba_angle_3")
print("\nlba_angle_3 config: boundary_name=%r boundary=%r boundary_params=%r"
      % (cfg["boundary_name"], cfg["boundary"].__name__, cfg.get("boundary_params")))

Output

ddm         default 1.889 -> hard_collapse 0.046   (honoured)
ddm_legacy  default 0.556 -> hard_collapse 0.562   (IGNORED)
lba_angle_3 default 2.863 -> hard_collapse 2.872   (IGNORED)
lba_angle_3 control: same model, a=1.5 -> a=0.3 gives 0.379 (engine does react to 'a', just not to boundary_fun)
/…/site-packages/ssms/basic_simulators/simulator_class.py:651: UserWarning: Callable boundary expects parameters ['a'] but none were found in theta (keys: ['beta', 'q0', 'q1', 'deadline', 's']). The boundary function may receive missing arguments.
  boundary_dict = make_boundary_dict(model_config_local, theta)
/…/site-packages/ssms/basic_simulators/simulator_class.py:651: UserWarning: Callable boundary expects parameters ['a'] but none were found in theta (keys: ['beta', 'q0', 'q1', 'q2', 'deadline', 's']). The boundary function may receive missing arguments.
  boundary_dict = make_boundary_dict(model_config_local, theta)
/…/site-packages/ssms/basic_simulators/simulator_class.py:651: UserWarning: Callable boundary expects parameters ['a'] but none were found in theta (keys: ['beta', 'q0', 'q1', 'q2', 'q3', 'deadline', 's']). The boundary function may receive missing arguments.
  boundary_dict = make_boundary_dict(model_config_local, theta)
/…/site-packages/ssms/basic_simulators/simulator_class.py:651: UserWarning: Callable boundary expects parameters ['a'] but none were found in theta (keys: ['r1', 'r2', 'k1', 'k2', 't', 'deadline', 's', 'r', 'k']). The boundary function may receive missing arguments.
  boundary_dict = make_boundary_dict(model_config_local, theta)
/…/site-packages/ssms/basic_simulators/simulator_class.py:651: UserWarning: Callable boundary expects parameters ['a'] but none were found in theta (keys: ['v0', 'v1', 'v2', 'A', 'b', 't', 'deadline', 's', 'v']). The boundary function may receive missing arguments.
  boundary_dict = make_boundary_dict(model_config_local, theta)

boundary silently ignored by 16/113 built-in models:
['addm', 'ddm_legacy', 'dev_rlwm_lba_pw_v1', 'dev_rlwm_lba_race_v1', 'dev_rlwm_lba_race_v2', 'inv_temp_softmax_2', 'inv_temp_softmax_3', 'inv_temp_softmax_4', 'lba2', 'lba3', 'lba4', 'lba_3_vs_constraint', 'lba_angle_3', 'lba_angle_3_vs_constraint', 'poisson_race', 'racing_diffusion_3']

engine signatures:
  cssm.ddm_flexbound            boundary_fun in signature: True
  cssm.ddm                      boundary_fun in signature: False
  cssm.addm                     boundary_fun in signature: False
  cssm.lba_vanilla              boundary_fun in signature: False
  cssm.lba_angle                boundary_fun in signature: False
  cssm.rlwm_lba_pw_v1           boundary_fun in signature: False
  cssm.rlwm_lba_race            boundary_fun in signature: False
  cssm.poisson_race             boundary_fun in signature: True
  cssm.racing_diffusion_model   boundary_fun in signature: False

lba_angle_3 config: boundary_name='constant' boundary='constant' boundary_params=None

(Only the venv path prefix is elided above. inv_temp_softmax_2/3/4 are choice-only simulators that return PLACEHOLDER_RT = -1.0 for every trial — ssms/basic_simulators/inv_temp_softmax.py:10 — so a flat mean RT there is trivially true; that takes the meaningful count from 16 to 13.)

Warning behaviour, checked separately with warnings.simplefilter("always") and n_samples=200:

addm                         warnings=NONE     metadata boundary_fun_type='addm_collapse'
ddm_legacy                   warnings=NONE     metadata boundary_fun_type='constant'
lba2 / lba3 / lba4           warnings=NONE     metadata boundary_fun_type='<absent>'
lba_3_vs_constraint          warnings=NONE     metadata boundary_fun_type='<absent>'
lba_angle_3                  warnings=NONE     metadata boundary_fun_type='<absent>'
lba_angle_3_vs_constraint    warnings=NONE     metadata boundary_fun_type='<absent>'
dev_rlwm_lba_pw_v1           warnings=NONE     metadata boundary_fun_type='<absent>'
dev_rlwm_lba_race_v1/v2      warnings=NONE     metadata boundary_fun_type='<absent>'
poisson_race                 warnings=["Callable boundary expects parameters ['a'] but none were fou…"]
racing_diffusion_3           warnings=["Callable boundary expects parameters ['a'] but none were fou…"]

Why

Python layer builds and forwards the boundary without checking that the engine accepts one:

  • ssms/basic_simulators/simulator_class.py:388-400_apply_custom_boundary sets boundary/boundary_name/boundary_params with no engine-capability check.
  • ssms/basic_simulators/simulator_class.py:651-667simulate() calls make_boundary_dict and splats the result into model_config_local["simulator"](**theta, **boundary_dict, ...).
  • ssms/basic_simulators/simulator.py:285-339, 782-788 — same construction and unconditional splat on the functional path.
  • ssms/basic_simulators/simulator_class.py:321-358_infer_simulator_requirements already computes supports_boundary = "boundary_fun" in params, but it is never called anywhere in the package (grep -rn _infer_simulator_requirements ssms/ returns only the def line).

Engines with no boundary_fun parameter, so it lands in **kwargs and is never read:

  • cssm/ddm_models.pyx:349-363ddm (model ddm_legacy); line 561 hardcodes boundary_fun_name='constant' into metadata.
  • cssm/addm_models.pyx:320-338addm; line 459 hardcodes boundary_fun_name='addm_collapse'.
  • cssm/lba_models.pyx:36-46, 144-156, 253-266, 384-396lba_vanilla, lba_angle, rlwm_lba_pw_v1, rlwm_lba_race. grep -c boundary_fun cssm/lba_models.pyx0.
  • cssm/race_models.pyx:917-931racing_diffusion_model.
  • cssm/poisson_race_models.pyx:59 — declares boundary_fun = None, # unused, kept for interface compatibility, so signature introspection gives a false positive here.

Contrast: cssm/ddm_models.pyx:589-606 (ddm_flexbound) declares boundary_fun/boundary_params and calls compute_boundary (cssm/_utils.pyx:655-666).

Suggested fix

Refuse loudly in the Python layer rather than plumbing a boundary the engine cannot use. Wire up the existing dead code in _apply_custom_boundary (and the equivalent for drift):

reqs = self._infer_simulator_requirements(config["simulator"])
if not reqs["supports_boundary"]:
    raise ValueError(
        f"Model {config['name']!r} uses simulator "
        f"{config['simulator'].__name__!r}, which computes its boundary "
        "internally and cannot accept a custom boundary function."
    )

Signature introspection alone is not enough because of poisson_race, so prefer an explicit "supports_custom_boundary": bool on each model config (or a module-level set of engines that genuinely honour it), falling back to introspection when absent. The engines to mark False are cssm.ddm, cssm.addm, cssm.lba_vanilla, cssm.lba_angle, cssm.rlwm_lba_pw_v1, cssm.rlwm_lba_race, cssm.racing_diffusion_model, cssm.poisson_race (plus cssm.full_ddm_hddm_base, ddm_models.pyx:55, which no registry model currently routes to). Put the guard in or immediately after make_boundary_dict so simulator.py:771 is covered too. The sweep above is directly reusable as a parametrised regression test over get_model_registry().list_models(): each model must either shorten mean RT measurably under a collapsing bound or raise.

Is this intended?

Partly, and the issue concedes it: these engines deliberately compute their own boundary (lba_angle derives a linear collapse from theta; addm computes ±(a - b·t) internally; poisson_race:59 says so in a comment). The complaint is not that they lack a general boundary hook — it is that the Python layer accepts, validates and forwards a boundary to them with no check, when the check already exists as dead code.

Two things the report explicitly does not claim: the value of a is not dropped (make_boundary_dict does route it into boundary_params, and a is also passed independently — the a=1.5 → 0.3 control moves mean RT 2.863 → 0.379); and "no warning at all" is not universally true, since 5 of the 16 do warn — but incidentally, with the text "expects parameters ['a'] but none were found in theta", which is a missing-parameter complaint, not "this boundary will be ignored".

Unrelated observation while measuring: the LBA engines appear to ignore random_state (repeat simulate(..., random_state=42) calls give mean RTs differing by ~0.3%). That wobble is why the ignored rows above are not bit-identical; the honouring models move 60-98% over the same comparison, well clear of it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions