Bench runs fully offline using mock directives embedded in prompts (# task_key:, # echo:, # answer:,
# wrong_answer:, # unsafe_tool:, etc.), stripped before any live call. This is how contributors write
deterministic offline tests, but it isn't documented in one place.
Document the directive format in docs/development.md: list each directive, what it does, escaping rules,
and a short runnable example. Source of truth: the stripping logic in tjbench/models/openai_compatible.py
and the mock clients in tjbench/models/mock_client.py / mock_agent_client.py.
Acceptance criteria: a directive table + one runnable offline example in docs/development.md.
Bench runs fully offline using mock directives embedded in prompts (
# task_key:,# echo:,# answer:,# wrong_answer:,# unsafe_tool:, etc.), stripped before any live call. This is how contributors writedeterministic offline tests, but it isn't documented in one place.
Document the directive format in
docs/development.md: list each directive, what it does, escaping rules,and a short runnable example. Source of truth: the stripping logic in
tjbench/models/openai_compatible.pyand the mock clients in
tjbench/models/mock_client.py/mock_agent_client.py.Acceptance criteria: a directive table + one runnable offline example in
docs/development.md.