Teams want to catch a model regression before it ships. Add a reusable GitHub Action (and an example
workflow) that runs a chosen suite, compares the result to a stored baseline, and fails the job on a
significant_regression. Build on the existing CI harness (tjbench/ci_benchmark.py).
To see the full picture, install both tokenjam and tokenjam-bench: TokenJam surfaces the candidate to
try (Analyze), and bench proves whether it holds in CI before you switch (Prove).
Acceptance criteria: a documented reusable action + example workflow; runs a suite, diffs against a
baseline, exits non-zero on significant regression; honest verdict surfaced in the job summary.
Teams want to catch a model regression before it ships. Add a reusable GitHub Action (and an example
workflow) that runs a chosen suite, compares the result to a stored baseline, and fails the job on a
significant_regression. Build on the existing CI harness (tjbench/ci_benchmark.py).To see the full picture, install both
tokenjamandtokenjam-bench: TokenJam surfaces the candidate totry (Analyze), and bench proves whether it holds in CI before you switch (Prove).
Acceptance criteria: a documented reusable action + example workflow; runs a suite, diffs against a
baseline, exits non-zero on significant regression; honest verdict surfaced in the job summary.