DynamicsBench ETF Artifact

This repository is the anonymous code and data artifact for the submitted paper on testing whether time-series forecasters learn temporal evolution. It is organized by manuscript section so reviewers can inspect the exact source code, logs, raw data, and frozen results used for each claim.

The included result directories are frozen evidence. Reproduction commands write to new output directories and refuse to overwrite existing results.

Directory map

Path	Contents
code/main_reproduction/	Main CPU reproduction code, synthetic data builders, lightweight models, paired-response evaluation, and tests.
code/deep_models/	Deep model training and evaluation code used for the breadth experiments.
code/foundation_models/	Foundation model evaluation and adaptation code.
code/lightweight_ablation/	Lightweight ablation code for context length and alpha settings.
sections/04_paired_evaluation_protocol/	Protocol notes for paired state interventions and response-pair construction.
sections/06_results_main_rq1_rq2_rq3/	Main RQ1/RQ2/RQ3 frozen results, including raw pairs, per-sample CSVs, aggregates, tables, figures, and checksums.
sections/appendix_D_extended_results/	D2 aligned small-model, deep-model, and foundation-model result evidence.
sections/appendix_JK_multiscale/	D4 intermediate and D8 scale-matched multiscale result evidence.
sections/appendix_L_ablation_tables/	Context and alpha ablation raw result tables.
sections/appendix_M_deep_fm/	Deep and foundation-model extended result evidence.
sections/appendix_rq2_mechanisms/	RQ2 mechanism-decomposition and frozen deep evidence.
sections/appendix_arithmetic_progression/	Arithmetic-progression control result evidence.
logs/	Run provenance and experiment logs retained for auditability.
reports/	Claim-to-evidence and experiment-result summaries.
docs/	Artifact-level reproducibility notes.

Setup

Use Python 3.10 or newer.

python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

For RQ2/deep-model checks, also install:

python -m pip install -r requirements-deep.txt

For foundation-model experiments, also install:

python -m pip install -r requirements-fm.txt

Quick verification

Run the lightweight unit and integration checks:

make test

Run a small end-to-end smoke reproduction of the main paired-evaluation pipeline:

make smoke

The smoke output is written to sections/06_results_main_rq1_rq2_rq3/results/artifact_smoke_check by default. Choose a new name if that directory already exists:

make smoke SMOKE_VERSION=artifact_smoke_check_2

Full main reproduction

To rerun the full CPU main reproduction:

make main-full RESULT_VERSION=artifact_full_reproduction

This creates:

sections/06_results_main_rq1_rq2_rq3/results/artifact_full_reproduction/

The main frozen result used by the paper is:

sections/06_results_main_rq1_rq2_rq3/results/revision_v2/

That directory contains raw response pairs under raw/seed_*/<system>/pairs/, per-sample CSV files under raw/, aggregate CSV files under aggregated/, paper tables under tables/, figures under figures/, and checksums under provenance/.

Optional checks

Run the RQ2 mechanism tests after installing the deep dependencies:

make rq2-test

The deep and foundation-model directories are provided for inspection and rerunning on suitable hardware. Frozen numeric evidence is already included in the corresponding sections/appendix_* directories.

Anonymity and retained data

Personal paths and local machine identifiers have been replaced with neutral placeholders such as <workspace>, <home>, <remote_workspace>, and <remote_cache>. Chinese text in copied code, logs, and result metadata has been translated or replaced with English.

The artifact intentionally excludes virtual environments, Python caches, trained checkpoints, and temporary binary model files. The retained raw evidence is the reproducibility-critical CSV, JSON, NPZ, log, table, and figure data.