Stop ranking queues on payload size; publish CPU instead

.grok/skills/implement-experiment/SKILL.md

@@ -40,7 +40,7 @@ export PATH="${HOME}/.local/go/bin:${HOME}/.cargo/bin:${HOME}/.dotnet:${HOME}/.l
 2. **One question per folder.** Number it `NN-short-kebab` (next free `NN` after existing dirs).
 3. **Edit `experiment.yaml` only** for sample, languages, libraries, run mode, grouping cut-offs. `run.yaml` is generated. Do not keep a second library list.
 4. **Shared sample** at the experiment root (`sample.json`). Language folders hold **results** only (`results.md`, `results.json`). **Do not commit experiment logs** (CSVs under `<lang>/logs/`). Those stay on the machine that ran the timing. Saved results are enough.
-5. **Do not compare write times across languages.** Size is the only roughly fair cross-language number, and only when both sides write the same field description.
+5. **Do not compare times across languages.** Different runtimes. Directional only. Do not invent a size contest — payload bytes are the **data type**, not a library score.
 6. **Do not crown a single winner.** Use `top_group` (similar / close / slower via Cliff’s delta vs the fastest library in the comparison set). Not “top 5%.”
 7. **Textbook language** in `experiment.yaml` (`story.example`, `story.tradeoff`, `story.why`), `README.md`, `results.md`, the Dashboard, and the PLAN update. Write for a high-school student: complete sentences, no slang, no telegraphic fragments, no unexplained jargon. The Dashboard copies `story` from `experiment.yaml`.
 8. **Do not overwrite** published site tables (`docs/<lang>/results.md`) or dashboard `*_latest.json.gz` unless the user asked to publish suite numbers.

.grok/skills/improve-docs/SKILL.md

@@ -87,7 +87,7 @@ Cut scope creep, ensure dismissible UI is not permanent noise, prefer small diff
 - Edit sources under `dashboard/` then **`npm run build`** in `dashboard/` so `docs/dashboard/` updates.
 - **README:** follow `README_EDITING.md` — short lede, compact audience table, section order Who → languages → Try it → Quick start; no slogan/meta CTA prose; no unsolicited role expansion.
 - MkDocs storefront (`docs/index.md`) only if asked or if it clearly contradicts a user-approved README fact.
-- Follow `STYLE.md` (simple, no new frameworks, terminology: **data type** not fixture).
+- Follow `STYLE.md` (simple, no new frameworks). **Terminology: always “data type”, never “fixture”** — in user copy, comments, identifiers, and JSON keys.
 
 ### 2.5 Critique the implementation
 

.grok/skills/improve-docs/references/STYLE.md

@@ -2,6 +2,16 @@
 
 Keep it simple. Prefer delete and clarify over decorate.
 
+## Terminology (binding)
+
+| Say | Never say |
+|-----|-----------|
+| **data type** (`message`, `document`, `telemetry`, `strings`, `event`) | fixture, fixtures, fixtureKey, `dataset.fixtures` |
+
+This is the catalog payload shape. Use **data type** in README, Dashboard copy, skills, comments, function names, and new JSON keys. Do not keep `fixture*` as an internal alias. Old `configs.json` may still contain `fixtures`; read it as a fallback, write `data_types`.
+
+Do not measure or display payload **size** as a library result. Size is the data type, not a score.
+
 ---
 
 ## Scope of “docs” in this skill
@@ -18,8 +28,9 @@ Keep it simple. Prefer delete and clarify over decorate.
 ## Content style
 
 - **One idea per paragraph.** Prefer tables for role/path matrices **on the site / Dashboard**, not by fattening the root README.
-- **User terms:** data type, mode (bytes/stream), ops/s, latency, median size, Pareto, baseline.
-- **Avoid in user copy:** fixture (internal OK), harness (prefer benchmark runner), unexplained IQR/P95.
+- **User terms:** data type, mode (bytes/stream), ops/s, latency, Pareto, baseline.
+- **Never say “fixture”.** The catalog entry is a **data type** (`message`, `document`, …). Say it in docs, Dashboard copy, skills, comments, identifiers, and JSON keys. Do not keep `fixture*` as an internal alias “until later.”
+- **Avoid in user copy:** fixture, median size (we do not measure payload size as a result), harness (prefer benchmark runner), unexplained IQR/P95.
 - **Honesty line** when ranks appear: within one language; cross-lang directional — prefer **one** place (e.g. Statistics / Method), not a second essay block on README.
 - **Links:** prefer site paths that match MkDocs nav labels (Dashboard, Learn, Method). Avoid “storefront” / “CTA” wording in user-facing labels.
 - **No emoji spam** in product UI; README badges OK.

README.md

@@ -131,9 +131,9 @@ Every language writes the same columns (nanoseconds). Domain mapping:
 | `TimeDeq` | Dequeue ns |
 | `TimeHandoff` | Handoff ns |
 | `Pattern` | `bytes` = **SPSC**, `stream` = **MPMC** (not I/O) |
-| `Size` | Payload bytes |
+| `CpuTimeNs` | Process CPU time (spin vs block) |
 
-See [architecture](https://leo-gan.github.io/queue-benchmark/analysis/architecture/).
+See [Metrics](https://leo-gan.github.io/queue-benchmark/analysis/METRICS/) and [architecture](https://leo-gan.github.io/queue-benchmark/analysis/architecture/).
 
 ---
 

analysis/src/benchmark_analysis/abi.py

@@ -20,7 +20,6 @@
     "LibraryVersion",
     "TimeEnq",
     "TimeDeq",
-    "Size",
     "TimeHandoff",
     "OpPerSecEnq",
     "OpPerSecDeq",
@@ -29,8 +28,6 @@
     "FidelityScore",
     "DataTypeInstanceCount",
     "TypeConfigHash",
-    "SizeGzip",
-    "SizeZstd",
     "NativeKind",
     "StreamMode",
     "RunOrder",
@@ -133,7 +130,6 @@ def canonicalize_csv_record(row: Mapping[str, Any], language: str = "") -> Dict[
         "LibraryName": str(pick(row, "LibraryName", "") or ""),
         "TimeEnq": _as_int(pick(row, "TimeEnq", 0)),
         "TimeDeq": _as_int(pick(row, "TimeDeq", 0)),
-        "Size": _as_int(row.get("Size", 0)),
         "TimeHandoff": _as_int(pick(row, "TimeHandoff", 0)),
         "OpPerSecEnq": _as_float(pick(row, "OpPerSecEnq", 0)),
         "OpPerSecDeq": _as_float(pick(row, "OpPerSecDeq", 0)),
@@ -146,8 +142,6 @@ def canonicalize_csv_record(row: Mapping[str, Any], language: str = "") -> Dict[
         "MemoryPeakBytes",
         "CpuTimeNs",
         "DataTypeInstanceCount",
-        "SizeGzip",
-        "SizeZstd",
         "RunOrder",
         "SchedulePosition",
     )

analysis/src/benchmark_analysis/environment.py

@@ -3,7 +3,7 @@
 Writes a ``*.configs.json`` sidecar beside the result CSV:
 
 - ``environment`` — hardware, OS, runtimes, git (preferred)
-- ``dataset`` — optional: seed, fixtures, repetitions (best-effort)
+- ``dataset`` — optional: seed, data_types, repetitions (best-effort)
 - ``queues`` — optional: names from the run (best-effort)
 - ``run`` — optional: mode, metrics profile, timestamp
 
@@ -187,7 +187,7 @@ def _dataset_block() -> Dict[str, Any]:
 
         cfg = load_master_config()
         block["config_path"] = "config/benchmark_config.yaml"
-        # Catalog is normative for suite fixtures.
+        # Catalog is normative for suite data types.
         block["catalog_file"] = dig(
             cfg, "test_data.catalog_file",
             dig(cfg, "data_model_v2.catalog_file", "schemas/data_catalog_v2.yaml"),
@@ -196,7 +196,7 @@ def _dataset_block() -> Dict[str, Any]:
         block["test_data_config"] = block["catalog_file"]
         types = dig(cfg, "test_data.types") or []
         if isinstance(types, list) and types:
-            block["fixtures"] = [
+            block["data_types"] = [
                 {
                     "name": t.get("name"),
                     "category": t.get("category"),

analysis/src/benchmark_analysis/metrics_catalog.py

@@ -17,8 +17,7 @@
     "avg_time_deq_ns": "medium",
     "handoff_mean_ns": "medium",
     "avg_ops_per_sec": "high",
-    "median_size_bytes": "high",
-    "mean_fidelity": "high",
+    "mean_fidelity": "medium",
     "library_version": "high",
     "runs": "high",
     "handoff_ci_low_ns": "medium",
@@ -110,8 +109,9 @@ def rank_by_field(metrics_cfg: Optional[Dict[str, Any]] = None) -> str:
     ("enq_median_ns", "Median enqueue (µs)", True, False),
     ("deq_median_ns", "Median dequeue (µs)", True, False),
     ("avg_ops_per_sec", "Ops/s (from mean)", False, True),
-    ("median_size_bytes", "Median size (B)", False, False),
-    # runs / mean_fidelity intentionally omitted from multi-way Summary.
+    ("handoff_p99_ns", "Handoff p99 (µs)", True, False),
+    ("msgs_per_cpu_sec", "Msgs / CPU-s", False, True),
+    # Fidelity is a gate, not a ranked score.
     ("library_version", "Version", False, None),
     ("effect_vs_fastest_cliffs_label", "δ vs fastest", False, None),
 )

analysis/src/benchmark_analysis/regression.py

@@ -210,7 +210,6 @@ def save_baseline(
                 pick_stats(stat, "handoff_ci_high_ns") or handoff
             ),
             "avg_ops_per_sec": float(stat.get("avg_ops_per_sec") or 0.0),
-            "median_size_bytes": float(stat.get("median_size_bytes") or 0.0),
             "runs": int(stat.get("runs") or 0),
         }
         if store_samples:

analysis/src/benchmark_analysis/reports.py

@@ -56,7 +56,7 @@ def _stat_get(entry: Any, field_id: str) -> Any:
 _FIXTURE_KEY_RE = re.compile(r"^(.*?)(?:@n=(\d+))+$", re.IGNORECASE)
 
 
-def _format_fixture_display(label: str) -> str:
+def _format_data_type_display(label: str) -> str:
     """Decode cryptic ``message@n=100`` keys for titles and table headers.
 
     Examples:
@@ -77,7 +77,7 @@ def _format_fixture_display(label: str) -> str:
     else:
         base, n = s, None
 
-    # Suite type_ids are lowercase words; legacy fixtures are already Title/Pascal.
+    # Suite type_ids are lowercase words; older labels may already be Title/Pascal.
     if base and base == base.lower() and re.fullmatch(r"[a-z][a-z0-9_]*", base):
         pretty = base.replace("_", " ").title()
     else:
@@ -206,13 +206,13 @@ def _generate_violin_plot(
     top_n: Optional[int] = None,
     data_source: str = "",
 ) -> Optional[str]:
-    """Generate combined mean-bar + violin figure for one fixture.
+    """Generate combined mean-bar + violin figure for one data type.
 
     Layout (shared Y = queue rank, both linear µs from 0):
       left  — horizontal bars at **mean** enqueue / dequeue (easy ranking; aligns with ops/s)
       right — split violins of full sample density (spread / shape)
 
-    Embeds mapping metadata (fixture, language id, log path, modes, n) in the
+    Embeds mapping metadata (data type, language id, log path, modes, n) in the
     title/footer so plots can be tied back to CSV results.
     """
     if melted_df.empty or data_type not in melted_df['TestDataName'].values:
@@ -333,8 +333,8 @@ def _generate_violin_plot(
             leg_v.remove()
 
         lang_key = (lang_id or _lang_file_key("", language)).lower().replace("#", "sharp")
-        safe_fixture = data_type.replace(" ", "_")
-        img_name = f"{lang_key}_{safe_fixture}.png"
+        safe_type = data_type.replace(" ", "_")
+        img_name = f"{lang_key}_{safe_type}.png"
         src = data_source or f"logs/{lang_key}/benchmark-log.csv"
         modes = sorted(
             {str(m) for m in subset.get("Pattern", pd.Series(dtype=str)).dropna().unique()}
@@ -344,7 +344,7 @@ def _generate_violin_plot(
         top_note = f" · Top {int(top_n)}" if top_n and int(top_n) > 0 else ""
 
         fig.suptitle(
-            f"{language or lang_key} · {_format_fixture_display(data_type)}{top_note}",
+            f"{language or lang_key} · {_format_data_type_display(data_type)}{top_note}",
             fontsize=12,
             y=1.02,
         )
@@ -400,7 +400,7 @@ def _generate_violin_plot(
         return None
 
 
-# Latency distributions: always show this many fastest serializers per fixture (all languages).
+# Latency distributions: always show this many fastest queues per data type (all languages).
 VIOLIN_TOP_N_SERIALIZERS = 5
 
 # CSV StringOrStream values → human labels (not "number of bytes")
@@ -655,7 +655,7 @@ def _pick_entry(matching: List[Dict]) -> Optional[Dict]:
             else:
                 cell[(rv, cv)] = None
 
-    # all@all-style row average: mean of per-fixture cell values (type × n).
+    # all@all-style row average: mean of per-data-type cell values (type × n).
     row_avg: Dict[str, Optional[float]] = {}
     if include_row_average:
         for rv in row_vals:
@@ -875,7 +875,7 @@ def _category_pivot_md(stats: Dict, lang_id: str, title: str) -> str:
         "",
     ]
     for cat in sorted(by_cat.keys()):
-        # average ops per serializer across fixtures; rows sorted by name
+        # average ops per queue across data types; rows sorted by name
         acc: Dict[str, List[float]] = defaultdict(list)
         for ser, ops in by_cat[cat]:
             acc[ser].append(ops)
@@ -952,8 +952,9 @@ def _config_section_md(lang_id: str, csv_path: Optional[str]) -> str:
         )
     if doc:
         ds = doc.get("dataset") if isinstance(doc.get("dataset"), dict) else {}
-        if ds.get("fixtures"):
-            names = [f.get("name") for f in ds["fixtures"] if isinstance(f, dict) and f.get("name")]
+        types = ds.get("data_types") or ds.get("fixtures") or []
+        if types:
+            names = [f.get("name") for f in types if isinstance(f, dict) and f.get("name")]
             if names:
                 body.append(
                     f"- **Data types (config):** {', '.join(str(n) for n in names)}"
@@ -1074,7 +1075,7 @@ def _scientific_summary_md(stats: Dict, profile: str = "multi_way") -> str:
     if not cols:
         return ""
 
-    # One row per queue: prefer bytes mode, average medians across fixtures if needed
+    # One row per queue: prefer bytes mode, average medians across data types if needed
     by_ser: Dict[str, List[Dict]] = {}
     for e in stats.values():
         if not isinstance(e, dict):
@@ -1133,7 +1134,7 @@ def _scientific_summary_md(stats: Dict, profile: str = "multi_way") -> str:
 
     # Shared K/M scale per column (same rule as ops/s and pivot tables)
     col_units: Dict[str, tuple] = {}
-    for field_id in ("avg_ops_per_sec", "median_size_bytes"):
+    for field_id in ("avg_ops_per_sec", "msgs_per_cpu_sec"):
         vals_in_col = [cell_vals[(ser, field_id)] for ser in serializers if (ser, field_id) in cell_vals]
         if vals_in_col:
             col_units[field_id] = _pick_column_unit(vals_in_col)
@@ -1180,7 +1181,7 @@ def _scientific_summary_md(stats: Dict, profile: str = "multi_way") -> str:
             elif is_time:
                 # ns → µs; fixed-point only (never 1.17e+03)
                 text = _format_sig(num / 1000.0, sig=3)
-            elif field_id in ("avg_ops_per_sec", "median_size_bytes"):
+            elif field_id in ("avg_ops_per_sec", "msgs_per_cpu_sec"):
                 # 3 significant digits + shared column K/M (thousands / millions)
                 div, unit = col_units.get(field_id) or _pick_column_unit([num])
                 text = _format_in_unit(num, div, unit, sig=3)
@@ -1393,7 +1394,7 @@ def generate_language_results_pages(
             "| Term | Meaning |",
             "|------|---------|",
             "| **data type** | Sample shape: `message`, `document`, `telemetry`, `strings`, or `event` "
-            "(CSV `TestDataName`; older text may say “fixture”) |",
+            "(CSV `TestDataName`) |",
             "| **bytes mode** | In-memory buffer API (encode to bytes / decode from a buffer). "
             "On C# this is often the **string** path — see [Modes](../analysis/modes.md). |",
             "| **stream mode** | Stream-style API (write/read through a stream). "
@@ -1453,11 +1454,11 @@ def generate_language_results_pages(
                 base = str(e2.get("test_data") or "")
                 if n not in (None, ""):
                     try:
-                        e2["test_data"] = _format_fixture_display(f"{base}@n={int(n)}")
+                        e2["test_data"] = _format_data_type_display(f"{base}@n={int(n)}")
                     except (TypeError, ValueError):
-                        e2["test_data"] = _format_fixture_display(base)
+                        e2["test_data"] = _format_data_type_display(base)
                 else:
-                    e2["test_data"] = _format_fixture_display(base)
+                    e2["test_data"] = _format_data_type_display(base)
                 display_stats[k] = e2
 
             lines.append("## Summary tables")
@@ -1539,7 +1540,7 @@ def generate_language_results_pages(
             )
             lines.append("")
             for dtype, fname in items:
-                pretty = _format_fixture_display(dtype)
+                pretty = _format_data_type_display(dtype)
                 lines.append(f"### {pretty}")
                 lines.append("")
                 lines.append(f"![{pretty}]({plot_rel_from_lang}/{fname}){{ width=\"80%\" }}")
@@ -1624,7 +1625,7 @@ def generate_violin_plots(
     )
 
     violin_images: Dict[str, str] = {}
-    # lang_id -> {fixture -> plot filename} plus source path for results.md
+    # lang_id -> {data type -> plot filename} plus source path for results.md
     plot_meta: Dict[str, Dict] = {}
 
     for lang_id in lang_ids: