Skip to content

Add Experiment 13: which payload sizes change ranking - #32

Closed
leo-gan wants to merge 3 commits into
masterfrom
exp-13-payload-size-sweep
Closed

Add Experiment 13: which payload sizes change ranking#32
leo-gan wants to merge 3 commits into
masterfrom
exp-13-payload-size-sweep

Conversation

@leo-gan

@leo-gan leo-gan commented Sep 1, 2026

Copy link
Copy Markdown
Owner

Summary

  • Asks which SPSC payload sizes change the ranking (1 B, 64 B, 256 B, 4 KiB, 64 KiB).
  • Ran in Python, Rust, JavaScript, C#, and C (full, 99 trials after warmup).
  • Combined page: experiments/13-payload-size-sweep/results.md.

Finding

  • You were right that queue benches should split on size, not serializer-style data types. Runners already build opaque byte strings.
  • Two published sizes are enough: 256 B (message) and 4 KiB (document).
  • 1 B and 64 B match 256 B (same first place; mid-pack swaps are ties).
  • 4 KiB is the first copy-cost knee in Rust (pack collapses, ~9× slower).
  • 64 KiB inverts the Rust ranking (~280× slower). Keep it as this experiment, not a third default type.
  • event / telemetry / strings (512 B / 1 KiB / 2 KiB) add no ranking question.
  • Item count 1 / 100 / 10000 is a different axis. n=1 is wakeup. n=1000 is amortization. Do not use those as payload sizes.
  • Default config/library/default.yaml is not collapsed in this PR.

How to read

  • Times in two languages are not one contest.
  • Read the similar / close sets. Do not crown a single winner.

Validation

  • Timed runs completed for: python, rust, javascript, csharp, c. Failed: none.
  • No language harness source changed; published site Results / dashboard language payloads were not regenerated.

Notes

  • run.yaml is generated from the experiment settings (checked in).
  • Experiment logs stay local. Do not commit them.

SPSC ranking is a size question, not a named data-type question.
Sweep 1 B / 64 B / 256 B / 4 KiB / 64 KiB at n=100 with full reps.
Finding: default matrix needs 256 B and 4 KiB; 64 KiB is research-only.
Voice in STYLE.md now covers replies to the user. Experiment findings
must follow that section instead of restating a shorter rule.
@leo-gan

leo-gan commented Sep 1, 2026

Copy link
Copy Markdown
Owner Author

Superseded by #33, which includes Experiment 13 and the default-matrix collapse.

@leo-gan leo-gan closed this Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant