topic 44: e-graphs as a database, and the two testing systems topic 16 only named

[AviAvni]

Topic 44 — E-graphs as a Database: Relational E-matching & egglog

The sequel to topic 21, placed in this repo because the fix for equality saturation's next bottleneck came out of the database literature. E-matching is 60–90% of equality saturation's run time (POPL'22 §1) and it is a conjunctive query: the e-graph is a set of tables, the pattern is a query, and the equality constraint a backtracking matcher checks last is a join key.

The measured headline (./verify.sh 44, Apple M3 Pro)

On the POPL'22 Figure 2 e-graph — 3N e-nodes standing for N²+2N terms — the pattern f(a, g(a)) has N matches:

      N   e-nodes   matches    bt visits     bt µs   gj probes  index µs    gj µs  speedup
    100       300       100        10101     137.9         500      71.7     24.8    1.43x
    400      1200       400       160401    1119.8        2000      92.4     37.2    8.64x
   1600      4800      1600      2561601   10152.5        8000     322.4    145.0   21.72x

bt visits = N²+N+1, gj probes = 5N. Both are closed forms and reproduce exactly; only the µs columns are hardware-dependent.

And the negative result is a column, not a caveat. Rename the repeated variable — f(a, g(b)), a linear pattern — and every candidate becomes an answer: generic join does the same work through a more expensive instruction and comes out 0.56×, i.e. 1.8× slower. POPL'22 reports its own version (Table 1's Worst column: 0.76, and 0.03 with index building charged). Theorem 10 predicts both rows.

What's in the package

• experiments/ — a minimal e-graph, egg's Bind/Compare/Scan VM with the op index so the baseline is a real strategy rather than a strawman, Figure 8's unnesting, tries, generic join. 6 provided tests pass, 4 stub tests are the spec (semi-naive evaluation; a left-deep binary-join plan for the triangle multi-pattern).
• Four reading guides: relational e-matching, egglog, the egglog source, Free Join.

The source guide's finding is that the papers understate the codebase — core-relations is a database: semi-naive evaluation is a binary search on a clustered sort column, the planner does hypertree decomposition with a min-fill heuristic, and congruence closure is compiled into a rule rather than implemented. Its union-find declines union-by-rank for union-by-min-id and explains why, which is the same class of finding as egg's non-compressing find.

Topic 16 — the two systems it had only name-dropped

• reading-hypothesis.md — shrinking the choice sequence rather than the value; shortlex order worked on real indices; the shrink-pass determinism invariant; the DataTree as a trie over executions; and Hypothesis's documented deviation from the swarm-testing paper. find_integer(100) was executed rather than hand-traced: 16 calls, not the 17 a trace produced.
• reading-antithesis.md — read against the open SDK, since the platform is closed. The no-caching contract on get_random only makes sense for a branching simulation, not a replayed one; Sometimes as a coverage property (this topic's crash_matrix None row is the hand-rolled version); the linker-assembled assertion catalog. No Antithesis figure is quoted, because none can be checked from this side, and the guide says so.

Reviewer notes

• One benchmark bug, caught by the counters disagreeing with the clock: gj allocated a Vec per intersection key, which left every counter identical and doubled the wall clock. Recorded in notes.md.
• Lane 2's µs is the noisiest figure in the topic (6.5–11.0 ms across runs on one machine) because it materialises 20,008 substitution vectors. Its probe count is exact. notes.md says so.
• The pin table regeneration does more than add rows: GraphRAG-SDK had disappeared from ~/repos, so the tool dropped it and would have orphaned topic 38's anchors — re-cloned at its pinned f42ab3d, so no row is lost. Three of the maintainer's own clones have advanced since the table was last written (FalkorDB ccb449a9a → aa75821ab, falkordb-py, falkordb-rs-next-gen), so FalkorDB anchors in existing guides now name a newer commit than they were verified against. Easy to pin back if preferred.
• Pre-existing untracked topics/*/experiments/Cargo.lock files (topics 17–32) were left alone as out of scope.

Gates

• ./verify.sh 44 — PASS
• check-reading-depth.py --check --all — 236/236
• pin-table.py --check — current
• mdbook build clean; mermaid validated with @mermaid-js/mermaid-cli; 0 broken relative links; -D warnings build clean

🤖 Generated with Claude Code