Skip to content

fix(normalisation): retry on ValueError when sampling null ensemble - #52

Merged
cbueth merged 3 commits into
mainfrom
fix/flaky-reciprocity-test
Aug 12, 2026
Merged

fix(normalisation): retry on ValueError when sampling null ensemble#52
cbueth merged 3 commits into
mainfrom
fix/flaky-reciprocity-test

Conversation

@cbueth

@cbueth cbueth commented Jul 30, 2026

Copy link
Copy Markdown
Owner

The @normalise_against_random decorator generates directed $G(n,m)$ random graphs and feeds them to the metric function to build the null distribution. For small graphs, a random graph can coincidentally end up with a perfectly symmetric adjacency matrix (all edges reciprocated). The reciprocity metric rejects such matrices with ValueError, causing a flaky test failure.

Fix: catch ValueError from the metric function during the sampling loop and retry with a fresh random graph. A for...else guard raises RuntimeError if valid samples cannot be collected within n_rand_val x 10 attempts, ensuring the null distribution always has exactly the requested number of samples.

@cbueth cbueth self-assigned this Jul 30, 2026
@cbueth cbueth added the bug Something isn't working label Jul 30, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 30, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 24.76%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 1 regressed benchmark
✅ 27 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
test_detrend_method[z_score-p10] 1.4 ms 1.8 ms -24.76%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing fix/flaky-reciprocity-test (c29af52) with main (33476c0)

Open in CodSpeed

@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (875adcd) to head (c29af52).
⚠️ Report is 15 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #52   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           29        29           
  Lines          842       866   +24     
  Branches       151       150    -1     
=========================================
+ Hits           842       866   +24     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cbueth
cbueth merged commit 9424a89 into main Aug 12, 2026
12 of 13 checks passed
@cbueth
cbueth deleted the fix/flaky-reciprocity-test branch August 12, 2026 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant