test(morphology): assert the 2D morphology features against MATLAB, scikit-image and imea - #431
Open
darkclad wants to merge 1 commit into
Open
test(morphology): assert the 2D morphology features against MATLAB, scikit-image and imea#431darkclad wants to merge 1 commit into
darkclad wants to merge 1 commit into
Conversation
…cikit-image and imea
Closes the 2D morphology rows that read status=vetted in oracle_coverage.csv while no in-tree
oracle test asserted them, by running each claimed oracle and pinning what it produces.
The planned gap was 17 rows. It is 19: AREA_PIXELS_COUNT and MAJOR_AXIS_LENGTH were credited to
test_2d_morphology_analytic.h, which reads them only as inputs to its CIRCULARITY and ROUNDNESS
formula checks and asserts neither. Running the oracles turned up a second, unplanned gap of 7 more
rows in the imea family, so 26 rows are settled here.
Promoted (22)
- 17 vs MATLAB (GNU Octave 11.3.0 + image 2.20.0 regionprops) on the shape2d fixture: the areas,
centroids, weighted centroids, bounding box, aspect ratio, extent, the ellipse triple and the
Euler number. Every one agrees to ~1e-15. MATLAB applies the same +1/12 pixel finite-size
second-moment correction Nyxus does, which is why MAJOR/MINOR_AXIS_LENGTH and ECCENTRICITY vet
here where scikit-image (which omits it, differing 1.4%) could not.
- DIAMETER_EQUAL_AREA vs scikit-image equivalent_diameter_area, exact.
- 4 vs imea on the caliper ellipse: the three caliper _STDDEV statistics and ALLCHORDS_MIN.
Demoted to status=regression, with the measurement in the audit reports (4)
- ROI_RADIUS_MEDIAN. ROI_RADIUS_* report SQUARED distances: RoiRadiusFeature::calculate feeds
Pixel2::min_sqdist() straight into the mean/max/median. Measured on filled disks, MAX = (R-1)^2
(82, 362, 1522 for R = 10, 20, 40) - it scales as R^2, so no skimage radius statistic can
reproduce it. The units defect, and the fact that min_sqdist() is the approximate hill-descent its
own header documents as overestimating (exact_min_sqdist() sits beside it), are behaviour changes
and are left for their own branch.
- The three caliper _MODE statistics. The mode of a caliper distribution is an artifact of the
angular sampling step: imea's own mode ranges over 19..24 as dalpha goes 5 -> 30, further than the
Nyxus-imea gap, so no tolerance separates agreement from sampling noise.
Corrected attributions
- PERIMETER was asserted in test_2d_morphology_matlab.h under a recipe documented as
nnz(bwperim(...)) = 846. That recipe returns 846 perimeter PIXELS and regionprops('Perimeter')
returns 952.848, while the golden was 999.26 - which is scikit-image's measure.perimeter
(999.259018078045, agreeing with Nyxus to 3.8e-15). The assertion moved to the skimage file at
full precision and the registry now names the oracle that backs it.
- morphology_2d_imea_shape2d_ref_vals pinned 19 caliper/chord statistics that had never been
compared to imea; imea's own values on that 8x8 raster differ by 3.9-79.3%. They are Nyxus
snapshots and now live in test_2d_morphology_regression.h. An oracle-named test asserting a
snapshot table passes every automated check in the repo, because the oracle is attributed from the
function-name suffix - only running the tool finds it.
- The imea ellipse goldens mixed dalpha 9 and 10 and one (Nassenstein MIN = 16.0) matched no run at
any step. Regenerated at dalpha=10, the step Nyxus' own calipers sweep (rot_angle_increment,
caliper.h), which drops the worst residual from 8.9% to 4.99% and lets the assertion tolerance
tighten from 10% to 6%.
- DIAMETER_EQUAL_PERIMETER, GEODETIC_LENGTH and THICKNESS were each pinned twice, in a ref-vals
table and again as a literal in the assertion, with the assertions reading the literals; the two
copies of DIAMETER_EQUAL_PERIMETER had already drifted. The assertions now read the tables.
Registry hygiene for the whole family, per the revet.txt checklist: every one of the 99 rows now
carries a real config_recipe id (three new recipes - morphology.shape2d_native,
morphology.perimeter_circles, morphology.caliper_ellipse - plus morphology.fractal_blob512), 28 rows
whose target_test was already satisfied have it cleared, and the fraclac goldens gained the SPEC 6.4
provenance pointer they were missing. The six CellProfiler rows are left as the family's one
remaining source=tracker claim: no CellProfiler version, config or generator is recorded, so the
honest closure is a generator run or a demotion, not a relabel.
Include hygiene: each morphology header now includes the standard headers it uses directly
(<string>, <vector>, <unordered_map>, <algorithm>, <cmath>) instead of relying on
test_2d_morphology_common.h to drag them in, and imea's include comment named two symbols that live
elsewhere or not at all.
Also repairs one pre-existing registry row: 3D/3ROBUST_MEAN was written with an unquoted comma
inside candidate_oracle ("[P10,P90]"), splitting it into 15 fields and shifting every column after
it, so source read empty and notes read "tracker". coverage_report.md is regenerated, which also
corrects a stale intensity_histogram count left from PolusAI#429.
Generators re-verify every pinned golden in the header they feed and exit non-zero on a mismatch or
on a pin they cannot produce: gen_morphology_matlab.m 33/33, gen_morphology_skimage.py 6/6,
gen_morphology_imea.py 20/20. All three read their fixtures out of tests/test_data.h rather than
carrying a second copy.
Verified: runAllTests 779/779 on Windows and 779/779 under ASan+UBSan on Linux (0 AddressSanitizer
reports, 0 runtime errors, 0 leaks); pytest tests/python/ 92 passed, 1 skipped (Arrow-enabled
build); check_coverage.py --check, check_test_names.py --check and
audit/scan_morphology_coverage.py --check all clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the 2D morphology rows that read status=vetted in oracle_coverage.csv while no in-tree oracle test asserted them, by running each claimed oracle and pinning what it produces.
The planned gap was 17 rows. It is 19: AREA_PIXELS_COUNT and MAJOR_AXIS_LENGTH were credited to test_2d_morphology_analytic.h, which reads them only as inputs to its CIRCULARITY and ROUNDNESS formula checks and asserts neither. Running the oracles turned up a second, unplanned gap of 7 more rows in the imea family, so 26 rows are settled here.
Promoted (22)
Demoted to status=regression, with the measurement in the audit reports (4)
Corrected attributions
Registry hygiene for the whole family, per the revet.txt checklist: every one of the 99 rows now carries a real config_recipe id (three new recipes - morphology.shape2d_native, morphology.perimeter_circles, morphology.caliper_ellipse - plus morphology.fractal_blob512), 28 rows whose target_test was already satisfied have it cleared, and the fraclac goldens gained the SPEC 6.4 provenance pointer they were missing. The six CellProfiler rows are left as the family's one remaining source=tracker claim: no CellProfiler version, config or generator is recorded, so the honest closure is a generator run or a demotion, not a relabel.
Include hygiene: each morphology header now includes the standard headers it uses directly (, , <unordered_map>, , ) instead of relying on test_2d_morphology_common.h to drag them in, and imea's include comment named two symbols that live elsewhere or not at all.
Also repairs one pre-existing registry row: 3D/3ROBUST_MEAN was written with an unquoted comma inside candidate_oracle ("[P10,P90]"), splitting it into 15 fields and shifting every column after it, so source read empty and notes read "tracker". coverage_report.md is regenerated, which also corrects a stale intensity_histogram count left from #429.
Generators re-verify every pinned golden in the header they feed and exit non-zero on a mismatch or on a pin they cannot produce: gen_morphology_matlab.m 33/33, gen_morphology_skimage.py 6/6, gen_morphology_imea.py 20/20. All three read their fixtures out of tests/test_data.h rather than carrying a second copy.
Verified: runAllTests 779/779 on Windows and 779/779 under ASan+UBSan on Linux (0 AddressSanitizer reports, 0 runtime errors, 0 leaks); pytest tests/python/ 92 passed, 1 skipped (Arrow-enabled build); check_coverage.py --check, check_test_names.py --check and audit/scan_morphology_coverage.py --check all clean.