Skip to content

Do not report versioning findings whose declaring assembly is outside the closure - #1

Open
sakanni wants to merge 2 commits into
developfrom
fix/ci-versioning-closure-aware-classification
Open

Do not report versioning findings whose declaring assembly is outside the closure#1
sakanni wants to merge 2 commits into
developfrom
fix/ci-versioning-closure-aware-classification

Conversation

@sakanni

@sakanni sakanni commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

ci-versioning failed BHoM/Revit_Toolkit#1688 with two findings while BHoMBot's versioning passed on the same SHA. Neither finding is a regression, and neither is caused by that PR. Both are dataset entries whose recorded declaring assembly is not in the closure we build:

  • Compute.TryGetValueFromSource is declared by Revit_ModelQA_Engine_2022, another repo that happens to declare types into BH.Revit.Engine.Core
  • Create.ProjectParameter is declared by Revit_Core_Engine_2024, a configuration we do not build

In both cases another loaded assembly answered for the type, so the signature probe resolved against code the dataset was never describing, and the finding was counted real. BHoMBot does not have this problem because CloneInstaller.cs:23 builds every alternate configuration and the whole installer set before versioning runs.

Two changes, and they need to ship together.

Classifier. When the recorded declaring assembly is absent from the closure and some other assembly answered for the type, the finding is reported as unverified rather than real, split into ForeignDeclaringAssembly and ConfigurationNotBuilt so the reason is legible. It stays real when nothing answered for the type, which is how a genuine removal presents, and when the family is ours but no configuration of it is loaded at all.

altConfigs. ci-versioning now builds the calling repo's own Release* entries from altConfigs.txt. Without this the classifier change is a straight loss of coverage on Revit repos: a method deleted from a configuration we never compile becomes unverified instead of caught. Measured on a sandbox Revit_Toolkit over three runs on windows-2025-vs2026: 47 to 59 seconds for the first configuration, 4 to 8 seconds for each additional one, 76 seconds total against a 47 second Release-only baseline. That is 1.6x, not the 5x a per-configuration estimate suggests, because the first build carries the warm-up. Build output grows 1.2 MB per configuration.

Validation: suite is 100/100 with 13 new cases; four mutations of the classifier and two of the config parsing are each caught by the intended test. Six synthetic dataset entries covering deleted methods, arity changes and moved types stay real on a complete closure, before and after. On a closure shaped like CI's, the two known false positives reclassify and the control finding does not move. Detail in the session note.

Also fixes a counter this change would otherwise desynchronise: the CI_Toolkit#161 ambiguity warning counted every diagnostic while printing detail only for real findings.

Do not merge yet. Building five configurations puts about 25 assemblies in Build\, which changes the --subject-assemblies surface and should stop ConfigurationNotBuilt firing on Revit repos at all. That combination is the one that ships and it has not been run end to end. A sandbox run of ci-versioning with both changes on a Revit repo is the gate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant