Make the .tsv-based reflection system more flexible

[pawelbaran]

Origins:

As explained in BHoM/Revit_Toolkit#1682, BHoM for Revit currently does not leverage the .tsv-based reflection mechanism introduced in #531. There is a few reasons for it, main ones being:

• in Revit_Toolkit we only need reflection (AssemblyResolver), no need for the whole call to BH.UI.Base.Global.Initialisation.Activate
• Revit_Toolkit would load Revit-specific code, which should not land in the main .tsv, to make it more fun, the .tsv will differ between Revit versions
• following the above, there is a need to combine 2 .tsv files created based on different sets of assemblies (one general BHoM dataset, the other Revit-specific)

...what is more, some Revit tools will require an even more complex setup:

• we would want to make use of AssemblyResolver
• separate .tsv as in case of Revit_Toolkit, but additionally the records should store extra information, such as e.g. a dedicated icon that should appear already in search (i.e. before the actual assembly is loaded)

Having written all the above, I hope my rationale is clear - if I were to express myself in form of a statement, I would say:

.tsv-based reflection at its core is not related to UI - it is a system to lazy load BHoM assemblies and their contents. BHoM should allow UIs and adapters to consume it in a modular, extensible manner, i.e. to combine the core dataset with UI-specific ones.

I've been planning to tackle this for quite some time already, finally landed in our roadmap this milestone, so I am giving it a go, everyone invited to join the journey 🚀

High level objectives:

1. Enable .tsv-based reflection in Revit_Toolkit:
   • let Revit_Toolkit trigger only the reflection layer of the current BH.UI.Base.Global.Initialisation.Activate() method
   • capture Revit_Core_Engine assembly in the system to make the methods discoverable via TryRunExtensionMethod:
     • requires different Regex filter than the one hardcoded in BH.Engine.UI.Compute.LoadNewAssemblies
     • requires a dedicated .tsv path not to mix it with non-Revit code (otherwise GH would reflect the Revit code)
2. Enable extension of CodeElementRecord class with extra properties, to be then used in combination with yet another Regex filter and .tsv file

Refactoring

As one can reckon, achieving the above would require a serious bit of refactoring. When planning the job, I did not know much about the whole system, so was learning on the job and extending the list below. What I ultimately landed with as feasible and (I believe) desirable is:

• Decouple reflection layer of UI initialisation and migrate it to BHoM and BHoM_Engine (closer to the actual assembly resolution)
• While doing this, restructure the code in a way that would make it callable and extendable (mainly making the code more parametric in terms of file paths, filters etc., plus tweaking signatures of the key methods to return actual CodeElementRecords rather than bools etc.)
• Drop Json property in the CodeElementRecord class - I originally did not plan it, but since ToJson method depends on Serialiser_Engine, I could not put the code in neither BHoM_Engine nor Reflection_Engine. That brought me to an investigation whether the property could not go altogether and... it can! It is used in 2 way across the code base:
  • to capture first parameter of a method, this can also be achieved by taking first item from OutputKeys
  • deserialise a method/type from json (in SearchMenu and ItemSelectorMenu) - actually, since the CodeElementRecords perfectly match runtime code (they are created/updated at startup), we can write a method that would find the relevant item based on CodeElementRecord.DisplayText property value - I wrote a prototype-ish method that proves the concept (BH.Engine.Base.Query.ItemByKey)
    ...this means that we can get rid of serialisation, making the whole process much more lightweight. That's what I did in my PRed code.
• Change the CodeElementType enum - now it is very UI-oriented, while at the reflection level some separations like Adapter constructor vs Request constructor do not matter - I gave it a go and managed to maintain the current UI functionality with much more concise enum.

I will raise a chain of PRs in a while to show how could the above look like.