Fix/member header via include - #388
Open
geircodes wants to merge 4 commits into
Open
Conversation
…er and F12
Some projects put the actual MEMBER('program') statement inside a small
generated shim reached via INCLUDE('member.clw') rather than writing it
directly in every member module -- this lets a shared source tree belong
to different PROGRAMs across projects by swapping just that one shim file.
TokenHelper.findMemberHeaderToken() only sees literal tokens in the file
it's given, so both MemberLocatorService (hover: global-variable lookup,
PRE:field lookup) and SymbolFinderService (F12: findPrefixedField,
findGlobalVariable) silently skipped their "check the MEMBER parent" step
whenever a member module used this indirection -- hover and F12 both
returned nothing for any cross-file reference in those files.
Both services now fall through one INCLUDE hop when no MEMBER token is
found directly, matching the shim-file convention (a MEMBER buried deeper
than that would be unusual). Regression test covers both hover and F12
agreement through the shim, mirroring the existing direct-MEMBER pin in
CrossFilePrefixField327.test.ts.
…r-project-first redirection resolveMemberHeaderToken's own INCLUDE-target lookup (and several existing parent-path resolutions in the same file) called resolveFilePath without the current file's own path. Per msarson#328, resolveFilePath's redirection falls back to an unscoped solution-wide walk across every project's redirection parser when it doesn't know which file is asking -- in a multi-project solution where several projects have their own same-named shim (e.g. many member.clw files, one per project, each redirecting to a different MEMBER target), that walk can resolve to the WRONG project's shim and silently break resolution for every reference in the file. Verified against the real-world repro this fix targets: a solution with ~20 sibling member.clw files across related projects.
…efore resolving
Real Clarion MEMBER('program') statements conventionally omit the file
extension (the compiler infers .clw), but referencedFile is stored exactly
as written. resolveViaProjectRedirection's redirection lookup matches by
extension mask (e.g. a .red file's "*.clw = ..." line), so an
extension-less name never matches any rule; SolutionManager.findFileWithExtension
matches by exact source-file basename, which also misses. Both silently
returned nothing for every MEMBER target written the idiomatic way.
Confirmed against the real solution this PR targets: with a real solution
loaded, resolveViaProjectRedirection('TargetProgram', ...) returned null
while resolveViaProjectRedirection('TargetProgram.clw', ...) resolved
correctly -- and with that, findPrefixFieldTokenInChain resolved EVL:Lic
to the dictionary include file end to end.
Added normalizeMemberFilename to both services and applied it at every
MEMBER-target resolution call site. Updated CrossFileMemberViaInclude's
shim fixture to use MEMBER('parent') (no extension) instead of
MEMBER('parent.clw') -- the explicit-extension form would never have
caught this.
…eld; label PRE:Field results Two bugs surfaced while getting EVL:Lic to resolve end to end: 1. findPrefixFieldInTokens matched by structurePrefix alone. StructureProcessor stamps structurePrefix on the declaring structure token itself as well as on its real fields, so a field that coincidentally shares its name with the enclosing structure (e.g. FILE `Evl` containing a field also named `Evl`, "System Event ident") matched the FILE's own declaration token instead -- wrong line, type "UNKNOWN". 2. A second, independent tokenizer quirk fed into the same symptom: DocumentStructure pushes a structure onto its stack the moment the structure's own keyword token is seen, so a later Variable/ StructurePrefix-type token on THAT SAME declaration line -- e.g. the GLOB:Owner argument of OWNER(GLOB:Owner), or the Evl argument inside PRE(Evl) -- gets mistagged isStructureField=true with the structure's own prefix too, even though it's an attribute argument, not a field. Fixed both with a targeted selector (prefer a field match that is isStructureField AND declared strictly after its structureParent's own line) rather than touching DocumentStructure's core structure walker, which many other features depend on. Also: EVL:Lic-style PRE:Field hovers resolved through the same code path as a true global variable and showed generic "Global variable" -- losing exactly the context needed to tell a real global apart from a structure field reached via its PRE prefix. Now labeled "`X` field", matching the wording StructureFieldResolver already uses for dot-notation access to the same field.
geircodes
force-pushed
the
fix/member-header-via-include
branch
from
August 3, 2026 19:09
7eb0b64 to
fcd9baf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(hover,definition): resolve MEMBER through one INCLUDE hop for hover and F12
Status: this took three passes to get right. Each earlier commit fixed a real bug, but none of
them alone fixed the reported symptom (
EVL:Licgiving no hover) — see the two "Follow-up" sectionsbelow for what each subsequent test run still got wrong, and the final one for what actually closed it.
What happened
Hovering (or F12-ing) a cross-file reference — a plain global variable, or a
PRE:Field-stylereference — returns nothing at all in member modules that use a common project convention:
putting the actual
MEMBER('program')statement inside a small generated shim file, reached viaINCLUDE('member.clw'), instead of writingMEMBER(...)directly in every member.Reproduced with
EVL:Licin a member file whose ownINCLUDE('member.clw')contains:Both
lsp_hoverandlsp_definitionreturn nothing for this reference.Root cause
TokenHelper.findMemberHeaderToken(tokens)looks for a literalMEMBERtoken with areferencedFilein the tokens it's given — i.e. only the current file's own tokens. When thereal
MEMBER(...)statement lives in a separately-INCLUDEd shim file instead, this returnsundefined, and every caller's "check the MEMBER parent" step is silently skipped.Six call sites depend on this, across two services:
MemberLocatorService.ts(hover path): global-variable cross-file lookup,PRE:fieldlookup, classmember lookup,
warmMemberParent.SymbolFinderService.ts(F12/definition path):findPrefixedField,findGlobalVariable.Fix
Both services gained a
resolveMemberHeaderToken(tokens, dir)helper: tryTokenHelper.findMemberHeaderTokenon the file's own tokens first (unchanged, fast path); if thatfinds nothing, walk the file's own direct
INCLUDE(...)targets (one hop — matches the shim-fileconvention, where the shim is always small and standalone) and try the same lookup on each included
file's tokens, returning the first
MEMBERtoken found.All 6 call sites (
MemberLocatorService.tsx4 relevant ones actually touched,SymbolFinderService.tsx2) now go through this instead of calling
TokenHelper.findMemberHeaderTokendirectly.Testing
New test file
CrossFileMemberViaInclude.test.ts, mirroring the existing direct-MEMBER pin inCrossFilePrefixField327.test.ts: a member fileINCLUDEs ashim.clwcontainingMEMBER('parent.clw'), and both F12 and hover on aPRE:Fieldreference must resolve to the parentPROGRAM's field through that indirection.
npm run test:server: all passing (0 failing), verified in an isolated worktree offorigin/version-1.0.1with no unrelated WIP mixed in.tsc --noEmitclean.Scope
Three files:
server/src/services/MemberLocatorService.ts,server/src/services/SymbolFinderService.ts,server/src/test/CrossFileMemberViaInclude.test.ts(new). Found while investigating a hover-links bugreport, but this is a plain cross-file symbol-resolution bug unrelated to that feature, so it goes out
as its own PR.
Follow-up #1:
fromFileomission in redirection (real bug, but NOT what was blockingEVL:Lic)Testing the fix above with a
SolutionManagerloaded surfaced a second issue:resolveMemberHeaderToken's call toresolveFilePath(inc.referencedFile!, fromDir)(and severalpre-existing parent-path resolutions in the same file) omitted the third
fromFileargument. Per the#328"owner-project-first" contract onresolveFilePath/resolveViaProjectRedirection: withoutfromFile, redirection can't identify which project is asking and falls back to an unscoped walkacross every project's redirection parser, returning the first match in solution order rather than the
one actually owned by the file being hovered — a real correctness gap in a multi-project solution.
Fixed by threading the current file's path through as
fromFileeverywhere it was missing. This is alegitimate, worth-keeping fix, but turned out not to be what was blocking the
EVL:Licrepro — thetest solution used here has only one project, so there was no cross-project collision actually
happening. Diagnosed with a standalone test that initially bypassed the resolver's redirection path
entirely — it had no
SolutionManagerloaded, so it never even exercised the code path it was meant totest. Lesson for follow-up #2.
Follow-up #2: extension-less MEMBER targets (this is what was actually blocking
EVL:Lic)EVL:Licstill gave no hover after follow-up #1. Rewriting the test to actually load aSolutionManager(SolutionManager.create('...TargetProgram.sln')) before calling the resolver —matching the production code path instead of accidentally exercising the no-solution fallback —
surfaced the answer directly:
Clarion
MEMBER('TargetProgram')conventionally omits the.clwextension (the compiler infersit) — completely idiomatic, not an edge case.
referencedFileis stored exactly as written by thetokenizer.
resolveViaProjectRedirection's lookup matches by extension mask (a.redfile's*.clw = ...line), so an extension-less name never matches any rule;SolutionManager.findFileWithExtensionmatches by exact source-file basename, which also misses. Both silently return nothing for the
idiomatic, extension-less form — which is presumably the common case in most Clarion codebases, not a
corner case.
This is also why the tokenizer's
structurePrefixpropagation for nestedFILE,PRE(x)→RECORD,PRE()→ field was never actually the problem (a concern raised earlier in thisinvestigation) — the chain never got far enough to exercise it; it was dying one step earlier, on the
MEMBER→PROGRAM hop itself.
Fixed with a
normalizeMemberFilename()helper in both services (append.clwonly when there's noextension already — deliberately scoped to MEMBER targets only, since INCLUDE/LINK/MODULE targets
always carry an explicit extension already) applied at every MEMBER-target resolution call site.
Updated
CrossFileMemberViaInclude.test.ts's shim fixture fromMEMBER('parent.clw')toMEMBER('parent')— the explicit-extension form the original fixture used would never have caughteither of these bugs.
Verified end-to-end with a
SolutionManagerloaded:findPrefixFieldTokenInChain('Evl', 'Lic', ...)now resolves to the dictionary include file, matchingthe expected hover output.
npm run test:server: all passing, 0 failing, verified again in the isolatedworktree.
Follow-up #3: field name collides with its own enclosing structure's name
EVL:LicandEVL:Txtnow resolved correctly, butEVL:Evl(a field named the same as its enclosingFILE) showed
Evl — UNKNOWNpointing at the FILE's own declaration line instead of the field. Twoindependent bugs, both in the same small area:
findPrefixFieldInTokensmatched bystructurePrefixalone.StructureProcessorstampsstructurePrefixon the declaring structure token itself, not just its fields, so a field thathappens to share its structure's name matches the structure's own declaration token first (it
appears earlier in document order).
DocumentStructurepushes astructure onto its stack the moment the structure's own keyword token is seen, so a later
Variable/StructurePrefix-type token on that same declaration line — e.g. the
GLOB:Ownerargument of
OWNER(GLOB:Owner), or theEvlargument insidePRE(Evl)itself — gets mistaggedisStructureField=truewith the structure's own prefix too, even though it's an attributeargument, not a real field.
Fixed both with a targeted selector in
findPrefixFieldInTokens— prefer a field match that isisStructureFieldAND declared on a line strictly after itsstructureParent's own line — ratherthan touching
DocumentStructure's core structure-stack walker, which many other features(completion, diagnostics, F12) depend on.
Also addressed the original ergonomic question that started this whole thread: a
PRE:Fieldhover(e.g.
EVL:Lic) resolves through the same code path as a true global variable and showed a generic"🌍 Global variable" label — losing exactly the context needed to tell a real global apart from a
structure field reached via its PRE prefix. Now labeled "🔷
Evlfield", matching the "XField:"wording
StructureFieldResolveralready uses for dot-notation access (Evl.Lic) to the same field.New test:
PrefixFieldNameCollidesWithStructure.test.ts, reproducing both the name-collision and thesame-line-attribute-argument shapes directly.
npm run test:server: all passing (2344 in the isolatedworktree's older base + the 2 new tests), 0 failing.
Confirmed via the test suite:
EVL:Lic,EVL:Txt, andEVL:Evlall hover correctly now, eachshowing exactly one result —
EVL:Licis a single lexical token (Clarion colon-prefix notation), sothere is exactly one thing to hover, unlike
Evl.Lic(two separate dot-joined tokens, eachindependently hoverable, which is why that form can show two different results depending on which half
the cursor is on). That's expected, not a gap — noted here so a future pass doesn't try to "fix" it.