Allow a TOC entry to source content from outside the documentation set root

[clintandrewhall]

Prerequisites

• I have searched existing issues to ensure this feature hasn't already been requested
• I have tested using the latest version of docs-builder (1.33.0)

What problem are you trying to solve?

We publish Kibana's internal developer docs to Codex from a docs-dev/ docset, separate from the public docs/ docset at the repository root. The content documents packages that live under src/platform/, and the maintainers want each page to sit next to the code it describes, so that a change to a component and a change to its documentation appear in the same directory and the same review.

That isn't expressible today. TOC paths are resolved relative to the documentation set root — documentation-set-navigation.md states it directly ("All file paths in the toc section are relative to the documentation set root") — and there is no way to point an entry at a file above that root.

The workarounds both cost something real. Moving the docset root into the package tree colocates the content, but a repository effectively supports one docset per registry (FindDocsetFile returns the first match for the environment), so rooting it inside one package's directory prevents any other area of the repo from publishing internal docs later. Keeping the docset where it is means the documentation cannot live beside the code it documents.

Proposed Solution

Let a TOC entry keep its docset-relative position while sourcing its content from elsewhere:

toc:
  - file: index.md
  - file: feedback.md
    source: ../../src/platform/kbn-ui/feedback/feedback.md

file: stays the virtual, docset-root-relative path that drives the URL, navigation and link reference. source: is the on-disk location, resolved relative to the directory holding the toc.yml/docset.yml, and used only for reading.

Keeping the two separate matters: it preserves the invariant the navigation model depends on ("All paths become relative to docset root", two-phase-loading.md), keeps generated URLs free of ../ segments, and keeps the output path inside the output directory. Allowing a bare - file: ../../… instead would push a path that escapes the root through URL generation and output writing — and, as detailed under Alternative Solutions, nothing currently stops that from happening quietly.

Read-only, single files. No need for folder globs to escape the root.

Examples and Research

The pieces already exist in the codebase, which is the main reason this looks tractable rather than structural.

A TOC entry can already reference a path outside the content root — just only for one extension. From configure/content-set/extensions.md:

toc:
  - file: index.md
    detection_rules: '../rules'

The supporting mechanisms are also in place:

• IDocsBuilderExtension.ExternalScopeRoots exists precisely to widen the ScopedFileSystem for directories outside the working root, and DetectionRulesDocsBuilderExtension resolves them with Path.GetFullPath(f, Build.DocumentationSourceDirectory.FullName), so ../-relative roots are already handled.
• DetectionRuleFile.SourcePath already maps an external file into a virtual path inside the docs source directory, which is the same source/virtual split proposed here.
• CLI references already derive a virtualRoot that differs from the on-disk layout, so navigation is not assumed to mirror the filesystem.

In a normal docs-builder invocation the read filesystem is scoped by RealGitRootForPath, which resolves to the git root rather than the docset directory, so a sibling path inside the same repository is already within the scope root. DocumentationSetFile.LoadAndResolve falls back to ScopeSourceDirectory(docsetDir) when no filesystem is supplied, so that path would need the same treatment.

Worth noting that the detection-rules precedent does not generalize on its own. DetectionRulesDocsBuilderExtension.CreateDocumentationFile returns null for anything that isn't .toml (the single .md exception is a synthetic "deprecated rules" overview matched by name), so pointing detection_rules: at a directory of Markdown yields no pages. Enabling the extension is also docset-wide and claims every index page — CreateMarkdownFile returns a DetectionRuleOverviewFile for any file named index.md — so it can't be borrowed as a general include without side effects on unrelated pages.

Given the above, the work looks like: accept and resolve the new key, extend the scope roots the way the extension mechanism already does, and map the external file to its virtual path before URL/output derivation. The validation that already runs on the virtual name (IsValidFileName, lowercase-only) would continue to apply unchanged.

Alternative Solutions

• Move the docset into the package tree. Colocates content, but concedes the repo to a single internal docset rooted inside one package.

• Symlinks. Rejected: docset.yml is explicitly checked for symlinks as a path-traversal guard, and the detection-rules walker skips them, so this clearly cuts against the intended design.

• Duplicate or generate the file into the docset. Adds a sync step and a second copy that will drift.

• Bare ../ in file:. Simpler to express, but it appears to fail quietly rather than loudly, which is the main argument for spending the extra surface on an explicit source: key.

  ResolveFileRef in DocumentationSetFile.cs builds its path by string concatenation, with no containment check:

  var fullPath = string.IsNullOrEmpty(parentPath)
      ? fileRef.PathRelativeToDocumentationSet
      : $"{parentPath}/{fileRef.PathRelativeToDocumentationSet}";

  So - file: ../../src/platform/kbn-ui/feedback/feedback.md is accepted at parse time, and that same relative path goes on to drive URL generation and the output location — writing the page outside the output directory with ../ segments in its URL. The check that would otherwise catch a malformed name doesn't catch this one: IsValidFileName is applied to Path.GetRelativePath(Context.OutputDirectory.FullName, outputFile.FullName), and FilePathRegex (^[a-z0-9\s\-_\.\/\\+]*[a-z0-9_\-+]\.([a-z]+)$) permits dots and slashes anywhere in the path portion, so a leading ../.. matches.

  This is from reading main rather than running it end to end, so it's offered as a design argument for the explicit key rather than as a bug report. If you'd prefer it tracked separately as missing validation, happy to split it out.

Additional Context

Related but separate: #3797, where --path bounds git-root discovery and leaves DocumentationCheckoutDirectory null. It matters here only because the GitHub edit link for an externally-sourced page would be derived from that same checkout directory.

How important is this feature to you?

Nice to have — we have a working setup either way; this would remove a real trade-off between colocation and keeping the internal docset open to other teams.