Skip to content

docs: docs/ has silently diverged between main/develop and the orphan docs branch #408

Description

Summary

docs/ exists on two branches that have silently diverged. There is no
sync between them, so a docs fix landed on one side never reaches the other.
This has already produced real, user-visible defects.

Branch Role Link style
docs (orphan) source the published site is built from absolute (https://github.com/Azure/agentops/blob/main/...)
main / develop copy linked from README.md and CONTRIBUTING.md relative (../src/agentops/...)

Measured drift

Comparing origin/main:docs/ against origin/docs:docs/:

main docs/ files : 16
docs docs/ files : 28
only in main     : 4
only in docs     : 16
common files     : 12
DIVERGENT        : 8

8 of the 12 common files have different content, including the most
load-bearing pages:

docs/ci-github-actions.md
docs/concepts.md
docs/doctor-explained.md
docs/e2e-live-setup.md
docs/evaluation.md
docs/how-it-works.md
docs/release-process.md
docs/tutorial-prompt-agent.md

Four files exist only on main and are therefore invisible on the site:

docs/bundles.md
docs/media/agentops-diagrams.vsdx
docs/tutorial-end-to-end.md
docs/tutorial-hosted-agent-quickstart.md

main has no mkdocs.yml, so its docs/ tree is not buildable as a site.

Why this is not merely cosmetic

Both copies carried the same wrong "Pre-flight checks" section, which
documented agentops eval run --dry-run for CI gating. That flag does not
exist. Fixing it required two separate PRs (#406 on docs, #407 on
develop) for one defect, and nothing would have caught the second copy if
the drift had not been noticed by hand.

The two copies are also not merely stale relative to each other, they have
structurally diverged: one uses absolute GitHub blob links, the other uses
relative paths. A straight cherry-pick between them does not apply cleanly.

Failure modes this creates

  1. A contributor edits docs/how-it-works.md on develop, the PR merges,
    and the published site never changes. The work looks done and is not.
  2. A reader following README.md lands on the in-repo copy and gets
    instructions that were corrected on the site months earlier.
  3. Any documentation lint, link check or flag audit that runs in CI on
    develop never inspects the content that is actually published.

Suggested resolutions

Pick one, in rough order of preference.

A. Single source of truth on main/develop. Move mkdocs.yml into
the code branch and have deploy-docs.yml build straight from it. Delete
the orphan docs branch. This is the conventional layout and removes the
class of bug entirely.

B. Keep the orphan branch, automate the sync. Add a workflow that
opens a PR onto docs whenever docs/** changes on develop, rewriting
relative links to absolute ones. Still two copies, but drift becomes
visible and reviewable.

C. Keep the orphan branch, stop duplicating. Delete docs/ from
main/develop and repoint README.md, CONTRIBUTING.md and the
in-code references at the published site URLs. Least work, but loses
offline docs next to the code.

Whichever is chosen, the four main-only files above need a decision:
either port them to docs or delete them.

Evidence

Reproduce with:

git fetch origin
comm -3 <(git ls-tree -r --name-only origin/main -- docs/ | sort) \
        <(git ls-tree -r --name-only origin/docs -- docs/ | sort)

for f in $(git ls-tree -r --name-only origin/main -- docs/); do
  if git cat-file -e "origin/docs:$f" 2>/dev/null; then
    a=$(git rev-parse "origin/main:$f"); b=$(git rev-parse "origin/docs:$f")
    [ "$a" != "$b" ] && echo "DIVERGENT $f"
  fi
done

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions