Skip to content

docs: install the [agent] extra in README and correct a baseline comment - #403

Merged
Paulo Lacerda (placerda) merged 1 commit into
developfrom
chore/readme-agent-extra
Aug 8, 2026
Merged

docs: install the [agent] extra in README and correct a baseline comment#403
Paulo Lacerda (placerda) merged 1 commit into
developfrom
chore/readme-agent-extra

Conversation

@placerda

Copy link
Copy Markdown
Contributor

Two text-only corrections found while auditing the published docs. No behavior changes, no new tests needed.

1. README sends readers down an incomplete install

README.md opens with:

python -m pip install agentops-accelerator
agentops init

and then links straight to the Prompt Agent and Hosted Agent tutorials. Both tutorials use commands that live behind the [agent] extra (agentops cockpit, agentops agent serve, the observability commands), so a reader who follows the README verbatim gets an ImportError partway through the tutorial.

[agent] carries fastapi, uvicorn, azure-monitor-query, and the azure-mgmt-* SDKs. agentops init, agentops eval, and agentops doctor work without it, because azure-identity and azure-ai-evaluation are base dependencies. The published tutorials were already corrected to install the extra; this brings the README in line and states which commands need it.

2. A code comment claims --baseline gates, and it does not

The comment above _PR_BASELINE_PATH in src/agentops/services/cicd.py says passing --baseline makes "threshold comparisons run against the committed baseline".

That is not what happens:

# src/agentops/pipeline/orchestrator.py
def exit_code_from(result: RunResult) -> int:
    return 0 if result.summary.overall_passed else 2

overall_passed is decided by the configured thresholds. pipeline/comparison.py produces regressed, current_passed, baseline_passed, and baseline_overall_passed for the report, and none of them reach the exit code. A run that regresses against the baseline but still clears its thresholds exits 0 and the PR gate stays green.

This matters because the comment is what a reader consults when deciding whether committing a baseline buys them a regression gate. It does not. The comment now says so and points at the function that decides.

The same misstatement existed in the published tutorial and was corrected separately in #394.

Two text-only corrections. No behavior changes.

README's Get started block installs plain `agentops-accelerator`, then
sends the reader straight to the Prompt Agent and Hosted Agent tutorials.
Both tutorials use commands that live behind the `[agent]` extra, so a
reader who follows the README verbatim hits an ImportError partway
through. The published docs were corrected to install the extra; this
brings the README in line and says which commands need it.

The comment above `_PR_BASELINE_PATH` in services/cicd.py claims passing
`--baseline` makes "threshold comparisons run against the committed
baseline". It does not. `exit_code_from` in pipeline/orchestrator.py
returns 0 or 2 purely from `result.summary.overall_passed`, which the
configured thresholds decide. comparison.py produces `regressed`,
`current_passed`, and `baseline_passed` for the report and nothing that
reaches the exit code. A run that regresses against the baseline but
still clears its thresholds exits 0. The comment now says so, since it
is the thing a reader consults when deciding whether committing a
baseline gives them a gate.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bcb9c0b6-d506-46dc-90d2-8120413166ee
@placerda
Paulo Lacerda (placerda) merged commit a2b491c into develop Aug 8, 2026
13 checks passed
@placerda
Paulo Lacerda (placerda) deleted the chore/readme-agent-extra branch August 8, 2026 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant