Skip to content

feat(cli): add agent-friendly diagnostic output - #4127

Merged
codeshaunted merged 4 commits into
canaryfrom
avery/agent-diagnostics
Jul 22, 2026
Merged

feat(cli): add agent-friendly diagnostic output#4127
codeshaunted merged 4 commits into
canaryfrom
avery/agent-diagnostics

Conversation

@codeshaunted

@codeshaunted codeshaunted commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Summary

  • centralize output preset, color, hyperlink, and diagnostic-format resolution
  • select compact diagnostics automatically in known coding-agent environments
  • add an agent renderer with exact ranges and primary, secondary, and related locations
  • expose granular CLI and environment overrides while preserving the existing human renderer

Example

type_malformed.baml:7:6-7:9 error[E0010]: Expected =, found identifier

Configuration

  • --output-preset / BAML_OUTPUT_PRESET
  • --color / BAML_COLOR
  • --hyperlinks / BAML_HYPERLINKS
  • --diagnostic-format / BAML_DIAGNOSTIC_FORMAT

Tests

  • cargo test -p baml_compiler_diagnostics
  • cargo test -p baml_cli --lib
  • cargo clippy -p baml_compiler_diagnostics -p baml_cli --all-targets -- -D warnings
  • cargo fmt --all -- --check

Linear: B-407

Summary by CodeRabbit

  • New Features

    • Added configurable CLI output presets with environment-variable support, including an agent-optimized diagnostic format.
    • Color and hyperlink behavior are now driven by a unified output policy with sensible automatic defaults (or explicit overrides).
    • Hyperlinks and styling adapt per output stream (stdout vs stderr) based on terminal capability.
  • Bug Fixes

    • Standardized diagnostic rendering configuration across check, generate, pack, and cache-related validations for more consistent output.

@vercel

vercel Bot commented Jul 22, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
beps Ready Ready Preview, Comment Jul 22, 2026 10:56pm
promptfiddle2 Ready Ready Preview, Comment Jul 22, 2026 10:56pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
promptfiddle Skipped Skipped Jul 22, 2026 10:56pm

Request Review

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: f02291fb-330f-45a0-9a7c-c341cf7f8d67

📥 Commits

Reviewing files that changed from the base of the PR and between 69721b0 and 0fd9bb4.

📒 Files selected for processing (5)
  • baml_language/crates/baml_cli/src/check_command.rs
  • baml_language/crates/baml_cli/src/generate.rs
  • baml_language/crates/baml_cli/src/output.rs
  • baml_language/crates/baml_cli/src/pack_command.rs
  • baml_language/crates/baml_cli/src/paint.rs
🚧 Files skipped from review as they are similar to previous changes (3)
  • baml_language/crates/baml_cli/src/generate.rs
  • baml_language/crates/baml_cli/src/pack_command.rs
  • baml_language/crates/baml_cli/src/check_command.rs

📝 Walkthrough

Walkthrough

The CLI centralizes output configuration through presets and environment-backed overrides, supports agent-formatted diagnostics, routes command diagnostics through the resolved policy, and tracks color and hyperlinks independently.

Changes

CLI output and diagnostics

Layer / File(s) Summary
Agent diagnostic rendering
baml_language/crates/baml_compiler_diagnostics/src/render.rs
Adds compact agent diagnostics with disambiguated paths, related locations, error codes, and UTF-8-aware columns.
Output policy resolution
baml_language/crates/baml_cli/src/output.rs, baml_language/crates/baml_cli/src/lib.rs, baml_language/crates/baml_cli/Cargo.toml
Adds preset, color, hyperlink, and diagnostic-format controls with CLI and environment-variable wiring, runtime resolution, and tests.
CLI integration and rendering call sites
baml_language/crates/baml_cli/src/commands.rs, baml_language/crates/baml_cli/src/paint.rs, baml_language/crates/baml_cli/src/check_command.rs, baml_language/crates/baml_cli/src/generate.rs, baml_language/crates/baml_cli/src/pack_command.rs, baml_language/crates/baml_cli/src/diagnostics_cache.rs
Initializes the unified policy, separates painter color and hyperlink behavior, and uses resolved diagnostic rendering across CLI commands.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant RuntimeCli
  participant output_init
  participant Painter
  participant render_diagnostics
  RuntimeCli->>output_init: init(OutputArgs)
  output_init->>Painter: install stream color state
  Painter->>output_init: query stream color and hyperlink policy
  render_diagnostics->>output_init: obtain diagnostic render configuration
Loading

Possibly related PRs

Suggested reviewers: rossirpaulo

Poem

I’m a rabbit with paths in my paws,
Compact diagnostics follow new laws.
Colors and links now hop side by side,
Agent-friendly output takes stride.
Presets bloom where old flags once grew—
A tidy CLI says, “thump-thump” to you!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately summarizes the main change: agent-friendly diagnostic output in the CLI.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch avery/agent-diagnostics

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@vercel
vercel Bot temporarily deployed to Preview – beps July 22, 2026 21:32 Inactive
@github-actions

Copy link
Copy Markdown

⏭️ Performance benchmarks were skipped

Perf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to canary/main.

To run them on this PR, do any of the following, then push a commit (or re-run CI):

  • Add RUN_CODSPEED=1 to the PR description, or
  • Include run-perf or /perf in the PR title or any commit message.

@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 July 22, 2026 21:39 Inactive
@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown

Binary size checks passed

7 passed

Artifact Platform File Gzip Gated on Baseline Delta Status
baml-cli Linux 🔒 25.6 MB 10.9 MB file 25.3 MB +346.0 KB (+1.4%) OK
packed-program Linux 🔒 17.2 MB 7.1 MB file 17.0 MB +161.3 KB (+0.9%) OK
baml-cli macOS 🔒 19.9 MB 9.5 MB file 19.6 MB +281.3 KB (+1.4%) OK
packed-program macOS 🔒 13.4 MB 6.2 MB file 13.2 MB +198.3 KB (+1.5%) OK
baml-cli Windows 🔒 21.4 MB 9.7 MB file 21.1 MB +296.4 KB (+1.4%) OK
packed-program Windows 🔒 14.3 MB 6.3 MB file 14.2 MB +158.2 KB (+1.1%) OK
bridge_wasm WASM 16.2 MB 🔒 4.4 MB gzip 4.4 MB +11.5 KB (+0.3%) OK

🔒 = the size this artifact is GATED on (ceiling + delta). Binaries gate on file size (installed binary); WASM gates on gzip (download size). The other size is shown for information only.


Generated by cargo size-gate · workflow run

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
baml_language/crates/baml_cli/src/output.rs (1)

232-246: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Generic agent env-var names risk false-positive auto-detection.

AGENT_ENV_VARS includes bare names like "AGENT" and "AI_AGENT", which are plausible generic environment variable names unrelated to coding agents (e.g., set by unrelated CI/build-agent tooling or user scripts). A false positive here silently switches OutputPreset::Auto to the no-color/no-hyperlink Agent preset for ordinary human/CI runs, which is subtle to diagnose since it's controlled entirely by ambient environment rather than an explicit flag.

Consider requiring one of the more specific/product-scoped vars, or at least documenting the false-positive risk so downstream users know --output-preset human overrides it.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@baml_language/crates/baml_cli/src/output.rs` around lines 232 - 246, Update
AGENT_ENV_VARS and running_in_agent so auto-detection relies only on specific
coding-agent or product-scoped environment variables, removing generic names
such as AGENT and AI_AGENT. Preserve explicit --output-preset human override
behavior and the existing truthy-variable detection.
baml_language/crates/baml_compiler_diagnostics/src/render.rs (1)

497-561: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Consider caching shortest_unique_path per render batch.

shortest_unique_path is recomputed from scratch (iterating the whole file_paths map per path-component) for every span, with no memoization across a render_diagnostics call — unlike the Ariadne path in the same function, which explicitly hoists SourceCache construction out of the loop specifically to avoid "a large fraction of baml check wall time" on warning-heavy projects (per the comment on render_diagnostics, lines 219-227). For large multi-file projects with many diagnostics/annotations, this repeats an O(files × path-depth) computation per span.

Consider threading a small HashMap<FileId, String> cache through render_agent/render_diagnostics (mirroring the SourceCache pattern) to compute each file's disambiguated suffix once per batch.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@baml_language/crates/baml_compiler_diagnostics/src/render.rs` around lines
497 - 561, Cache the result of shortest_unique_path once per render batch
instead of recomputing it for every span. Thread a HashMap<FileId, String>
through render_diagnostics and render_agent, populate it on first use, and have
format_span reuse the cached path while preserving the existing fallback and
disambiguation behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@baml_language/crates/baml_cli/src/output.rs`:
- Around line 232-246: Update AGENT_ENV_VARS and running_in_agent so
auto-detection relies only on specific coding-agent or product-scoped
environment variables, removing generic names such as AGENT and AI_AGENT.
Preserve explicit --output-preset human override behavior and the existing
truthy-variable detection.

In `@baml_language/crates/baml_compiler_diagnostics/src/render.rs`:
- Around line 497-561: Cache the result of shortest_unique_path once per render
batch instead of recomputing it for every span. Thread a HashMap<FileId, String>
through render_diagnostics and render_agent, populate it on first use, and have
format_span reuse the cached path while preserving the existing fallback and
disambiguation behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 696ca694-88c2-4373-86f4-550ca09002df

📥 Commits

Reviewing files that changed from the base of the PR and between f1c20aa and 57f2940.

📒 Files selected for processing (10)
  • baml_language/crates/baml_cli/Cargo.toml
  • baml_language/crates/baml_cli/src/check_command.rs
  • baml_language/crates/baml_cli/src/commands.rs
  • baml_language/crates/baml_cli/src/diagnostics_cache.rs
  • baml_language/crates/baml_cli/src/generate.rs
  • baml_language/crates/baml_cli/src/lib.rs
  • baml_language/crates/baml_cli/src/output.rs
  • baml_language/crates/baml_cli/src/pack_command.rs
  • baml_language/crates/baml_cli/src/paint.rs
  • baml_language/crates/baml_compiler_diagnostics/src/render.rs

@vercel
vercel Bot temporarily deployed to Preview – promptfiddle July 22, 2026 21:54 Inactive
@vercel
vercel Bot temporarily deployed to Preview – beps July 22, 2026 22:00 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 July 22, 2026 22:07 Inactive
@vercel
vercel Bot temporarily deployed to Preview – beps July 22, 2026 22:09 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 July 22, 2026 22:17 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle July 22, 2026 22:29 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle July 22, 2026 22:49 Inactive
@vercel
vercel Bot temporarily deployed to Preview – beps July 22, 2026 22:50 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 July 22, 2026 22:56 Inactive
@codeshaunted
codeshaunted added this pull request to the merge queue Jul 22, 2026
Merged via the queue into canary with commit 731964c Jul 22, 2026
67 checks passed
@codeshaunted
codeshaunted deleted the avery/agent-diagnostics branch July 22, 2026 23:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant