feat(appkit): mlflow tracing for agents (stack 1/5)#477
Open
MarioCadenas wants to merge 8 commits into
Open
Conversation
Trace agent turns and tool calls to MLflow via the mlflow-tracing SDK. Adds an optional 'experiment' resource to the agents plugin; when bound (MLFLOW_EXPERIMENT_ID), each turn opens an AGENT span and each tool call a nested TOOL span, with auth resolved from the app's Databricks credentials. A turn's trace can be linked to an evaluation run via mlflow.sourceRun. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
currentTraceId() used getLastActiveTraceId(), which is only set when a root span ends (on export). Read mid-turn it returned the previous turn's id — or another turn's under concurrency. Read the context-active span instead, matching linkTraceToRun. Also read mlflowRunId from the validated schema (capped at 64 chars) instead of casting raw req.body, and drop two unused imports. Adds tracing tests covering trace-id source and output auto-capture. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
# Conflicts: # pnpm-lock.yaml
Contributor
📦 Bundle size reportCompared against
|
| dist | raw | gzip |
|---|---|---|
| JS (runtime) | 734 KB (+6.1 KB) | 257 KB (+2.3 KB) |
| Type declarations | 275 KB | 94 KB (-1 B) |
| Source maps | 1.4 MB (+11 KB) | 478 KB (+3.9 KB) |
| Other | 11 KB | 3.7 KB |
| Total | 2.4 MB (+17 KB) | 832 KB (+6.2 KB) |
Per-entry composition (own code — deps external (as shipped))
| Entry | Initial (gz) | Lazy (gz) | Total (gz) | node_modules (min) | Own code (min) |
|---|---|---|---|---|---|
. |
79 KB | 2.5 KB | 81 KB | external | 258 KB |
./beta |
40 KB (+763 B) | 230 B | 40 KB (+763 B) | external | 119 KB (+2.0 KB) |
./type-generator |
19 KB | 0 B | 19 KB | external | 54 KB |
Chunks:
| Entry | Chunk | Load | Size (gz) |
|---|---|---|---|
. |
index.js |
initial | 75 KB |
. |
utils.js |
initial | 4.0 KB |
. |
remote-tunnel-manager.js |
lazy | 2.5 KB |
./beta |
beta.js |
initial | 31 KB |
./beta |
databricks.js |
initial | 5.7 KB |
./beta |
service-context.js |
initial | 3.2 KB |
./beta |
client-options.js |
initial | 220 B |
./beta |
databricks.js |
lazy | 127 B |
./beta |
index.js |
lazy | 103 B |
./type-generator |
index.js |
initial | 19 KB |
@databricks/appkit-ui
npm tarball (packed): 295 KB — gzipped download (dist + bin; excludes release-only docs/NOTICE).
| dist | raw | gzip |
|---|---|---|
| JS (runtime) | 350 KB | 116 KB |
| Type declarations | 201 KB | 72 KB |
| Source maps | 669 KB | 218 KB |
| CSS | 16 KB | 3.3 KB |
| Total | 1.2 MB | 410 KB |
Per-entry composition (consumer bundle — deps bundled, peerDeps external)
| Entry | Initial (gz) | Lazy (gz) | Total (gz) | node_modules (min) | Own code (min) |
|---|---|---|---|---|---|
./js |
4.3 KB | 49 KB | 54 KB | 208 KB | 12 KB |
./js/beta |
20 B | 0 B | 20 B | 0 B | 0 B |
./react |
428 KB | 49 KB | 476 KB | 1.3 MB | 163 KB |
./react/beta |
20 B | 0 B | 20 B | 0 B | 0 B |
Chunks:
| Entry | Chunk | Load | Size (gz) |
|---|---|---|---|
./js |
index.js |
initial | 4.2 KB |
./js |
chunk |
initial | 120 B |
./js |
apache-arrow |
lazy | 49 KB |
./js/beta |
beta.js |
initial | 20 B |
./react |
index.js |
initial | 426 KB |
./react |
tslib |
initial | 2.1 KB |
./react |
apache-arrow |
lazy | 49 KB |
./react/beta |
beta.js |
initial | 20 B |
/invocations and /responses drove tool calls through traceTool but had no traceAgent wrapper, so with tracing enabled each tool call became an orphan root TOOL trace instead of nesting under a turn. Wrap the adapter run in traceAgent, mirroring the streaming path, so both surfaces produce the same trace shape. Also drop a dead flushTraces mock left from the flushAgentTraces removal. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack 1/5 · targets
main.Adds MLflow tracing to the agents plugin. Agent turns and tool calls are traced to a bound MLflow experiment via the
mlflow-tracingSDK (OpenTelemetry under the hood). Tracing is a no-op unless the plugin's optionalexperimentresource is set (MLFLOW_EXPERIMENT_ID); auth is resolved from the app's own Databricks credentials, so no tokens or OTLP headers are wired by hand.withAgentSpanwraps each turn (AGENT span) and tool dispatch (TOOL span, auto-nested).mlflow.traceNametag.experimentoptional resource in the agents manifest.This is the base of a 5-PR stack that builds out an agent evaluation framework. Reviewable on its own — touches only the agents plugin.