Skip to content

ci(profiling): opt crashtracker riot into Python 3.15 - #20007

Closed
vlad-scherbich wants to merge 1 commit into
mainfrom
vlad/315-profiling-crashtracker-315
Closed

ci(profiling): opt crashtracker riot into Python 3.15#20007
vlad-scherbich wants to merge 1 commit into
mainfrom
vlad/315-profiling-crashtracker-315

Conversation

@vlad-scherbich

@vlad-scherbich vlad-scherbich commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Description

Delta vs #19269: opts the crashtracker riot environment into Python 3.15 and adds its matching lockfile coverage.

This changes CI environment coverage only. It does not change crashtracker implementation, add the native build, widen requires-python, or schedule profiling on Python 3.15.

Layer Meaning Which PR
Compiled Native prerequisite #19269
Armed Crashtracker CI environment coverage this PR
Observable No standalone support claim

Testing

Risks

Additional Notes

changelog/no-changelog.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

.riot/requirements/1857594.txt                                          @DataDog/apm-python
riotfile.py                                                             @DataDog/apm-python

@vlad-scherbich vlad-scherbich added the changelog/no-changelog A changelog entry is not required for this PR. label Sep 1, 2026
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Sep 1, 2026

Copy link
Copy Markdown

Circular import analysis

⚠️ Existing circular imports

There are 1 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Sep 1, 2026

Copy link
Copy Markdown

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 230 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 230 highest severity)
ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=134)
ddtrace.llmobs._evaluators.runner -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=132)
ddtrace.llmobs._integrations.openai -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=132)
ddtrace.debugging._signal.model -×-> ddtrace.trace  (product:debugging -> product:tracing, score=132)
ddtrace.llmobs._integrations.vertexai -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=132)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Pipelines  Tests

⚠️ Warnings

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 5 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-py | compute_pipeline — 🔄 Retry may pass, looks flaky

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-py | os tests windows: [3.12] — 🔄 Retry may pass, looks flaky

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-py | contrib/integration_registry 1/1

View more details · View in GitLab

View all 5 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 6591316 | Docs | View more details | Give us feedback!

@pr-commenter

pr-commenter Bot commented Sep 1, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-09-09 20:48:14

Comparing candidate commit 6591316 in PR branch vlad/315-profiling-crashtracker-315 with baseline commit 4b843db in branch main.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 589 metrics, 10 unstable metrics, 7 known flaky benchmarks, 17 flaky benchmarks without significant changes.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:coreapiscenario-context_with_data_listeners

  • unstable execution_time [-748.876ns; +714.238ns] or [-6.782%; +6.469%]

scenario:coreapiscenario-core_dispatch_1_listener

  • unstable execution_time [-29.408ns; +36.773ns] or [-4.795%; +5.996%]

scenario:coreapiscenario-core_dispatch_50_listeners

  • unstable execution_time [-1689.885ns; +1587.656ns] or [-9.884%; +9.286%]

scenario:coreapiscenario-core_dispatch_exception_listeners

  • unstable execution_time [-1214.080ns; +1237.045ns] or [-9.276%; +9.451%]

scenario:coreapiscenario-core_dispatch_listeners

  • unstable execution_time [-340.747ns; +315.021ns] or [-9.246%; +8.548%]

scenario:coreapiscenario-core_dispatch_no_args_listeners

  • unstable execution_time [-246.242ns; +259.518ns] or [-8.424%; +8.879%]

scenario:coreapiscenario-core_dispatch_with_results_1_listener

  • unstable execution_time [-68.201ns; +85.927ns] or [-5.929%; +7.470%]

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

  • unstable execution_time [-4178.408ns; +3961.467ns] or [-10.100%; +9.575%]

scenario:coreapiscenario-core_dispatch_with_results_listeners

  • unstable execution_time [-706.126ns; +830.571ns] or [-8.712%; +10.247%]

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

  • unstable execution_time [-53.075µs; +58.162µs] or [-9.144%; +10.020%]

Known flaky benchmarks

These benchmarks are marked as flaky and will not trigger a failure. Modify FLAKY_BENCHMARKS_REGEX to control which benchmarks are marked as flaky.

scenario:httppropagationinject-ids_only

  • 🟥 execution_time [+2.085µs; +2.255µs] or [+12.404%; +13.419%]

scenario:iastaspects-upper_noaspect

  • 🟥 execution_time [+13.272µs; +16.754µs] or [+7.900%; +9.973%]

scenario:iastaspectsospath-ospathbasename_aspect

  • 🟥 execution_time [+130.457µs; +136.664µs] or [+33.493%; +35.087%]

scenario:iastaspectssplit-rsplit_aspect

  • 🟥 execution_time [+21.047µs; +24.719µs] or [+15.186%; +17.836%]

scenario:span-start

  • 🟥 execution_time [+1.475ms; +1.629ms] or [+10.148%; +11.205%]

scenario:telemetryaddmetric-1-count-metric-1-times

  • 🟥 execution_time [+493.310ns; +539.862ns] or [+17.763%; +19.439%]

scenario:tracer-small

  • 🟥 execution_time [+37.964µs; +40.021µs] or [+11.628%; +12.258%]

Known flaky benchmarks without significant changes:

  • scenario:errortrackingflasksqli-baseline
  • scenario:flasksimple-iast-get
  • scenario:iastaspects-casefold_aspect
  • scenario:iastaspects-casefold_noaspect
  • scenario:iastaspects-index_aspect
  • scenario:iastaspects-ljust_noaspect
  • scenario:iastaspects-lower_aspect
  • scenario:iastaspects-replace_aspect
  • scenario:iastaspects-rstrip_aspect
  • scenario:iastaspects-swapcase_aspect
  • scenario:iastaspects-title_noaspect
  • scenario:iastaspects-translate_aspect
  • scenario:iastaspects-translate_noaspect
  • scenario:packagespackageforrootmodulemapping-cache_off
  • scenario:packagespackageforrootmodulemapping-cache_on
  • scenario:sethttpmeta-all-enabled
  • scenario:telemetryaddmetric-record-100-metrics

@vlad-scherbich
vlad-scherbich force-pushed the vlad/ddtracepy-315-profiling-collectors branch from 989dea6 to 551605d Compare September 2, 2026 16:09
@vlad-scherbich
vlad-scherbich force-pushed the vlad/315-profiling-crashtracker-315 branch from 25e8773 to fc8ece4 Compare September 2, 2026 16:10
@vlad-scherbich
vlad-scherbich force-pushed the vlad/ddtracepy-315-profiling-collectors branch from 551605d to 33d1ad8 Compare September 2, 2026 19:10
@vlad-scherbich
vlad-scherbich force-pushed the vlad/315-profiling-crashtracker-315 branch from fc8ece4 to 15cf697 Compare September 2, 2026 19:10
Base automatically changed from vlad/ddtracepy-315-profiling-collectors to vlad/ddtracepy-315-profiling-native September 2, 2026 19:23
@vlad-scherbich
vlad-scherbich force-pushed the vlad/315-profiling-crashtracker-315 branch from 15cf697 to 3009a00 Compare September 2, 2026 19:34
@vlad-scherbich
vlad-scherbich force-pushed the vlad/ddtracepy-315-profiling-native branch from df5e2f7 to 6d624f8 Compare September 2, 2026 20:15
@vlad-scherbich
vlad-scherbich force-pushed the vlad/315-profiling-crashtracker-315 branch 2 times, most recently from 4586375 to f09e435 Compare September 3, 2026 02:10
@vlad-scherbich
vlad-scherbich force-pushed the vlad/ddtracepy-315-profiling-native branch from 79b8faf to ddba58a Compare September 4, 2026 18:57
@vlad-scherbich
vlad-scherbich force-pushed the vlad/315-profiling-crashtracker-315 branch from f09e435 to 514756f Compare September 4, 2026 19:02
Base automatically changed from vlad/ddtracepy-315-profiling-native to main September 9, 2026 14:25
@vlad-scherbich
vlad-scherbich force-pushed the vlad/315-profiling-crashtracker-315 branch 2 times, most recently from bf3e79f to 6591316 Compare September 9, 2026 20:18
@vlad-scherbich
vlad-scherbich requested a lite review from Copilot September 9, 2026 20:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The change is limited to CI environment coverage and the added lockfile aligns with existing Riot requirements conventions.

Pull request overview

This PR adjusts dd-trace-py’s Riot CI matrix to explicitly include Python 3.15 for the crashtracker test environment, and adds the corresponding pip-compile lockfile so dependency hashing stays consistent across supported versions.

Changes:

  • Opt the crashtracker Riot environment into Python 3.15 (without changing default max Python for other suites).
  • Add a new .riot/requirements lockfile generated with Python 3.15 for the crashtracker environment.
File summaries
File Description
riotfile.py Extends the crashtracker venv Python version selection to include 3.15 explicitly.
.riot/requirements/1857594.txt Adds the Python 3.15 pip-compile output matching the new crashtracker matrix coverage.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread riotfile.py
Venv(
pys=select_pys(min_version="3.12"),
# TODO(py-315): 3.15 explicitly opted in for crashtracker native validation.
pys=select_pys(min_version="3.12", max_version="3.14") + ["3.15"],
@vlad-scherbich

Copy link
Copy Markdown
Contributor Author

Closing in favor of #20203, which already covers crashtracker bump.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/no-changelog A changelog entry is not required for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants