Build-std: Add builtin dependencies by adamgemmell · Pull Request #16675 · rust-lang/cargo

adamgemmell · 2026-02-25T17:40:41Z

What does this PR try to resolve?

This PR internally introduces:

The builtin source, a way for Cargo to discover builtin packages
Opaque dependencies, the idea that a dependency shouldn't be handled by the main resolve but rather a separate resolve in order to allow things like multiple packages in the same resolve with differing minor versions. The concept has been proposed as a first-class cargo feature, but is only introduced partially here.
(Implicit) builtin dependencies, which are dependencies on builtin packages injected in order to allow packages to migrate to explicit builtin dependencies in the future.

It moves the -Zbuild-std implementation to use them as part of rust-lang/rfcs#3875. While not strictly required for build-std=always which is our immediate goal, I wanted to ensure the implementation moving forward had explicit dependencies in mind.

The main behaviour change in this PR is that host dependencies like build-scripts and proc_macros no longer use build-std. If this is desired we can re-introduce the behaviour in an unstable manner, but it never worked for -Zbuild-std in cross-compile mode.

This PR implements this as a whole by:

Registering the builtin source (lazily, when a dependency is discovered), which discovers packages with a RecursivePathSource.
Creating a default set of opaque implicit builtin dependencies the set of roots defined in std_crates, and injects them to every eligible crate during the resolve.
The builtin source, when queried for a package satisfying one of these dependencies, returns a builtin Summary which has been modified compared to the one discovered by RecursivePathSource. This package has no dependencies, because the dependency queried is opaque. This allows the resolve to succeed as normal without modifying the resolver itself. The intended behaviour, though yet untested, is to make use of all the patching/replacement logic already in the registry query.
Instead of the current -Zbuild-std unit generation step which attaches UnitDeps to each root of the std resolve, the main unit generation loop replaces any builtin Summarys with roots from the separate std resolve. Dependencies of the roots are then added afterwards.

This PR has a very large surface area and will inevitably change some behaviour of -Zbuild-std. In particular we need to be careful about where we expose builtin SourceIds and package specs. The intent here is to avoid disruption as build-std is a very commonly used unstable feature, but some is to be expected and will be fixed in future PRs according to the agreed RFC. The non-build-std route is well tested already and I do not expect to find regressions there as the majority of this code is gated on -Zbuild-std.

How to test and review this PR?

The PR is best read commit-by-commit - all commits are atomic and try to introduce features piecemeal where possible, though somewhat inevitably there's a big "waterfall" enabling commit in the middle. A commit 5bfab35 early in the branch adds a test showing that the unit-graph does not change compared to the old -Zbuild-std behaviour.

The behaviour can be manually tested locally by providing -Zbuild-std as a flag to any cargo command.

davidtwco · 2026-02-26T17:38:40Z

We aren't expecting that this will land prior to those RFCs being approved (unless Cargo want it to), the intent with this is just to get some feedback on the general direction for the implementation, we're happy to adjust as much or as little as required :) @adamgemmell will also be away for a couple of weeks starting next week, so we might not respond to feedback immediately.

epage · 2026-02-26T18:00:05Z

    /// A directory-based registry.
    Directory,
+    /// Package sources distributed with the rust toolchain
+    Builtin,


https://github.com/rust-lang/cargo/blob/master/crates/cargo-util-schemas/src/core/package_id_spec.rs is at least one other place that would need updating

This will impact the unique identifier for the packages from this source in cargo's json output when compiling, cargo metadata, cargo <cmd> -p, etc

I'll modify there too, and add a note to check the stdout in various use cases. The RFCs often make notes on what the output of various commands will be. Note that builtin doesn't actually appear in Units - they're all Path dependencies by that point.

An interesting point on cargo metadata is that we decided that we have an unresolved question regarding if deps of builtins should be shown on output, which will be a little hard here as they're not attached until unit generation.

I've opted to implement pkg spec input/output in a later PR. I've added tests for common output commands like metadata/tree. json output (from build --message-format=json) isn't impacted in this PR - builtins do not exist in the unit graph (see this test which isn't changed in the rest of the branch c1f8964). I plan to address that in a future PR but I'm not sure on the best way to do that at this stage.

epage · 2026-02-26T18:01:56Z

+                    .url
+                    .to_file_path()
+                    .expect("builtin sources should not be remote");
+                Ok(Box::new(PathSource::new(&path, self, gctx)))


Will using a PathSsource directly like this work?

This particularly could have interesting design questions

Moved this to a builtin source

A builtin source now wraps a RecursivePathSource, and there's only one source for all builtin packages. see 4832d04

epage · 2026-02-26T18:10:41Z

+        None
+    } else {
+        Some(builtins.iter())
+    };


I assume this is for implicit builtins. Is there a reason you chose to do this here?

Source replacements happen in dep_cache (see query) so it seemed simpler just to put this here for now. After digging a little more, it might just be the deprecated [replace] section though, in which case I can probably lift this out of the cache to a more appropriate place

This is in build_deps() now (see 1588221), which is a bit more appropriate. It could be moved up to the resolver activate function, but that's already massive, and moving it to PackageRegistry::query would involve modifying Summarys which are intended to be immutable.

epage · 2026-02-26T18:16:02Z

+        }
+    }
+
+    pub fn new_injected_builtin(name: InternedString) -> Dependency {


what do you see as the role of this compared to the other news?

The difference is that opaque is true (making a SourceId was also more complicated in a previous iteration of this patch). public should also be true now that I look closer.

Reworked, see 1588221#diff-3f0dca17873e498a977bdb06abeb78a4f4135fbbf8abf6323332fd0f8a3da1bd

epage · 2026-02-26T18:18:28Z

                )
            })?;

+        if dep.is_opaque() {


I'd like to find a way to ask the source for the opaque variant of the summary. The tricky thing will be that we need to work with both variants.

This pattern works well, see the new Summary::to_opaque_builtin_summary()

Reworked again to make this more like the other constructors: 4832d04

epage · 2026-02-26T18:20:20Z

+        // Injecting builtins earlier (somewhere with access to RustcTargetData) is needed instead of this
+        let home = std::env::var("HOME").expect("HOME is set");
+        let path = format!(
+            "file://{home}/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/"


The URL in source ids is public facing. I think we'll need something more generic and then a new Source

I definitely tried something more generic but found that it had to be valid at various points. Will experiment more with this.

I have hidden the URL in pkgids as per https://rust-lang.github.io/rfcs/3875-build-std-explicit-dependencies.html#cargo-subcommands:~:text=builtin%3A%2F%2F%2E%23core, but it may still leak out in various places. I plan to address this in future PRs that test individual subcommands' behaviour.

epage · 2026-02-26T18:21:35Z

            has_dev_units,
            crate::core::resolver::features::ForceAllTargets::No,
            dry_run,
+            true,


not really a fan of bool constants being used in parameter lists. Makes it a lot harder to figure what what is going on here

I've moved this specific bool, but there's probably scope to improve the passing around of the builtins path and dependencies set

I added this information to ResolveOpts, see 707ac6b

epage · 2026-02-26T18:22:06Z

    keep_previous: Option<Keep<'_>>,
    specs: &[PackageIdSpec],
    register_patches: bool,
+    inject_builtins: bool,


Why do we need to tunnel this through? Not thrilled with this

This bool or the required logic behind it could probably be packaged into the workspace or it's gctx

I added this information to ResolveOpts, see 707ac6b

adamgemmell

Thank you for leaving feedback. My latest set of changes does a bit of tidying and works towards getting the tests passing, but doesn't address these comments yet or otherwise make progress towards removing hacks.

View changes since this review

adamgemmell · 2026-03-18T17:20:22Z

+        None
+    } else {
+        Some(builtins.iter())
+    };


Source replacements happen in dep_cache (see query) so it seemed simpler just to put this here for now. After digging a little more, it might just be the deprecated [replace] section though, in which case I can probably lift this out of the cache to a more appropriate place

adamgemmell · 2026-03-18T17:24:10Z

+        }
+    }
+
+    pub fn new_injected_builtin(name: InternedString) -> Dependency {


The difference is that opaque is true (making a SourceId was also more complicated in a previous iteration of this patch). public should also be true now that I look closer.

adamgemmell · 2026-03-18T17:32:31Z

+        // Injecting builtins earlier (somewhere with access to RustcTargetData) is needed instead of this
+        let home = std::env::var("HOME").expect("HOME is set");
+        let path = format!(
+            "file://{home}/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/"


I definitely tried something more generic but found that it had to be valid at various points. Will experiment more with this.

adamgemmell · 2026-03-19T10:40:20Z

    keep_previous: Option<Keep<'_>>,
    specs: &[PackageIdSpec],
    register_patches: bool,
+    inject_builtins: bool,


This bool or the required logic behind it could probably be packaged into the workspace or it's gctx

This commit is currently an unused API and will be tested in later commits

… update status messages

generation for them The old unit generation used to insert dependencies to every package on the set of roots std was resolved with. The new unit generation replaces dependencies with the relevant root of std, or ignores them if the dependency is not needed. This commit enables tests for the previous set of commits.

This test's assertion was completely broken because of the leading spaces and use of `...` which doesn't work with stderr_does_not_contain. See the comments in this test for additional details of shared dependencies on why the test cannot be fixed right now. It is feasibly possible for Cargo to share the user dep_test dependency and the build-std dep_test, but this is complex as written as dep_test gains an implicit std dependency while the mock-std depends on it. It may be easier to share them with explicit builtin dependencies, but the situation is complex due to -Zforce-unstable-if-unmarked and prone to breakage due to RUSTC_BOOTSTRAP=1.

rustbot · 2026-08-24T17:14:58Z

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

adamgemmell · 2026-08-24T17:20:11Z

I could split this up into 3 PRs:

SourceKind::Builtin and all the resolver-tests around that, master..2963df7
The Builtin source (which generates opaque Summarys) and implicit builtin dependencies (required in order to test the Source), containing the next few commits up to 7b4c0c9
Enable implicit builtin dependencies for -Zbuild-std, adding the new unit generation steps and a bunch of tests around that to ensure that we don't change anything unintentionally, which is the rest of the branch.

There would still be a strong ordering dependency between these PRs. The issue with going into smaller chunks is testing - a lot of the areas touched don't have unit tests and instead use integration tests, which make it difficult to introduce in isolation. If you're happy with this split let me know and I'll do it (and whether you'd like to be assigned all of the PRs). Otherwise if you'd prefer smaller PRs that might not stand as well on their own I can go that route too.

On your other notes, I've gone over the tests and ensure they use detailed snapshots and are added in their own commits. The difference with the RFC there is intentional, in order to keep this PR a reasonable size I opted to maintain existing -Zbuild-std behaviour and give each subcommand full attention and testing in later PRs.

requested via `-p`

The ideal behaviour is probably to emit all packages, but currently only the roots of std are emitted. A future PR will change this.

epage · 2026-08-25T03:33:49Z

I could split this up into 3 PRs:

There would still be a strong ordering dependency between these PRs. The issue with going into smaller chunks is testing - a lot of the areas touched don't have unit tests and instead use integration tests, which make it difficult to introduce in isolation. If you're happy with this split let me know and I'll do it (and whether you'd like to be assigned all of the PRs). Otherwise if you'd prefer smaller PRs that might not stand as well on their own I can go that route too.

My interest is not in parallel PRs but to reduce the relevant context for reviewing. If I only need to look at one or two commits and follow that conversation, I can do that within my limited time, making us able to move forward a chunk at a time. If I have to have all conversations at once, this PR will continue to sit as-is.

While I generally prefer commits to be atomic and PRs to contain relevant context for why things were done and groups changes that may not be justified on their own, this is large enough to be more flexible on those points (C-ISOLATE winning out over other of my PR guidelines). This is also behind an unstable flag so its not like people will have access to the full feature. This does mean that more effort will need to be put into PR descriptions and/or comments on the code to help explain the larger context for a PR (with maybe keeping this as a draft as a reference point).

Note that I emphasized "commits" and "PRs" because this PR does not seem to follow atomic commits as requested by our contrib guide (e.g. adding unused APIs in 1657629). If we keep commits like these (see below), I would avoid calling them feat or fix as they have no end-user impact.

I would go so far as to say that each atomic "feat" or "fix" commit (and associated test commits) could be their own PR. That is especially the case with the later ones (vendor, tree, metadata). As I said, in similar situations, I've had a PR that is just adding the unstable feature flag and associated docs (basically just naming the flag and linking to the tracking issue since the flag won't do anything yet) and I would recommend doing that here.

Note that I've referred to atomic commits a couple of times. To be clear, commits like 1657629 are not atomic but dead code. Per our contrib guide, we generally want commits to be atomic. This may be a case where the change is big enough that dead code commits are fine as they help make the change more digestible. However, i wouldn't classify these as "feat" or "fix" commits (in their commit message or my guidance on splitting up PRs) because they have no end-user effect.

So to make this concrete, I could see the following PRs

typo fix
unstable feature flag + docs for that flag
ability to declare built-in deps (but unused)
- haven't checked if builtin source can be split out into its own PR
- if the deps have to be stripped when generating a Manifest and Summary to not cause problems further down the line, so be it
resolving of builtin deps
build support
vendor
add
metadata

Give or take (e.g. unsure if summaries could be its own). You are more familiar with this and likely have a better idea on some of the finer points.

As a heads up, I am unsure if the right direction to go is something like 1657629 or having all dependencies mentioned and stripping them later when needed.

epage reviewed Feb 26, 2026

View reviewed changes

Comment thread src/cargo/core/source_id.rs Outdated

epage reviewed Feb 26, 2026

View reviewed changes

biscuitrescue mentioned this pull request Mar 3, 2026

fix: ignore implicit std dependencies in unused-crate-dependencies lint #16677

Merged

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-author Status: The marked PR is awaiting some action (such as code changes) from the PR author. label Mar 6, 2026

adamgemmell force-pushed the dev/adagem01/opaque branch from 16dc1ab to ab7d8b0 Compare March 19, 2026 14:20

adamgemmell commented Mar 19, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

davidtwco mentioned this pull request Apr 14, 2026

build-std rust-lang/goals#274

Open

10 tasks

adamgemmell force-pushed the dev/adagem01/opaque branch from ab7d8b0 to 8607d8c Compare May 7, 2026 15:58

rustbot added A-directory-source Area: directory sources (vendoring) A-workspaces Area: workspaces labels May 7, 2026

adamgemmell added 13 commits August 24, 2026 16:12

test(unit-graph): Capture existing behaviour in unit-graph test

c4ebca0

test(resolver): Model builtin packages

15c1d49

feat(source): Allow specifying builtin sources

d73fed6

feat(resolver): Allow creating builtin dependencies

4c9c5dc

test(resolver): Add negative resolver tests

2963df7

feat(sources): generate opaque summaries to satisfy builtin dependencies

50ef28a

This commit is currently an unused API and will be tested in later commits

feat(resolver): Add directive to enable injecting builtins

1657629

This commit is currently an unused API and will be tested in later commits

test(resolver): Test for implicit builtin deps

e7b1e78

feat(resolver): Inject builtin dependencies

7b4c0c9

test(resolve): Add test that build-std does not affect the lockfile

4f35cdd

test(package-diff): Test that build-std packages do not show in cargo…

20cde82

… update status messages

adamgemmell force-pushed the dev/adagem01/opaque branch from 31580a4 to c7383de Compare August 24, 2026 17:14

adamgemmell force-pushed the dev/adagem01/opaque branch from c7383de to 20197f2 Compare August 24, 2026 17:25

adamgemmell added 8 commits August 24, 2026 18:30

test(build-std): Test when builtins are present as roots, such as when

49a012f

requested via `-p`

feat(unit generation): Replace builtin roots with std roots

a9aed01

test(metadata): Do not emit builtins with cargo metadata

8f0473b

The ideal behaviour is probably to emit all packages, but currently only the roots of std are emitted. A future PR will change this.

fix(metadata): Filter out builtin packages

fd1ca3e

test(build-std): Do not emit builtins with cargo tree

5e9211f

fix(tree): Don't emit builtins

ff222a5

test(vendor): builtins shouldn't be vendored

8c65c05

fix(vendor): Don't vendor builtins

6b31035

adamgemmell force-pushed the dev/adagem01/opaque branch from 20197f2 to 6b31035 Compare August 24, 2026 17:31

This comment was marked as off-topic.

Sign in to view

Uh oh!

Uh oh!

Conversation

adamgemmell commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR try to resolve?

How to test and review this PR?

Uh oh!

davidtwco commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamgemmell Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamgemmell Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamgemmell May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamgemmell Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamgemmell Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamgemmell Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamgemmell Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamgemmell Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamgemmell commented Feb 25, 2026 •

edited

Loading

davidtwco commented Feb 26, 2026 •

edited

Loading

adamgemmell Aug 7, 2026 •

edited

Loading

adamgemmell Aug 7, 2026 •

edited

Loading

adamgemmell May 7, 2026 •

edited

Loading

adamgemmell Aug 7, 2026 •

edited

Loading

adamgemmell Aug 7, 2026 •

edited

Loading

adamgemmell Aug 7, 2026 •

edited

Loading

adamgemmell Aug 7, 2026 •

edited

Loading

adamgemmell Aug 7, 2026 •

edited

Loading

adamgemmell left a comment •

edited by rustbot

Loading