Tracking issue for tools build error: `duplicate artfacts found when compiling a tool`

[mati865]

Recently tools builds on MSVC and GNU Windows platforms began to fail with errors like:

duplicate artfacts found when compiling a tool, this typically means that something was recompiled because a transitive dependency has different features activated than in a previous build:
  winapi 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)
    `clippy-driver` enabled features ["consoleapi", "dbghelp", "errhandlingapi", "fileapi", "handleapi", "ioapiset", "minwindef", "namedpipeapi", "ntdef", "processenv", "processthreadsapi", "std", "winerror", "winnt", "winsock2", "ws2def", "ws2ipdef", "ws2tcpip"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\libwinapi-7cb6d087db077e11.rlib"
    `cargo` enabled features ["consoleapi", "dbghelp", "errhandlingapi", "fileapi", "handleapi", "ioapiset", "jobapi", "jobapi2", "lmcons", "minschannel", "minwinbase", "minwindef", "namedpipeapi", "ntdef", "ntsecapi", "ntstatus", "processenv", "processthreadsapi", "profileapi", "psapi", "schannel", "securitybaseapi", "std", "synchapi", "sysinfoapi", "timezoneapi", "winbase", "wincon", "wincrypt", "winerror", "winnt", "winsock2", "ws2def", "ws2ipdef", "ws2tcpip"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\libwinapi-d6c198bedd24dbbb.rlib"
  backtrace 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)
    `clippy-driver` enabled features ["backtrace-sys", "coresymbolication", "dbghelp", "default", "dladdr", "libbacktrace", "libunwind", "winapi"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\libbacktrace-4d22e1f3c416866d.rlib"
    `cargo` enabled features ["backtrace-sys", "coresymbolication", "dbghelp", "default", "dladdr", "libbacktrace", "libunwind", "winapi"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\libbacktrace-1b3034da488e3e14.rlib"
  error-chain 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)
    `clippy-driver` enabled features ["backtrace", "default", "example_generated"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\liberror_chain-6878f7237b9efb05.rlib"
    `rls` enabled features ["backtrace", "default", "example_generated"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\liberror_chain-1e94c8f434bd4c43.rlib"
  cargo_metadata 0.5.4 (registry+https://github.com/rust-lang/crates.io-index)
    `clippy-driver` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\libcargo_metadata-03e90dedac875643.rlib"
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\libcargo_metadata-39ee2e9489759a28.rlib"

Affected PRs

• rustbuild: Fail the build if we build Cargo twice #49053 (comment) First PR that experienced this, possibly this is where the bug was introduced. I think this started with Cargo update, cannot find relevant comment right now.
• Did you mean to block nightlies on clippy? #51122 CI log Succeeded after adding dependencies hack: rust-lang/rust-clippy@41972f8
• Update RLS and Rustfmt #51677 CI log
• Update the clippy submodule #52012 CI log Succeed after updating the hack

cc @alexcrichton (creator of the first PR with this issue)

[oli-obk]

fixed in #52012 (clippy PR: rust-lang/rust-clippy#2889)

We needed to manually ensure that clippy's and cargo's depedencies all have the same features enabled

[alexcrichton]

Is this a flaky error happening on CI? Or is this preventing PRs from landing?

If it's the latter that's how this is supposed to be wokring?

[oli-obk]

No, the issue is that it's not quite trivial to sync the features that are active for dependencies of clippy-driver, rls and cargo

for rls and clippy-driver it should become easy because rls soon depends on clippy, but clippy-driver and cargo might still decide to diverge when dependencies change their features. We'll see that on CI though and "just" need to change the list of features used in Cargo.toml, even if the features are unused by that crate

[alexcrichton]

Ah ok, thanks for clarifying! This definitely is a bummer of an issue but we don't have the CI budget to build these tools multiple times on all CI runs. The biggest "offender" so far has been winapi and we could tweak these crates to just enable all features of winapi or some larger subset to prevent this from coming up too much.

[mati865]

@alexcrichton it actually prevents tool related PRs from landing unless they add hack to make them use the same features as cargo which @oli-obk mentioned earlier.
The hack looks like that: https://github.com/rust-lang-nursery/rust-clippy/blob/28daee4c919dd88772847ec5240eb850bb0dcbf3/Cargo.toml#L48-L84
Similar "thing" has to be applied to rls so it can build (and unblock nightlies).

To solve it without hacks build system would have to collect features cargo, clippy-driver, rls and possibly miri are using for each dependency and merge them in one lib. One lib to rule them all and contain every required feature.

Actually this issue might be not related to Windows.
All failed jobs come form AppVeyor and I connected it to Windows toolchains but it seems AppVeyor is just faster than Travis. So in fact Travis had no opportunity to fail just yet.

[alexcrichton]

Indeed it is a hack! And indeed it is a nasty and pretty bad hack! Unfortunately though we don't have many options, we can't rebuild cargo twice on CI, we don't have budget for that. If there's other ways to solve this though that'd be great!

[oli-obk]

I'm all for deduplicating as much stuff as possible. I just thought that workspaces already did that and that the entire rustc repo is one big workspace. At least we removed the clippy workspace to be able to integrate into the rustc workspace.

[kennytm]

If we just whitelist winapi from the duplication check would it slow down the CI too much?

[mati865]

Not sure if I'm getting it right but wouldn't whitelisting winapi mean enabling other duplicates as well?

See how all libs mentioned here have different hashes:

Click there to show

  winapi 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features ["basetsd", "consoleapi", "dbghelp", "errhandlingapi", "fileapi", "handleapi", "ioapiset", "jobapi", "jobapi2", "lmcons", "memoryapi", "minschannel", "minwinbase", "minwindef", "namedpipeapi", "ntdef", "ntsecapi", "ntstatus", "processenv", "processthreadsapi", "profileapi", "psapi", "schannel", "securitybaseapi", "std", "synchapi", "sysinfoapi", "timezoneapi", "winbase", "wincon", "wincrypt", "winerror", "winnt", "winsock2", "ws2def", "ws2ipdef", "ws2tcpip"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libwinapi-a2af08b7465a463b.rlib"
    `cargo` enabled features ["consoleapi", "dbghelp", "errhandlingapi", "fileapi", "handleapi", "ioapiset", "jobapi", "jobapi2", "lmcons", "minschannel", "minwinbase", "minwindef", "namedpipeapi", "ntdef", "ntsecapi", "ntstatus", "processenv", "processthreadsapi", "profileapi", "psapi", "schannel", "securitybaseapi", "std", "synchapi", "sysinfoapi", "timezoneapi", "winbase", "wincon", "wincrypt", "winerror", "winnt", "winsock2", "ws2def", "ws2ipdef", "ws2tcpip"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libwinapi-75d56a453e3fc962.rlib"
  curl-sys 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libcurl_sys-07fab75d282edb3b.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libcurl_sys-8286f1c6e816accc.rlib"
  wincolor 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libwincolor-b9517277038668bb.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libwincolor-db0e3193d82f202c.rlib"
  atty 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libatty-82425baca7e56514.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libatty-6d7d880ad7a533e9.rlib"
  rand 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features ["default", "libc", "std"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\librand-ccae812451f64066.rlib"
    `cargo` enabled features ["default", "libc", "std"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\librand-45fd55b0079002aa.rlib"
  same-file 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libsame_file-4b3a0bd413c6f3bf.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libsame_file-32b416b8f642697b.rlib"
  backtrace 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features ["backtrace-sys", "coresymbolication", "dbghelp", "default", "dladdr", "libbacktrace", "libunwind", "winapi"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libbacktrace-ee3dd5721ab352f7.rlib"
    `cargo` enabled features ["backtrace-sys", "coresymbolication", "dbghelp", "default", "dladdr", "libbacktrace", "libunwind", "winapi"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libbacktrace-ff4c23d1fcd36bf6.rlib"
  schannel 0.1.12 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libschannel-222a411de60b04b9.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libschannel-1d0e20c28468f83f.rlib"
  remove_dir_all 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libremove_dir_all-4c00c52afc68f3cb.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libremove_dir_all-d03baecab7e09aef.rlib"
  socket2 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libsocket2-9f55854fec963a15.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libsocket2-a1d6162231386262.rlib"
  crypto-hash 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libcrypto_hash-829a5f4933ddc563.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libcrypto_hash-150f2c6059b16f36.rlib"
  fs2 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libfs2-b478f9abf7de39bf.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libfs2-ffa7d7365906dcd1.rlib"
  home 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libhome-7eba4286448ff1e2.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libhome-b9e04836ccb5f42d.rlib"
  libgit2-sys 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features ["curl", "curl-sys", "https", "libssh2-sys", "openssl-sys", "ssh"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\liblibgit2_sys-70788a004ed18436.rlib"
    `cargo` enabled features ["curl", "curl-sys", "https", "libssh2-sys", "openssl-sys", "ssh"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\liblibgit2_sys-bc2def446260374b.rlib"
  termcolor 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libtermcolor-4ec3b8f48a87abb3.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libtermcolor-82d7088db599d8d1.rlib"
  failure 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features ["backtrace", "default", "derive", "failure_derive", "std"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libfailure-1c8564f83dd1bbfe.rlib"
    `cargo` enabled features ["backtrace", "default", "derive", "failure_derive", "std"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libfailure-bb025f71ee213898.rlib"
  walkdir 2.1.4 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libwalkdir-e04d6a5820d4bde8.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libwalkdir-d55d8351cb831957.rlib"
  jobserver 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libjobserver-d60e21157ea1f605.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libjobserver-025dea58ee953278.rlib"
  tempfile 3.0.2 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libtempfile-f33d685736427096.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libtempfile-33eda6798c98f526.rlib"
  curl 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libcurl-18f57ad33f4b9aff.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libcurl-f20d13a58db15bb6.rlib"
  miow 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libmiow-0111e55453ecb56f.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libmiow-432140ba76e7118e.rlib"
  clap 2.31.2 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features ["ansi_term", "atty", "color", "default", "strsim", "suggestions", "vec_map"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libclap-b58e20434cab7771.rlib"
    `cargo` enabled features ["ansi_term", "atty", "color", "default", "strsim", "suggestions", "vec_map"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libclap-320d6dcfb6f95db4.rlib"
  env_logger 0.5.8 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features ["default", "regex"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libenv_logger-3b7e25bde6ed0163.rlib"
    `cargo` enabled features ["default", "regex"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libenv_logger-a46304677ab46d96.rlib"
  git2 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features ["curl", "default", "https", "libgit2-sys", "openssl-probe", "openssl-sys", "ssh"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libgit2-902e54faf353c414.rlib"
    `cargo` enabled features ["curl", "default", "https", "libgit2-sys", "openssl-probe", "openssl-sys", "ssh"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libgit2-681ae662433517ca.rlib"
  crates-io 0.18.0 (path+file:///C:/projects/rust/src/tools/cargo/src/crates-io)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libcrates_io-8c0a2c21d5890dd7.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libcrates_io-0797dbfc3c5ef15d.rlib"
  ignore 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libignore-7e9781e66a26cbad.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libignore-3566e186859cf882.rlib"
  git2-curl 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)
    `rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libgit2_curl-f82d39cd31fa8234.rlib"
    `cargo` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2-tools\\x86_64-pc-windows-msvc\\release\\deps\\libgit2_curl-68b10254541374c7.rlib"

[oli-obk]

I'd say we keep what we have and fix the issues as they come. I'll try to cook up better diagnostics and investigate why workspaces don't already fix all this for us.

[alexcrichton]

@oli-obk workspaces deduplicate versions of dependencies but doesn't currently deduplicate and/or unify the activated features for dependencies (this is arguably a misfeature of Cargo).

@kennytm unfortunately winapi is the source of the problems here and we can't cover over it. If winapi is recompiled then everything that depends on winapi is also recompiled, which typically includes most of the crate graph.

Right now the best solution is a pretty bad solution (unfortunately) which is to just manually make sure that all our relevant projects depend on all the same features. That's what I've done in Cargo historically