compile-time evaluation: detect writes through immutable pointers by RalfJung · Pull Request #118324 · rust-lang/rust

RalfJung · 2023-11-26T16:09:53Z

This has two motivations:

it unblocks const interning: decide about mutability purely based on the kind of interning, not the types we see #116745 (and therefore takes a big step towards const_mut_refs stabilization), because we can now detect if the memory that we find in const can be interned as "immutable"
it would detect the UB that was uncovered in revert stabilization of const_intrinsic_copy #117905, which was caused by accidental stabilization of copy functions in const that can only be called with UB

When UB is detected, we emit a future-compat warn-by-default lint. This is not a breaking change, so completely in line with the const-UB RFC, meaning we don't need t-lang FCP here. I made the lint immediately show up for dependencies since it is nearly impossible to even trigger this lint without const_mut_refs -- the accidentally stabilized copy functions are the only way this can happen, so the crates that popped up in #117905 are the only causes of such UB (in the code that crater covers), and the three cases of UB that we know about have all been fixed in their respective crates already.

The way this is implemented is by making use of the fact that our interpreter is already generic over the notion of provenance. For CTFE we now use the new CtfeProvenance type which is conceptually an AllocId plus a boolean immutable flag (but packed for a more efficient representation). This means we can mark a pointer as immutable when it is created as a shared reference. The flag will be propagated to all pointers derived from this one. We can then check the immutable flag on each write to reject writes through immutable pointers.

I just hope perf works out.

rustbot · 2023-11-26T16:10:02Z

r? @cjgillot

(rustbot has picked a reviewer for you, use r? to override)

rustbot · 2023-11-26T16:10:04Z

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

Some changes occurred to the CTFE / Miri engine

cc @rust-lang/miri

Some changes occurred in compiler/rustc_codegen_cranelift

cc @bjorn3

The Miri subtree was changed

cc @rust-lang/miri

Some changes occurred to the CTFE / Miri engine

cc @rust-lang/miri

Some changes occurred in compiler/rustc_codegen_gcc

cc @antoyo

RalfJung · 2023-11-26T16:11:34Z

@bors try @rust-timer queue

bors · 2023-11-26T16:12:47Z

⌛ Trying commit f25c190 with merge b5c17f8...

…<try> compile-time evaluation: detect writes through immutable pointers This has two motivations: - it unblocks rust-lang#116745 (and therefore takes a big step towards `const_mut_refs` stabilization), because we can now detect if the memory that we find in `const` can be interned as "immutable" - it would detect the UB that was uncovered in rust-lang#117905, which was caused by accidental stabilization of `copy` functions in `const` that can only be called with UB When UB is detected, we emit a future-compat warn-by-default lint. This is not a breaking change, so completely in line with [the const-UB RFC](https://rust-lang.github.io/rfcs/3016-const-ub.html), meaning we don't need t-lang FCP here. I made the lint immediately show up for dependencies since it is nearly impossible to even trigger this lint without `const_mut_refs` -- the accidentally stabilized `copy` functions are the only way this can happen, so the crates that popped up in rust-lang#117905 are the only causes of such UB (in the code that crater covers), and the three cases of UB that uncovered have all been fixed in their respective crates already. I just hope perf works out.

RalfJung · 2023-11-26T16:34:51Z

+    /// Returns the `AllocId` of this provenance.
+    #[inline(always)]
+    pub fn alloc_id(self) -> AllocId {
+        AllocId(NonZeroU64::new(self.0.get() & !IMMUTABLE_MASK).unwrap())


We could use new_unchecked here, we have an invariant that the AllocId part of the provenance is always non-null. But let's first see if perf is fine without unsafe code.

bors · 2023-11-26T17:40:09Z

☀️ Try build successful - checks-actions
Build commit: b5c17f8 (b5c17f82cf090a61e41e7d3c33a8975322d8d5c3)

RalfJung · 2023-11-26T20:39:26Z

 use rustc_span::edition::Edition;
 use rustc_span::symbol::sym;

+declare_lint_pass! {


I moved this to the top, it was in a random spot in the middle of the file which surely doesn't make a ton of sense.

rust-timer · 2023-11-27T06:45:49Z

Finished benchmarking commit (b5c17f8): comparison URL.

Overall result: ❌ regressions - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.2%	[0.3%, 5.8%]	8
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.3%	[-2.7%, -2.0%]	2
Improvements ✅ (secondary)	-3.3%	[-4.4%, -2.2%]	3
All ❌✅ (primary)	-2.3%	[-2.7%, -2.0%]	2

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	5.2%	[4.8%, 5.8%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.0%	[-2.0%, -2.0%]	2
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 674.277s -> 673.76s (-0.08%)
Artifact size: 313.37 MiB -> 313.35 MiB (-0.00%)

ghost · 2023-11-27T07:45:23Z

Right, so there is a regression in ctfe-stress. I was worried this might happen. I guess now comes the usual phase of doing random changes and seeing how they affect perf...

ghost · 2023-11-27T08:12:31Z

@bors try @rust-timer queue

ghost · 2023-12-07T23:01:55Z

Finished benchmarking commit (0e7f91b): comparison URL.

Overall result: ❌ regressions - ACTION NEEDED

Next Steps: If you can justify the regressions found in this perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please open an issue or create a new PR that fixes the regressions, add a comment linking to the newly created issue or PR, and then add the perf-regression-triaged label to this PR.

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.9%	[0.3%, 1.4%]	12
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	3.4%	[0.4%, 6.4%]	2
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.0%	[-2.2%, -0.4%]	3
Improvements ✅ (secondary)	-2.1%	[-2.1%, -2.1%]	1
All ❌✅ (primary)	0.8%	[-2.2%, 6.4%]	5

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	5.4%	[5.3%, 5.5%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 674.821s -> 675.63s (0.12%)
Artifact size: 314.17 MiB -> 314.19 MiB (0.00%)

ghost · 2023-12-12T00:17:54Z

🤷 I feel like the remaining regression is small enough to be acceptable. We're doing more checks in const-eval, and there are no regressions in the builds of real crates.

(#118324 (comment))

Agreed.

rustbot assigned cjgillot Nov 26, 2023

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Nov 26, 2023

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Nov 26, 2023

RalfJung commented Nov 26, 2023

View reviewed changes

This comment has been minimized.

Sign in to view

RalfJung force-pushed the ctfe-read-only-pointers branch from f25c190 to 7a4d651 Compare November 26, 2023 17:54

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Nov 26, 2023

RalfJung force-pushed the ctfe-read-only-pointers branch from 7a4d651 to 37a50ac Compare November 26, 2023 18:10

RalfJung mentioned this pull request Nov 26, 2023

const interning: decide about mutability purely based on the kind of interning, not the types we see #116745

Closed

ghost reviewed Nov 26, 2023

View reviewed changes

Comment thread compiler/rustc_middle/src/mir/interpret/allocation.rs Outdated

ghost reviewed Nov 26, 2023

View reviewed changes

Comment thread compiler/rustc_middle/src/mir/interpret/allocation/provenance_map.rs Outdated

RalfJung commented Nov 26, 2023

View reviewed changes

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Nov 27, 2023

RalfJung force-pushed the ctfe-read-only-pointers branch from 7197e21 to be1e481 Compare November 27, 2023 07:33

RalfJung force-pushed the ctfe-read-only-pointers branch from be1e481 to 303872f Compare November 27, 2023 07:55

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Nov 27, 2023

adpaco mentioned this pull request Dec 8, 2023

Update Rust toolchain to nightly-2023-12-08 model-checking/kani#2926

Merged

RalfJung deleted the ctfe-read-only-pointers branch December 9, 2023 13:32

Mark-Simulacrum added the perf-regression-triaged The performance regression has been triaged. label Dec 12, 2023

RalfJung mentioned this pull request Dec 17, 2023

const-eval interning: get rid of type-driven traversal #119044

Merged

ojeda mentioned this pull request Feb 17, 2024

Rust unstable features needed for the kernel Rust-for-Linux/linux#2

Open

55 tasks

RalfJung mentioned this pull request Aug 17, 2024

make writes_through_immutable_pointer a hard error #129199

Merged

RalfJung mentioned this pull request Aug 18, 2024

const-eval interning: accept interior mutable pointers in final value #128543

Merged

ojeda mentioned this pull request Feb 8, 2026

Rust unstable features needed for the kernel — Done Rust-for-Linux/linux#1223

Open

66 tasks

Uh oh!

Uh oh!

Conversation

RalfJung commented Nov 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Nov 26, 2023

Uh oh!

rustbot commented Nov 26, 2023

Uh oh!

RalfJung commented Nov 26, 2023

Uh oh!

This comment has been minimized.

bors commented Nov 26, 2023

Uh oh!

RalfJung Nov 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bors commented Nov 26, 2023

Uh oh!

This comment has been minimized.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rust-timer commented Nov 27, 2023

Overall result: ❌ regressions - ACTION NEEDED

Uh oh!

ghost commented Nov 27, 2023

Uh oh!

ghost commented Nov 27, 2023

Uh oh!

This comment has been minimized.

ghost commented Dec 7, 2023

Overall result: ❌ regressions - ACTION NEEDED

Uh oh!

ghost commented Dec 12, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

RalfJung commented Nov 26, 2023 •

edited

Loading

RalfJung Nov 26, 2023 •

edited

Loading