Add 64-bit integer vectors and operations on them by Shnatsel · Pull Request #253 · linebender/fearless_simd

Shnatsel · 2026-06-23T13:03:12Z

There are two unrelated changes:

The documentation on load/store_interleaved_128 is fixed to match the implementation, which in turn mirrors the NEON intrinsics
Some generic fallback methods are refactored to match the fallback's fully unrolled style as opposed to array::from_fn() calls

Supersedes #97

Shnatsel · 2026-06-23T14:12:09Z

The documentation for load/store_interleaved_128 was misleading. Both formulations are valid for 32-bit elements but the 8- and 16-bit elements already behaved differently, following the NEON vld4/vst4 semantics rather than our documented semantics. This misled me into generalizing the op to 64-bit numbers incorrectly.

I've changed the implementation back to vld4/vst4 semantics in subsequent commits and updated documentation.

Add i64/u64 vector types and operations across the generated SIMD backends, with focused int64 coverage and optimized interleaved load/store paths where available.

LaurenzV · 2026-06-30T17:45:41Z

I'm curious, do you think this will overall impact the compile time for the crate a lot, even if none of the 64-bit stuff is used? Have you done any measurements?

Shnatsel · 2026-06-30T18:00:00Z

It really shouldn't. This is all generic code, so it is not actually instantiated and doesn't turn into MIR or LLVM IR until something actually calls it.

The downside of generics is that if we call the same function 5 times you get 5 different instantiations of it so 5x the IR for LLVM to chew through, but in our case we want all the intrinsics inlined anyway so this is unavoidable, generics or not.

Shnatsel · 2026-07-18T14:35:52Z

Marking this as draft until it's updated to latest main. Some refactoring should also be possible now that the SSE2 PR has laid the groundwork for nicer scalar fallback.

…64-bit-ints

…r SIMD levels

…Lake is available

…tures to cfgs

…ll scalar fallbacks use it, remove the code that duplicates it that originated in the i64 branch

Shnatsel · 2026-07-18T15:24:00Z

+        if vec_ty.scalar_bits == 64
+            && matches!(vec_ty.scalar, ScalarType::Int | ScalarType::Unsigned)
+            && method != "simd_eq"
+        {
+            return fallback_method(op, vec_ty);
+        }
+


I don't like that this fires only when it's not AVX-512 but this is not explicitly under an else or a non-AVX-512 branch. I want to refactor all the early returns out of the generator, but that's best left to a follow-up PR, I don't want to cram even more unrelated changes into this one.

Shnatsel · 2026-07-18T17:38:02Z

This should now be ready for review.

LaurenzV

Once again mostly relying on tests here, haven't tried to verify the actual logic of each operation. If it's possible, I do think it would be nice to split out the from_fn change, just to keep the diff clean (not like anyone else will ever read it, but still 🙂).

LaurenzV · 2026-07-22T17:01:03Z

What about rotate/shift_elements_left/right and swizzle?

that's covered in https://github.com/linebender/fearless_simd/pull/275/changes#diff-f8c0fbfc8f4c516a0abcc54128c32013ac7838e28285085fef7800fa7cce64c3

so not this PR but the follow-up that expands test coverage, stacked on top of this one

LaurenzV · 2026-07-22T17:11:48Z

+pub(crate) fn unrolled_array(len: usize, item: impl FnMut(usize) -> TokenStream) -> TokenStream {
+    let items = (0..len).map(item).collect::<Vec<_>>();
+    quote! { [#(#items),*] }


What's the motivation for this change? Did you verify whether this actually gives better assembly? Is that something that could be reverted for now to make the diff easier to read? Up to you.

I noticed that these fallbacks are not the same style as the actual Fallback implementation and went ahead and changed it, but I didn't verify that the fully unrolled implementations produce better assembly, I just assumed that the fallbacks are that way for a reason.

I've checked it for shl op and it seems LLVM unrolls those loops anyway, so it thinks that's profitable but I don't want to bet on LLVM preserving this optimization in the future so I'll keep the manual unrolling.

As far as I can tell these two formulations emit literally identical assembly, at least for shl as the op. I was surprised and suspected an error in my setup, but I double-checked and the results appear to hold.

I've inspected the optimized LLVM IR before lowering into assembly and it seems it gets recognized as a vector operation in both cases.

%values = bitcast <2 x i64> %a to <16 x i8> %shifts = bitcast <2 x i64> %b to <16 x i8> %masked = and <16 x i8> %shifts, splat (i8 7) %result = shl <16 x i8> %values, %masked

LaurenzV · 2026-07-22T17:18:46Z

@@ -392,8 +392,25 @@ impl<S: Simd> SimdBase<S> for i8x16<S> {
        block
    }
    #[inline(always)]
-    fn from_fn(simd: S, f: impl FnMut(usize) -> i8) -> Self {
-        simd.load_array_i8x16(core::array::from_fn(f))
+    fn from_fn(simd: S, mut f: impl FnMut(usize) -> i8) -> Self {


Piggybacking on my other comment, does this really yield better results than before? Seems a bit suspicious. 😄

There were some issues with it historically: rust-lang/rust#108765

But I guess it doesn't matter as much today. There's all sorts of considerations like panic safety and such, but measuring this rigorously is quite difficult, since you can put arbitrary functions in there and the properties probably vary depending on what you put there.

LaurenzV · 2026-07-22T17:23:42Z

I think we need to use wrapping_neg for negation, no? Also maybe add a test with i64::MIN to ensure it works.

Yes, it should. That's an issue with the current code, not restricted to 64-bit ints.

I've changed the scalar fallback behavior in this PR, and updated the tests in #275

… SIMD behavior more precisely in the ::MIN case

~~Stacked on top of linebender#253 which adds tests for 64-bit integer types~~ Rebased onto main after that was merged The first commit moves existing tests around. Tests that were haphazardly strewn about multiple files are now organized by operation, one file = one operation. This makes the tests much easier to navigate and makes the missing test coverage readily apparent. This also reduces the from-scratch build time with `cargo build --tests` from 19s to 14s on my machine. The two subsequent commits add 7k lines of tests to fill in coverage gaps. This was done with Python scripts and a bit of fixup. The generated tests are slightly different from the existing patterns in that they use `array::from_fn` as opposed to handwritten inputs and outputs. If that's a problem, one could probably put up an LLM to bringing them into the same style as the pre-existing tests, if desired.

Shnatsel commented Jun 28, 2026

View reviewed changes

Comment thread fearless_simd_gen/src/generic.rs Outdated

Add 64-bit integer vector support

34c0f3c

Add i64/u64 vector types and operations across the generated SIMD backends, with focused int64 coverage and optimized interleaved load/store paths where available.

Shnatsel force-pushed the 64-bit-ints branch from d5fae13 to 34c0f3c Compare June 28, 2026 21:25

Shnatsel marked this pull request as draft July 18, 2026 14:36

Shnatsel added 14 commits July 18, 2026 15:42

Merge main PR linebender#257: Merge LLM-generated 512-bit tests into …

b9ff3aa

…64-bit-ints

Merge main PR linebender#258: Multiversioning behind feature flags fo…

81c5a68

…r SIMD levels

Merge main PR linebender#260: Add comment in a macro

73c9072

Merge main PR linebender#261: Widen inputs in standalone examples

6767aff

Merge main PR linebender#263: Correctly omit AVX2 codepaths when Ice …

d6a030c

…Lake is available

Merge main PR linebender#266: Implement swizzle_dyn_within_blocks

6283d5a

Regenerate swizzle_dyn_within_blocks for 64-bit vectors

569298c

Merge main PR linebender#264: Change multiversioning control from fea…

569f91d

…tures to cfgs

Merge main PR linebender#262: Release 0.6.0

c53f32e

Record conflicts merging main PR linebender#274

e0fd448

Resolve PR linebender#274 for 64-bit vectors

6269b9d

Record conflicts merging main PR linebender#270

5e347dd

Resolve PR linebender#270 and refactor fallback routing

2ad9829

Move the SSE2 routing-to-fallback approach to a shared helper, make a…

97f5b59

…ll scalar fallbacks use it, remove the code that duplicates it that originated in the i64 branch

Shnatsel commented Jul 18, 2026

View reviewed changes

Shnatsel added 3 commits July 18, 2026 16:36

Add a comment

b345d0d

Fix comments showing incorrect layout

3a5c0bf

Add a comment explaining the f32 special case

f8ead1e

Shnatsel marked this pull request as ready for review July 18, 2026 17:38

Shnatsel mentioned this pull request Jul 18, 2026

Reorganize tests, expand test coverage #275

Merged

LaurenzV self-requested a review July 22, 2026 16:48

LaurenzV approved these changes Jul 22, 2026

View reviewed changes

Use wrapping_neg in fallback instead of neg for integers; matches the…

23d3c4f

… SIMD behavior more precisely in the ::MIN case

Shnatsel added this pull request to the merge queue Jul 22, 2026

Merged via the queue into linebender:main with commit ed5b353 Jul 22, 2026
22 checks passed

Shnatsel deleted the 64-bit-ints branch July 22, 2026 21:44

Uh oh!

Conversation

Shnatsel commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Shnatsel commented Jun 23, 2026

Uh oh!

Uh oh!

LaurenzV commented Jun 30, 2026

Uh oh!

Shnatsel commented Jun 30, 2026

Uh oh!

Shnatsel commented Jul 18, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Shnatsel commented Jul 18, 2026

Uh oh!

LaurenzV left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Shnatsel commented Jun 23, 2026 •

edited

Loading