Make the CUDA stream pool per-thread - #23672

Open

vuule wants to merge 38 commits into

NVIDIA:mainfrom

vuule:per-thread-stream-pool

Commits on Aug 13, 2026

Make the CUDA stream pool per-thread and per-device
vuule
committed
9734058

Commits on Aug 14, 2026

Merge remote-tracking branch 'origin/main' into per-thread-stream-pool
vuule
committed
abf3902

Commits on Aug 17, 2026

Address review: correct the pool thread-safety contract and harden tests
vuule
committed
71dd8f9

Commits on Aug 18, 2026

Use cuda::stream_ref and cuda::stream in the stream pool
vuule
committed
73ce477
Keep only the stream pool tests that catch silent regressions
vuule
committed
b72be07
Merge branch 'main' into per-thread-stream-pool
vuule
authored
f09cc08
Rename global_cuda_stream_pool to thread_cuda_stream_pool
vuule
committed
b3eabc8
Describe what pool streams actually guarantee
vuule
committed
3b0ac5d
Drop the stream pool size accessor
vuule
committed
7a08f9c
Drop the stream pool id-based accessor
vuule
committed
a3400fa
Point callers that need several streams at get_streams
vuule
committed
445f39a
docs
vuule
committed
3726678
Rename create_global_cuda_stream_pool to create_cuda_stream_pool
vuule
committed
1a2fb3e
Test the stream pool directly instead of through fork_streams
vuule
committed
f1c16c5
Name the test helper after what it returns
vuule
committed
3ba29c0
Correct why the concurrency test needs the latch
vuule
committed
f90ba7d
Name the accessor for the caller, not the pool's scope
vuule
committed
2b461f5
Store the pool cap in the pool
vuule
committed
f28f9c2
Say that the large request grows the pool
vuule
committed
de0f6e6
Compare against the whole pool the first thread leaves behind
vuule
committed
c3fcc3e
Hold the second thread's streams in a vector
vuule
committed
226f3eb
Share one oversized request size between the pool tests
vuule
committed
8f7933d
Rename the test helper and request past a 64-stream pool
vuule
committed
35f71b8

Commits on Aug 19, 2026

last of test clean up
vuule
committed
9d8d0d2
shorten comment
vuule
committed
cb741df
impl clean up
vuule
committed
8963469
Merge branch 'main' into per-thread-stream-pool
vuule
authored
b9fdd0f
Build the join_streams wrapper from cuda::stream_ref
vuule
committed
b5d2c4a
Update copyright header on the stream pool bindings
vuule
committed
5c2eca5
Merge branch 'main' into per-thread-stream-pool
vuule
authored
a8d462c
Refresh stream pool docs for cuda::stream_ref and include <cuda/devices>
vuule
committed
1f55ffa

Commits on Aug 20, 2026

Merge branch 'main' into per-thread-stream-pool
vuule
authored
e6482d2
Drop the unused stream from the pinned memory resource
vuule
committed
c7565b0
Check the batched memcpy stream argument on CUDA 12.8 and newer
vuule
committed
c93d3d1
Take the streams to join as a std::span
vuule
committed
9cb1d52
Say which requests the pool maximum is sized for
vuule
committed
da76fd4
recycle events as well
vuule
committed
eb7b11c
style: clang-format
vuule
committed
9226246