Description
The Level Zero adapter implementation of urKernelSuggestMaxCooperativeGroupCount ignores the dynamicSharedMemorySize argument. In the Level Zero v2 adapter:
|
ur_result_t urKernelSuggestMaxCooperativeGroupCount( |
|
::ur_kernel_handle_t hKernelOpque, ::ur_device_handle_t hDeviceOpque, |
|
uint32_t workDim, const size_t *pLocalWorkSize, |
|
size_t dynamicSharedMemorySize, uint32_t *pGroupCountRet) { |
|
auto hKernel = v2_cast(hKernelOpque); |
|
auto hDevice = common_cast(hDeviceOpque); |
|
(void)dynamicSharedMemorySize; |
|
|
|
uint32_t wg[3]; |
|
wg[0] = ur_cast<uint32_t>(pLocalWorkSize[0]); |
|
wg[1] = workDim >= 2 ? ur_cast<uint32_t>(pLocalWorkSize[1]) : 1; |
|
wg[2] = workDim == 3 ? ur_cast<uint32_t>(pLocalWorkSize[2]) : 1; |
|
ZE2UR_CALL(zeKernelSetGroupSize, |
|
(hKernel->getZeHandle(hDevice), wg[0], wg[1], wg[2])); |
|
|
|
uint32_t totalGroupCount = 0; |
|
ZE2UR_CALL(zeKernelSuggestMaxCooperativeGroupCount, |
|
(hKernel->getZeHandle(hDevice), &totalGroupCount)); |
|
*pGroupCountRet = totalGroupCount; |
|
return UR_RESULT_SUCCESS; |
|
} |
the argument is explicitly discarded:
(void)dynamicSharedMemorySize;
The implementation then calls:
zeKernelSetGroupSize(...)
zeKernelSuggestMaxCooperativeGroupCount(...)
without configuring the dynamic local-memory kernel argument. The Level Zero v1 adapter appears to have the same behavior.
Changing only dynamicSharedMemorySize does not affect the returned maximum cooperative group count. On an Intel Arc Pro B60 with a local work-group size of 256, I observed:
| dynamicSharedMemorySize |
Returned max group count |
| 0 |
160 |
| 16 KiB |
160 |
| 32 KiB |
160 |
| 64 KiB |
160 |
| 128 KiB |
160 |
However, when the corresponding Level Zero local-memory kernel argument is explicitly configured before calling zeKernelSuggestMaxCooperativeGroupCount, the results are:
| Configured local memory |
Returned max group count |
| 16 KiB |
160 |
| 32 KiB |
80 |
| 64 KiB |
40 |
| 128 KiB |
20 |
This indicates that the Level Zero driver accounts for dynamic local memory when it is represented in the native kernel state. The UR Level Zero adapter currently does not propagate the dynamicSharedMemorySize parameter into that state.
Expected behavior
dynamicSharedMemorySize should participate in the cooperative group residency calculation.
For kernels using dynamic local/shared memory, increasing this value may reduce the number of work-groups that can be resident concurrently. Therefore, pGroupCountRet should reflect the requested dynamic shared memory size. This is important for cooperative kernels containing a grid-wide barrier: launching more work-groups than can be resident concurrently may cause the kernel to deadlock.
Environment
- Device: Intel Arc Pro B60
- Compute Runtime package: 26.22.38646.6-1
26.04ppa1
- Level Zero driver: 1.15.38646+6
- zeDriverGetProperties: 1.3.38646
- Level Zero loader: 1.28.6
- oneAPI: 2026.1
- UR adapter: libur_adapter_level_zero_v2.so.0.12.0
- Linux kernel: 7.0.0-28-generic
Description
The Level Zero adapter implementation of
urKernelSuggestMaxCooperativeGroupCountignores thedynamicSharedMemorySizeargument. In the Level Zero v2 adapter:llvm/unified-runtime/source/adapters/level_zero/v2/kernel.cpp
Lines 663 to 683 in 8ec986f
the argument is explicitly discarded:
(void)dynamicSharedMemorySize;The implementation then calls:
without configuring the dynamic local-memory kernel argument. The Level Zero v1 adapter appears to have the same behavior.
Changing only
dynamicSharedMemorySizedoes not affect the returnedmaximum cooperative group count. On an Intel Arc Pro B60 with a local work-group size of 256, I observed:However, when the corresponding Level Zero local-memory kernel argument is explicitly configured before calling
zeKernelSuggestMaxCooperativeGroupCount, the results are:This indicates that the Level Zero driver accounts for dynamic local memory when it is represented in the native kernel state. The UR Level Zero adapter currently does not propagate the
dynamicSharedMemorySizeparameter into that state.Expected behavior
dynamicSharedMemorySizeshould participate in the cooperative group residency calculation.For kernels using dynamic local/shared memory, increasing this value may reduce the number of work-groups that can be resident concurrently. Therefore,
pGroupCountRetshould reflect the requested dynamic shared memory size. This is important for cooperative kernels containing a grid-wide barrier: launching more work-groups than can be resident concurrently may cause the kernel to deadlock.Environment
26.04ppa1