Skip to content

[UR][Level Zero] urKernelSuggestMaxCooperativeGroupCount ignores dynamicSharedMemorySize #22936

Description

@lchen2331

Description

The Level Zero adapter implementation of urKernelSuggestMaxCooperativeGroupCount ignores the dynamicSharedMemorySize argument. In the Level Zero v2 adapter:

ur_result_t urKernelSuggestMaxCooperativeGroupCount(
::ur_kernel_handle_t hKernelOpque, ::ur_device_handle_t hDeviceOpque,
uint32_t workDim, const size_t *pLocalWorkSize,
size_t dynamicSharedMemorySize, uint32_t *pGroupCountRet) {
auto hKernel = v2_cast(hKernelOpque);
auto hDevice = common_cast(hDeviceOpque);
(void)dynamicSharedMemorySize;
uint32_t wg[3];
wg[0] = ur_cast<uint32_t>(pLocalWorkSize[0]);
wg[1] = workDim >= 2 ? ur_cast<uint32_t>(pLocalWorkSize[1]) : 1;
wg[2] = workDim == 3 ? ur_cast<uint32_t>(pLocalWorkSize[2]) : 1;
ZE2UR_CALL(zeKernelSetGroupSize,
(hKernel->getZeHandle(hDevice), wg[0], wg[1], wg[2]));
uint32_t totalGroupCount = 0;
ZE2UR_CALL(zeKernelSuggestMaxCooperativeGroupCount,
(hKernel->getZeHandle(hDevice), &totalGroupCount));
*pGroupCountRet = totalGroupCount;
return UR_RESULT_SUCCESS;
}

the argument is explicitly discarded:

(void)dynamicSharedMemorySize;

The implementation then calls:

zeKernelSetGroupSize(...)
zeKernelSuggestMaxCooperativeGroupCount(...)

without configuring the dynamic local-memory kernel argument. The Level Zero v1 adapter appears to have the same behavior.

Changing only dynamicSharedMemorySize does not affect the returned maximum cooperative group count. On an Intel Arc Pro B60 with a local work-group size of 256, I observed:

dynamicSharedMemorySize Returned max group count
0 160
16 KiB 160
32 KiB 160
64 KiB 160
128 KiB 160

However, when the corresponding Level Zero local-memory kernel argument is explicitly configured before calling zeKernelSuggestMaxCooperativeGroupCount, the results are:

Configured local memory Returned max group count
16 KiB 160
32 KiB 80
64 KiB 40
128 KiB 20

This indicates that the Level Zero driver accounts for dynamic local memory when it is represented in the native kernel state. The UR Level Zero adapter currently does not propagate the dynamicSharedMemorySize parameter into that state.

Expected behavior

dynamicSharedMemorySize should participate in the cooperative group residency calculation.

For kernels using dynamic local/shared memory, increasing this value may reduce the number of work-groups that can be resident concurrently. Therefore, pGroupCountRet should reflect the requested dynamic shared memory size. This is important for cooperative kernels containing a grid-wide barrier: launching more work-groups than can be resident concurrently may cause the kernel to deadlock.

Environment

  • Device: Intel Arc Pro B60
  • Compute Runtime package: 26.22.38646.6-126.04ppa1
  • Level Zero driver: 1.15.38646+6
  • zeDriverGetProperties: 1.3.38646
  • Level Zero loader: 1.28.6
  • oneAPI: 2026.1
  • UR adapter: libur_adapter_level_zero_v2.so.0.12.0
  • Linux kernel: 7.0.0-28-generic

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinglevel-zeroIssues related to the Level Zero backendunified-runtime

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions