Workgroup size performance change?

Question

I know, that when using compute shader, the invocations should to be grouped with respect to a multiple of the hardware minimum workgroup size (Nvidia 32, AMD 64). Otherwise cores are blocked. In case of no communication between invocations is required, does it makes sense to use a bigger workgroup size than the minimum workgroup size of 32 or 64?

So let's say we have 320 particles which need to be calculated. No collision detection, no communication between them... and we are working on a Nvidia machine. Is there a performance boost to create one workgroup with 320 invocations instead of using 10 times one workgroup with 32 invocations?

Or to put it another way, does the size of the workgroup make a difference, aside from the ability to use shared memory?

Are there benefits to use smaller workgroup sizes?

Answer 1

In your specific example, where there are no race conditions, as long as the workgroup size is a multiple of what the architecture requires (wavefront/warp) there would be no difference as the GPU/driver will schedule as it see's fit and in all likelyhood, the GPU would be fully saturated in both cases.

However, if a specific shaders computation is quite heavy (each shader doing a lot), or they are unbalanced wrt to workload (one invocation in a group takes a lot longer than the rest of the group/other groups, 'delaying' the completion of that group), this may trigger a TDR (Timeout, Delay and Recovery) when invoking a lot of workgroups in a single dispatch.

In these scenarios it is better to lighten the work each shader does (split algorithm into more digestable stages so that the workload is more balanced across the workgroup, and ideally across all workgroups) and/or dispatch batches, flushing/finishing inbetween batches so to keep the OS happy. Balancing the workgroup count vs number of dispatches can be quite difficult when targeting a wide gamut of hardware to achieve optimal performance and may require a 'calibration'/dry-run at runtime to deduce a safe batch size/workgroup count.