Compute shader workgroups execution and size [closed]

Question

I want to clarify how are workgroups executed on various GPUs, thus I have several questions. I know that different GPU architectures work differently, but I expect an answer YES if there is single (popular) GPU for which some statement is true. I really would like to get simple Yes/No answers in addition to complicated discussion about specific architectures.

1. Is Work Group a synonym to wavefront and warp? So it means a bundle of threads?
2. A Compute Unit, typically, has 4 SIMD units inside. Each SIMD executes 1 instruction per cycle for (typically?) 16 threads. Question: if WG size is 64 (equal to total width of all SIMDs): can one CU execute one WG per SIMD, so 4 SIMDs execute 1/4 of each 4 WG in one cycle? Or/and can it execute one whole WG on different SIMDs, so 4 SIMDs finish together 1 WG in one cycle?
3. If WG size is bigger than SIMD size (or CU size), it may be executed on different SIMDs (or even different CU units) in parallel (right?). But if WG size is less than SIMD size: can one SIMD execute threads of different Work Groups (of course such WGs need to execute same shader code) in the same cycle?
4. Is it ok to define Work Group size as (1, 1, 1)? I saw an example of such compute shader in D3D documentation. This relates to previous Question 3: it seems to be ok if one SIMD can execute 16 threads each belonging to a different WG of size 1. But if it can't, it is a waste of SIMD power.
5. Can execution/memory barriers be omitted, if Work Group size is equal or less than SIMD size? Relates to Question 4. They can't be omiited if threads of one WG share some data but are executed on different SIMDs or CUs.

Answer 1

Since you mentioned CUs and SIMDs, I wrote this mainly from the perspective of AMD's GCN architecture, but broadly speaking the answers apply to other GPU architectures as well (except for #2 which is really a detail specific to GCN). Other GPUs have parts that are at least somewhat analogous to CUs and SIMDs, though the details vary a lot.

1. Not exactly. They are both bundles of threads, but a workgroup is a software concept, while wavefronts/warps (generically referred to as "waves") are a hardware concept. The size of workgroup is defined by your code when you write the compute shader, but the size of a wave is defined by the hardware. A workgroup can be anywhere from 1 to 1024 threads, but a wave on NVIDIA (a warp) is always 32 threads, a wave on AMD (a wavefront) is 64 threads—or, on their newer RDNA architecture, can be set to either 32 or 64 by the driver (but is always one or the other for any given shader).

   As far as I know, a workgroup may be made up of multiple waves but not the other way around. That is, multiple workgroups will not be coalesced together into a single wave.

2. The way AMD's GCN architecture works is that a SIMD is physically 16-wide, but it is "quad-pumped". This means that each instruction is issued 4 times in a row, operating on 1/4th of the wave each time. So, a 64-wide wave will be executed on a single SIMD, with the 16 physical lanes cycling through the 64 threads of the wave. A wave will not be split across multiple SIMDs; rather, other SIMDs are used to run other waves in the same dispatch.

3. If the workgroup size is bigger than a wave, it will be split up into waves, and all those waves will execute on the various SIMDs inside a single CU (there is no splitting of a workgroup across different CUs, because of shared memory—all threads in the workgroup need to be able to access the shared memory, and it's part of the CU).

   As mentioned earlier, as far as I am aware, multiple workgroups do not ever get packed into the same wave, so they would not execute together on a single SIMD.

4. It is not advisable to define a workgroup of size (1, 1, 1) unless you truly need just one single thread in the whole dispatch. This is uncommon, but it comes up sometimes when some small calculation needs to be done on the GPU, such as setting up indirect draw parameters based on output of an earlier pass.

   Generally, you should use workgroups that are a multiple of 64 threads when you actually want to do a lot of calculations in parallel. Otherwise, as you said, a bunch of the lanes in the SIMDs are left unoccupied, which is a waste.

5. Maybe, sometimes. If you know the exact hardware and can be 100% assured that the workgroup is within a single wave, then it may be possible to omit some barriers.

   However, barriers serve other purposes besides inter-wave synchronization: they also inform the compiler not to re-order memory operations across the barrier when it does optimization, and they may also be needed at runtime to tell the SIMD to wait for memory writes before proceeding, even if it's only a single wave, due to pipelined execution (without the barrier, later instructions might start before the earlier writes had completed). Both of these could be required for the correctness of the algorithm you're implementing.

   Generally, I would not advise removing barriers unless you're a stone-cold expert and working with a specific hardware and driver stack that you understand well, as opposed to a broad spectrum of GPUs.

   Also, you might be aware that there are "wave operations" exposed by most APIs now, that allow some safe intra-wave communication without barriers. These are often useful for reducing the need for barriers in optimized versions of algorithms.