Threads per compute shader group

Question

The maximum allowed number of threads per compute shader group is 1024 for Shader Model 5.0. Is it advisable with regard to performance to stay close to this maximum number?

In order to resolve SSAA and MSAA (down-scaling with appropriate tone mapping), I wrote some compute shaders.

One general compute shader works in all cases and performs one task without synchronization for each output texel and uses [numthreads(16, 16, 1)] (= 256 threads/group which is a multiple of NVidia's warp size of 32 and AMD's wavefront size of 64). Algorithmically, this does not result in the best performance, since for MSAA $T$x, each task performs $T$ texture loads from the radiance, normal and depth textures, and for SSAA $T$x, each task performs $T \times T$ texture loads from the radiance, normal and depth textures.

So instead of dispatching jobs in terms of output texels, I have some specialized compute shaders performing tasks with synchronization for each input texel (for SSAA) or for each subsample of an input texel (for MSAA). Here, each task performs $1$ texture load from the radiance, normal and depth textures and stores the results in group shared memory. Next, a GroupMemoryBarrierWithGroupSync takes place and only one job per output texel continues to finish the work. This is not ideal in case of many SSAA and MSAA samples, but for few samples such as MSAA 2x, one expects to see some gains. In practice, however, the general compute shader outperforms all the specialized compute shaders for both small and large numbers of samples/ouput texel?

SSAA 2x [numthreads(4, 16, 16)] = 1024 (multiple of 64): ~655 FPS vs ~577 FPS

SSAA 3x [numthreads(9, 8, 8)] = 576 (multiple of 64): ~265 FPS vs ~200 FPS

SSAA 4x [numthreads(16, 8, 8)] = 1024 (multiple of 64): ~127 FPS vs ~75 FPS

Can this be solely due to dispatching more threads/group? Can this be due to the synchronization of all threads per group (close to the maximum allowed number of threads)? Or is there some performance one needs to sacrifice in case of using a 3D instead of 2D block of threads per group?

Answer 1

There shouldn't be any performance penalty inherent in using a 3D group over a 2D one, the dimensions are just an aid for programmers to map invocations on to memory locations (since you can read or write wherever you like).

The group size affects which invocations can access the same block of groupshared memory, so it forces all warps from the same group to run on the same SMP. If you have a shader that needs to move stuff around a lot in its local memory block (e.g. a bitonic sorter), a large group size is preferable, but in your case it shouldn't make much difference.

I suspect your problem is with the idle threads within your warps. If I understand your question properly, you're using a thread per input texel or sample to do the texture reads, but then only a subset of these threads to do the processing? If so, you're sacrificing a lot of compute power within each warp. Remember, texture reads are done in minimum 2x2 blocks, and the results get cached in a way optimized for locality for the dimension of the texture (see this question). So although your general purpose shader might seem slow with all the texture reads, a lot of them will be effectively almost free. Also, the GPU does latency-hiding on slow memory reads, swapping in other warps until the result of the read is available. So your general purpose shader actually seems like the smart option here, as it maximizes compute power.