1
$\begingroup$

I want to know about "If there is branch command in compute shader, and it leads some workgroup ends earlier than others, then will the 'processing units' allocated for that workgroup be used for another workgroup which is not processed yet?(before complete of other still running workgroups)"

I know this is not important for implementation, but I want to know about performance factors.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

Yes, the workgroups are independent of each other. This means that when one workgroup has finished its work, it is released and the next workgroup can start its work while other workgroups are still in progress. Even better! If there are two different compute shaders (different code) that need to be executed, the released workgroup capacity can be used by the second compute shader workgroup while the last workgroups of the first compute shader are executed at the same time.

When branching, the total calculation time of a compute shader can vary. Let's take a look at an example:

Example:

Let's assume we have 20 workgroups whose program code has a branching option. And only 1 workgroup takes 1.5 times longer because of this branching, let's call it workgroup 1. Let's assume that your GPU can process 19 workgroups simultaneously. So the GPU loads 19 workgroups and after one of the workgroups has finished its work, the last workgroup is loaded.

The minimum execution time of all workgroups would be: $2 * x$, where x is the time of one of the 19 fast workgroups.

But you can also be unlucky. That is, the 19 fast workgroups are executed first, and workgroup 1 (the slow one) is executed at the end. Then your execution time is: $x + 1.5 * x = 2.5 * x$.

To be honest, there is a time offset for loading the workgroups and memory etc. which is not specified in this form to keep it clear.

Improve this answer
$\endgroup$
0
$\begingroup$

If we only talk about individual compute shaders. Then, inside a wave, the entire wave will be held up waiting for all compute shaders inside that wave to complete. The compute shaders in a wave run in lockstep, branching causes divergence and can effect performance per a wave. We don't work with waves at the API level though, instead we work with workgroups. A workgroup can consist of multiple waves depending on the settings, so a new workgroup could get held up if there isn't enough waves available to send a new workgroup.

Finding out the wave size of a GPU and setting workgroup size to be a multiple of the wave size (usually 1x or 2x) can help mitigate the issue and help overall performance.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for detailed answer! If we also consider waves, then would above answer be "one waves per workgroup" case? $\endgroup$
    – idev
    Commented Mar 8 at 3:05
  • $\begingroup$ I wouldn't make that assumption. We need to know the wave size, the max wave count, the workgroup size and the total work groups to figure that out. We don't have the workgroup size in this case. In the example thomas gave it would only take a single compute shader to hold up the slow workgroup is the point of my little extension to the @thomas answer. $\endgroup$
    – pmw1234
    Commented Mar 8 at 12:02
  • $\begingroup$ in my answer, i only focused on the workgroups because i wanted to keep the wave topic out of it. My example assumes that a workgroup has 32(Nvidia) or 64(AMD) invocations and thus represents the entire wave. The maximum number of waves was assumed to be 19 (this number is of course completely random). The example should only show that the execution of the individual workgroups is not ordered and therefore the calculation times can vary. $\endgroup$
    – Thomas
    Commented Mar 8 at 12:39
  • $\begingroup$ I see. Thanks for clearing up misconception. $\endgroup$
    – idev
    Commented Mar 25 at 8:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.