In cases of Laptops, some will have two GPUs. One is built-in (e.g. Intel) and another one attachable (e.g. Nvidia).
Is it better to have each of them operate different software to divide the load?
or
is it better to have one of them operate everything because the switching between GPUs might cause slower performance?
This is more of a long comment than an answer.
As with most yes or no questions in graphics, the answer is "it depends on the task". However, one particular task that you might find having multiple gpu's useful is debugging. For example if you are using cuda-gdb for debugging your CUDA application, it would be nice to have multiple gpu's, one for your application, the other for the display manager of your system. Note that cuda-gdb also works on a single gpu as well, but I had problems with it when I run the debugger and the display manager of the system on a single gpu (screen tends to freeze, no response from the system, etc).
Update 19-01-2021 with respect to comment.
Here is the main criteria that I would use for deciding if the use of single or multiple gpus are better for the task.
GPUs simply execute one task after another, so if you send a heavy task, it will do only that during the process and you won't be able to do anything else with it. This usually means that your system would freeze until the gpu is done with your task. If you are going to launch the task and won't plan to do anything with the machine until it is finished, then sure, the use of multiple gpus would get it done quicker. If you plan to use the machine during the process for other stuff, then it would be more reasonable to allocate a single gpu for the task.
If you are setting up some kind of rendering server where the server would do only gpu related computation, it depends on the type of request you are planning to process by the server.
