Latency difference between texture copy on GPU vs CPU?

Question

Does anyone happen to know the latency difference between copying a texture on cpu (using, say, memcpy) compared to gpu (using, say, CopyResource)? I would do a perf test myself, but I'm not actually sure how to do the trace for the CopyResource. A rule of thumb would be fine, it doesn't have to be exact, I just figure that the GPU is faster than the cpu for this but am just curious how expensive a copy of, say, a 1920x1080 texture is.

Answer 1

You are talking apples and oranges here. Copying memory to memory on the CPU is using completely different hardware then memory to memory copies on the GPU, even in a unified memory architecture GPU's tend to have specialized memory.

The actual difference is going to depend heavily on the model of the GPU, a high end gddr5 gpu will easily beat DDR4 dual channel memory on the CPU.

Also, the driver implementation is going to have an effect on the gpu copy since you are really asking the driver for the copy on the gpu, which it then translates into code that does the actual copy.

The size of the chunk of memory also plays a major role in how fast the copy happens, as well as how large the cache on the cpu vs gpu is.

You should generally expect copies that are GPU to GPU to be much faster then CPU to CPU copies. CPU to GPU along with GPU to CPU copies have to cross the AGP bus on the PC while unified architecture systems have an advantage they still have synchronization issues and all systems have to synchronize access and flush cache's.

What are the actual differences?(very roughly) DDR4 runs from 1600 to 3200 MB per second and DDR5 run's at roughly 3200 to 6400 MB per second. So very roughly, a poorly balanced system will have up to 4x performance diffence, but you should expect about a 2x performance differnce with the GPU being faster. This difference fades somewhat(mostly) with unified systems. (But the gpu will probably still have memory set aside so it can work with framebuffers and what not in an efficient manner)

Answer 2

You can use Query for mesuring time on GPU. Idea is follow: Create buffer with two int64 values. Write time stamp before and after CopyResource() on GPU to this buffer. Then retrive values on CPU and substract them. Substraction is time on GPU.

Answer 3

For anyone wondering how to deal with it, please refer to mjp (Matt Pettineo) github - https://github.com/TheRealMJP/DXRPathTracer/blob/master/SampleFramework12/v1.02/Graphics/Profiler.cpp

MJP's code is DX12. For DX11, you can refer to my github - https://github.com/komilll/dxFramework/blob/master/dxFramework/Profiler.cpp

In both DX11 and DX12, you basically just creates timestamps at the beginning and at the end of some portion of code. Then you have to resolve it, based on GPU speed. GPU might work with different clock frequency based on current workload. When you have a full PBR engine with shadow or SSAO, you should already have no problem with changing clock. However, if you're just starting creating your engine/framework, keep in mind that time to execute might not be precise, because you're not using full GPU power.

In DX12 there is ID3D12Device::SetStablePowerState but I've seen multiple times that people are recommending not to use it. Maybe there are some use cases, but for typical hobby usage, I won't bother with that.