Blit and scroll

Question

Could anyone please explain to me how drawing works with respect to scrolling?

Suppose there is a window, which has an area where one can draw to (a "canvas"). Are there two copies of this canvas? One for CPU and one for GPU? I mean, I know one could copy from main memory to graphic card memory very quickly with OpenGL (PBO) so one way to draw is first draw with software in main memory and the "blit" it to GPU, right?

Now, I would assume that when the window scrolls there is some GPU functionality which would rapidly copy over the non-dirty parts in the appropriate positions without leaving GPU-land and then ask the CPU for redrawing the dirty/missing parts. But now I have two un-synced copies of the canvas! So how does it work? Do I do the scrolling in main memory in software and upload the whole canvas to GPU as the first time? Or perhaps I scroll the canvas in GPU and "download" then download it to CPU-land and "repair" it there? Or something else entirely? Perhaps somehow I don't have to care that the copies are unsynchronized?

Answer 1

Traditionally, there are two ways to display something on the screen:

1. Re-draw it from scratch every frame (or, every time something changes)
2. Track the sections of the screen (ie. "dirty rectangles") that change each frame, and update only those portions.

The first option is what most modern games / animations will do. Each frame, you re-submit all the draw calls / re-upload any textures to the GPU that have changed, then ask the GPU to draw everything and show it on the screen.

The second option is what most old 2D games did, as well as most window applications (ie. GUI programs that don't have a lot of change going on). One implementation would be to have a local CPU copy of the screen, a GPU copy, and a series of buffers to transfer data between them. So for every frame where there were changes:

1. Blit pixel data of the dirty rectangle areas to one of the data transfer buffers.
2. Copy the buffer to the GPU
3. Use the GPU to blit the pixel data from the data transfer buffer to the GPU copy.
4. Present

For more info, I suggest looking up how some window managers do compositing. There is a decent explaination of how WPF works here. And an excellent post about how Chrome accelerates their browser rendering here.

Answer 2

Let's take web browsers as an example. On mobile, both Chrome and Firefox draw the page content on the GPU, and they typically have a lot of scrolling, so it's a relevant example.

They work a little like Google Maps or Open Street Map's tileserver. They slice the page up into tiles and draw each tile into a texture. (The textures may well be in a texture atlas; I'm afraid I don't know the specifics.) Drawing the final screen simply consists of drawing a small number of textured squares in the correct positions. This approach makes scrolling very easy.

When you scroll, it doesn't have to regenerate the whole page starting from the DOM (the web page equivalent of the scene graph): it just moves the quads around so that each textured tile moves around the viewport. When a new row or column of tiles is about to come into view, the page renderer has to draw the new tiles before you want to see them. At the other end of the screen, you can keep tiles that are no longer in view, but they can be disposed of if you start to run out of texture memory. Because all the tiles are the same size, you can simply draw the new tiles over old textures you don't want any more.

It's just the same as in Google Maps: when you scroll in that, you can usually see a placeholder texture if new tiles are visible before they've finished downloading. And the page keeps a cache of tiles you've looked at before, but it can destroy any that are no longer visible in order to free some memory if necessary.

The tile technique also allows progressive refinement, which is important to let you scroll at 60 fps on a mobile platform without enough compute power. If it's rendering the tiles too slowly, it can draw the textures at lower resolution first (while you're scrolling), and then redraw them at full resolution later, if they stay on screen.

In short, there isn't any special functionality for blitting some of the last frame into a new position in the new frame, but by rendering to texture, we can access the same texture from the next frame, throwing away any parts that are no longer useful. Some games also use this technique for mostly static content backgrounds, 2D world maps, &c.

If you have to update small portions of the scrolling content, it's easy: just redraw the tiles that changed. The code that updates the tiles doesn't care what screen position they're being displayed, because the final frame renderer puts each tile in the right place. You don't have "two un-synced copies of the canvas" and it doesn't matter whether you're drawing each tile on the CPU or GPU (or both).

The word "static" there was important. If much of the background is changing every frame, you don't get any advantage from tiles, because you have to update every tile every frame. In that case, you should just throw away the old frame and draw the new frame from scratch. That's why many 2D games don't use this technique even if they show scrolling content (e.g. a Mario game).

In summary:

• If your scrolling content is completely static and small enough to fit in memory, draw it in a big texture and render a textured quad per-frame.
• If your scrolling content is too big to fit in memory, or small regions of it need updating sometimes, draw it in tiles, render all the visible tiles per-frame, and add something to load and unload tiles when necessary.
• If most tiles change most frames, just draw the visible region every frame.

Knowing which case you're in, and being able to separate out the static parts from the changing parts, constitutes a lot of the work of the graphics developer :-)