I've heard a lot of people working on VR talk about scanline racing and that it's supposed to help improve latency for motion-to-photon. However, it isn't clear to me how this can be done with OpenGL. Could someone explain how scanline racing works, and how it can be implemented on modern GPUs.
When your GPU displays a new frame on the screen, it transfers the image over the HDMI cable (or whatever kind) in a process called "scanout". The pixels are sent out in linear order, usually left-to-right and top-to-bottom. The process is timed so that it takes most of the duration of a refresh interval to do this. For instance, at 60Hz, one frame is ~17 ms. Each scanout will take probably around 15-16 ms, with 1-2 ms of vblank in between (the exact values vary according to the display and video mode).
Traditionally, rendering is double-buffered, which means there are two buffers stored in GPU memory: one that is currently being scanned out ("front buffer"), and one that is being rendered to ("back buffer"). Each frame, the two are swapped. The GPU never renders to the same buffer that's being scanned out, which prevents artifacts due to potentially seeing parts of an incomplete frame. However, a side effect of this is increased latency, since each frame may sit around in the buffer for several ms before it starts being scanned out.
VR is very latency-sensitive, so this isn't desirable. An alternative approach is to render directly to the front buffer, but time things out very carefully so that you have rendered each line of the image shortly before the scanout gets there. That's called "scanline racing" or "racing the beam" (the "beam" harkening back to the CRT days of yore). This more or less requires that you render the image in scanline order, i.e. the same order that the pixels get scanned out. It doesn't literally have to be rendered one line at a time—it could be rendered in thin strips a few pixels high, but it does have to be done in order, as you can't go back and edit pixels that have already been scanned out.
There are a lot of disadvantages to this approach; it has very stringent performance requirements, has to be timed very carefully against vsync, and it greatly complicates the rendering process. But in principle it can shave milliseconds off your latency, which why VR folks are interested in it.
The great thing is we can finally predict scan-line exact raster accuracy with no access to a per-scanline query:
I have come up with the exact microsecond-accurate formulas as a VSYNC offset, to predict the position of a tearline. Tearlines during VSYNC OFF are always raster-exact, so you can steer them out of visibility during strip-level "simulated front-buffer rendering" via repeated VSYNC OFF buffer swapping.
Pay attention to forum thread -- there is some open source code being continually added -- https://forums.blurbusters.com/viewtopic.php?f=10&p=32002
If it's of interest, the Dreamcast had a "racing the beam" rendering mode, whereby it was able to dedicate a relatively small fraction of memory to framebuffer pixels (e.g. 64 scan lines) and it would render rows of 32 in turn, synchronised to the display update. This however, was only used to save memory. I doubt anyone was generating "modified" geometry for latter parts of the display.