1
$\begingroup$

In virtual reality, the motion-to-photon time is very important. Oculus says it has to be less than 20ms. Maybe 10~15ms is better.

Some people try to introduce Frameless rendering technology into VR. I think it's good to reduce the latency. But I don't know how the display quality is. According to the paper: "Adaptive Frameless Rendering" and the video (https://www.youtube.com/watch?v=ycSpSSt-yVs), it seems not very good. But, in this paper "Construction and Evaluation of an Ultra Low Latency Frameless Renderer for VR", the authors used FPGA instead of GPU to render and used Frameless Rendering to reduce the latency to 1ms.

In my mind, FPGA is very hard to replace GPU in 3D rendering area. And, Frameless Rendering needs "racing the beam", it is also very hard to program.

What do you think? Is it realistic to use FPGA and frameless rendering?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ The reason they're using a FPGA, is they require hardware features that the GPU does not have. So they emulate a GPU using a FPGA. $\endgroup$
    – RichieSams
    Commented Apr 6, 2016 at 14:24
  • $\begingroup$ It seems that in second paper the authors used "ray casting" instead of rasterization. So they used FPGA directly, not emulate a GPU? And it also seems they didn't support any API like OpenGL or DirectX. Do you mean the authors just need FPGA to "race the beam"? $\endgroup$
    – Hao Zhang
    Commented Apr 7, 2016 at 6:38
  • $\begingroup$ As Nathan mentioned in his anwer, in order to "race the beam", you need a very tight synchronization of the "graphics hardware" and the OS/CPU. Current GPUs and Graphics APIs don't have this support. So the authors created the hardware themselves using a FPGA. In theory, the hardware they created in FPGA could be manufactured into "real" hardware. $\endgroup$
    – RichieSams
    Commented Apr 7, 2016 at 14:59
  • $\begingroup$ You say "real hardware", does it mean ASIC? $\endgroup$
    – Hao Zhang
    Commented Apr 8, 2016 at 0:31
  • $\begingroup$ No, just any chip. It could be a modified GPU. When I say 'real', I just mean that it's silicon logic, rather than programmed logic, ie. a FPGA. $\endgroup$
    – RichieSams
    Commented Apr 10, 2016 at 15:17

1 Answer 1

Reset to default
5
$\begingroup$

First of all, the "frameless rendering" technique is in the context of raytracing, not rasterization. It's not obvious how it could be made to work effectively with rasterization, given that the basic idea of it is to update an image by a combination of temporal reprojection plus firing rays specifically at areas where the algorithm thinks the image is undersampled.

So this technique is, prima facie, not compatible with rasterization-based graphics applications. But if you're already doing raytracing for other reasons, this technique would be interesting to look at; it certainly appears to improve quality relative to an image raytraced from scratch each frame, with the same number of rays per second.

Raytracing on a GPU is certainly possible; you don't need an FPGA for that. I've only skimmed the second paper, but my reading of it is that the main reason for the FPGA is to get a close coupling between the display scanout and the rendering activity: namely they "race the beam" and evaluate pixels just before they're about to be scanned out, thus obtaining low latency.

The GPU equivalent of this is probably to split the image in thin horizontal strips, and kick off a compute dispatch to render each strip just before it starts to be scanned out. Today, this is difficult to accomplish as it requires either millisecond-precise scheduling that desktop OSes are not currently set up for, or it requires the GPU to be able to dispatch based on an interrupt from the scanout unit—a hardware feature that doesn't currently exist AFAIK (or if it does, it isn't exposed in any graphics APIs). Or you might be able to make it work with a long-running asynchronous compute dispatch, if you can find a way to stay in sync with scanout.

So, there are obstacles, but they aren't insurmountable, and I think if there was sufficient interest in racing-the-beam-style rendering, then OS and GPU vendors could come up with a way to do it in the future. So I don't think an FPGA is required for this kind of technique to work. On the other hand, the fact that it's based on raytracing is a much bigger obstacle to using it in "real-world" apps and games.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The second paper use "ray casting" -- a simplified version of "ray tracing". It could reduce the computing power dramatically. I think the rendering quality is not as good as full ray tracing rendering. But, is it acceptable in most cases? And, as what you said, frameless rendering, or racing the beam, could be implemented as spliting the screen into several blocks and "racing the block" instead of "racing the beam". Of course it need modify the GPU and display. It seems wonderful, does it? $\endgroup$
    – Hao Zhang
    Commented Apr 7, 2016 at 6:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.