# What is the definition of “motion to photon” in VR?

The rough definition of "MTP" is clear. But the exact one is not. You can refer to these links:

But from different online articles, we get different definitions regarding how the value is measured:

1. From Atman Binstock (Oculus) blog, we can see:
MTP = T2(time to flip) - T1(Time of PO).
But, T2 is just beginning to display the image, not finish displaying.

2. From John Carmack (Oculus) blog:
MTP = T2(finish displaying) - T1(Time of PO).

3. From Alex Vlachos (Valve) slides: Advanced VR Rendering, GDC 2015:
MTP = T2(finish displaying) - T1(Time of PO)

4. From Qualcomm blog (sorry it is in Chinese):
MTP = T2(half of finish displaying) - T1(Time of PO)

I think Qualcomm's explanation is the best. If it is correct, we can relax the rendering time budget. What is your opinion?

• @Julien Guertault, Thank you for refining the format. I don't know how to do that:) – Hao Zhang Aug 22 '16 at 0:34
• At your service. :) Here's an overview if you're interested: stackoverflow.com/editing-help – Julien Guertault Aug 22 '16 at 2:16

Ideally, motion-to-photon latency is exactly what it says. It measures time from when the user physically makes a motion (say, begins to turn their head) to when the photons physically coming out of the display change to reflect that motion.

With a bit of apparatus, this can be measured directly—for instance by using a high-speed camera aimed at the headset. You give the headset a light bump with your finger (or use some automated device to bump it), and you can see in the high-speed video when the motion occurred and when the image visible on the screen updated to reflect the motion.

The complication leading to different definitions occurs because many VR headsets have rolling displays, where the physical pixels on the display light up progressively, one row at a time, as the image is scanned out from the host PC. So, when you define motion-to-photon latency, you have to say which photons you're talking about. The very first ones to scan out? The very last ones? Somewhere in the middle?

You can pretty much make whichever definition you want here about what point in the scanout process to use as your reference. There may be an argument for one definition or another based on psychovisual effects (i.e. how much of your visual field must be updated in order for your brain to perceive motion?) but this is beyond my knowledge.

On the other hand, if a headset has a global display instead of a rolling one, then the pixels are kept dark while being updated and then the whole display is flashed on and off at the end of scanout. In this case it's clear that the appropriate reference point to measure is the beginning of the flash, since all the photons come out at the same time.

Finally, it's worth noting that latency measurements incorporated into an app or API are approximations. The software can't observe the motion nor the photons directly, so instead we must use software-visible proxy events.

• The app querying for the latest head pose is a proxy for user motion, but note that this discards any latency in the sensors, USB driver, sensor fusion processing, and time the data spent sitting around in a buffer waiting for the app to query it.
• The GPU flipping its display buffers is a proxy for photons beginning to come out, but this discards any latency added in the scanout and display process.

So, ideally an app-measured latency figure should include some corrections to at least estimate the above factors. Or accept that you're not really measuring the full motion-to-photon latency, but just the app-query-to-flip latency or some such.