# Learning light transport using Q-Learning

I am trying to reproduce the results obtained by Dahm et al. in the paper Learning Light Transport the Reinforced Way.
This method takes advantage of the similarity between the Bellman equation (Q-Learning) and the rendering equation.
Here's the overview of the code authors inserted in the first version of their paper:

Question 1

What is, in the first function, idxPrev? Intuition suggests it is the index of the previous cell (the scene space needs to be discretized, using for example the Voronoi algorithm). But in the function, authors pass ray.o, which I believe is the outgoing ray. So, that would not make sense.
EDIT: ray.o is probably the origin.
Moreover, in the same function, I understand that getLastAttenuation() returns the color of idxCurr, but i don't understand what qmax[idxCurr] returns (intuitively, the action so the stratum of the hemisphere that maximizes the Q-value).

Question 2

The action space is discretized, apparently, dividing the hemisphere built on the hitting point in different stratums. According to what I understand, the Q-Learning algorithm finds the best action (so the best stratum) for each cell in the space, and then a ray is scattered randomly in that stratum. But a stratum covers 360 degrees, so I don't see how this is helpful to scattered a ray towards the light.
In the function sampleScatteringDIrFromQtable(), the PDF is calculated based on the ps (not clear) multiplied by the number of patches. So, the action is related to circular stratums of the hemisphere, or the number of patches?

• ray.o almost always denote the origin, for what it's worth. – Hubble Apr 13 at 17:20
• @Hubble Thank you, so the first point is clear. – maurocomi yesterday