It's the algorithm which combines path tracing and reinforcement learning. I can't understand what $p_\omega$ is. 

[![enter image description here][1]][1]


  [1]: https://i.sstatic.net/PWbSf.png

The algorithm is clear. The actions are the directions and the states are the hit points. For updating the Q-values, the hit points are considered as parts of a discretized diagram like Voronoi diagram and the directions of rays are generated by a discretization of a hemisphere surface which divides it into equal areas. The Q-values are initialized as equal for each state. 

First, we generate a camera ray, and when it hits a diffuse surface, it scatters according to a discrete PDF which is calculated by normalizing the Q-values of the state of hit point. 

Second, The Q-values are updated along the path, we have the state $s$ and the next state $s\prime$ is generated from the direction generated by PDF of Q-values in $s$. 

The area of each path on the hemispheres is $2\pi/n$, and integral in the update equation of Q-values is calculated like this:

$\dfrac{2\pi}{n}\sum_{k=0}^{n-1}Q_k(y)f_s(\omega_k, y, -\omega)cos\theta_k$

I understand all of the algorithm, but I don't know what $p_\omega$ is in the second part. Is it $1/2\pi$ or $n/2\pi$ ? Could someone prove what it really is? I need the proof. 

I guess it is $1/2\pi$ because the path tracer here only use the PDF generated by Q-values. It doesn't use the Q-values at all. Am I correct? If I am correct, how can I prove it?