Monte Carlo Importance Sampling

Question

I am following the "Ray Tracing - the Rest of your Life" book by Peter Shirley and I am facing some troubles in the implementation of Monte Carlo importance sampling, and Direct light sampling.
Monte Carlo theory is clear to me. During the implementation, the way rays are scattered does not actually change and remains random. What changes looks to be the contribution of each ray.
So my first question is:

Do I understand this correctly? Rays, after hitting a surface, are still scattered randomly, but the value of the color is adjusted based on the probability of that direction?

For those familiar with the book
When the author implements the Direct light sampling, he modifies the function color(), while we previously modified scatter() and scattering_pdf(). This way, scatter() and scattering_pdf() are sampling based on a specific pdf - proportional to cos(theta) - while color() introduces a new pdf related to the direction of the light. It is not clear to me how these two things coexist, and if I am supposed to disregard the previous changes to scatter() and scattering_pdf().

Answer 1

During the implementation, the way rays are scattered does not actually change and remains random.

Actually the way rays are scattered does change, specifically when you sample a light. In chapter 8 he makes a mixture pdf in order to sample either the light or the bsdf.

What changes looks to be the contribution of each ray.

This does change, but not specifically due to the sampling of the lights but rather due to importance sampling. But that was already introduced in chapter 4.

Basically you have the following: $$I = \int_{\Omega}f(x)\,dx = \int_{\Omega}\frac{f(x)}{p(x)}p(x)\,dx = E\left[\frac{f(X)}{p(X)}\right] = \frac{1}{N}\sum_{i=1}^{N}E\left[\frac{f(X_i)}{p(X_i)}\right] \approx \frac{1}{N}\sum_{i=1}^{N}\frac{f(x_i)}{p(x_i)}$$ Where $X$ and $X_i$ are distributed according to the probability density $p$. As evident the main difference to the Monte Carlo formulation you are probably used to is the $p(x)$ factor, and the fact that $x_i$ are not uniformly sampled but rather sampled according to $p$. Note that for uniform sampling $p(x)=1$ and the equality still holds. Thus this is simply a generalization.