Through reading various sources online, I've become a bit confused. I'll briefly outline what I think I do understand:
My understanding is that microfacet-based BRDFs assume there is always a perfect reflection, but the true microscopic normal changes according to some Normal Distribution Function (NDF), resulting in a non-perfect macroscopic reflection. When ray tracing, you sample this NDF to generate a "true" normal, and then you can compute a bounce ray using your incident ray, and a specular reflection with your "true" normal.
My confusion comes from a few things:
What is the reason that some microfacet-based BRDFs are diffuse only (such as Oren-Nayar) or specular only (such as Cook-Torrance)? My assumption would be that if a roughness value of 0 were used, all of the normals should align and thus you would get a purely specular reflection. Similarly, if a roughness value of 1 was used, the normals would be all scattered, and you'd obtain a more diffuse reflection. Yet, Wikipedia states for the Oren-Nayar model: "In the case of $\sigma = 0$ (i.e., all facets in the same plane), and thus the Oren-Nayar model simplifies to the Lambertian model" If all of the microfacets align to the same plane, how could the computed reflection still be diffuse? What happens when I sample a bounce ray, as surely if the normals are all aligned, the sampled ray should be specular?
If Oren-Nayar is a BRDF, and Cook-Torrance is a BRDF, but you need both to fully represent a single material's diffuse and specular components, is the combination of them also a BRDF? Is combining simple BRDFs that capture only a single aspect of reflection how you create a full PBR BRDF? (Looking at the OpenPBR white-paper, it seems that a single material is defined as a mixing of multiple simpler BRDFs)
If you require multiple BRDFs to describe a single material, do they share the same NDF? When path tracing, how do you sample a bounce ray? If you are always simply performing a specular reflection on some sampled normal, how would you importance sample a BRDF or a light source?
I think I am getting confused by the fundamental difference between a BRDF and an NDF, and the roles they play.
Edit:
Someone has pointed out that microfacet-models don't all assume that reflections are specular, some (like Oren-Nayar) assume they are lambertian. But it still seems like a collection of misaligned specularly reflecting microfacets would produce a diffuse effect (where, as roughness increases, you'd slowly transition from a perfect mirror, through glossy "soft" reflections, to diffuse). So why is a separate diffuse component needed? What is the conceptual reason for the distinction?
I have seen it said many times, that in reality all reflections, at the smallest scale, are only specular. Is it that Cook-Torrance is modeling each microfacet as a perfectly smooth mirror, but Oren-Nayar is modeling each microfacet as itself being composed of much smaller microfacets exhibiting diffuse behavior?
I'm still struggling to understand why both components are needed separately, and how a bounce ray is actually to be sampled. What does a roughness value even mean then, if a perfectly smooth surface still has a diffuse component from the Oren-Nayar model?
Edit 2:
After reading some more on the fresnel effect, as well as this answer on another question, I believe I understand what is happening now.
The Fresnel equations describe the reflection and transmission of light when it is incident on an interface between two different mediums. It tells us how much of the light is transmitted into the new medium, and how much is reflected away. The reflection is always specular, but can appear more or less diffuse based on how well-aligned the microfacets are (e.g. how rough the surface is). The refracted ray that is transmitted into the material, if the material is opaque, quickly interacts with the internal structure and is reflected back out in a different direction, hence appearing diffuse. But this is separate from the diffuse nature caused by the microfacets on the surface. A colleague of mine included this diagram in his dissertation:
(It isn't 100% clear to me how it differs from subsurface scattering. Perhaps, the only difference is the depth the rays are able to penetrate?)
So when we write the BRDF like this:
$$ f(\omega) = \biggl((1 - F)f_{\text{Lambert}}(\omega) + Ff_{\text{C-T}}(\omega) \biggr)\cos\theta $$
we're using the Fresnel equations to tell us how much light is going to be reflected off the surface (specularly, off the microfacet structure). The remainder of the light ($1-F$) is transmitted into the structure but then quickly reflected back out diffusely.