# How is lighting done in Rasterization based pipeline?

So I wanted to know how do Game Engines like Unity3D and Unreal do lighting. I'm familiar with Ray-tracing, pathtracing based renderers etc. But not much with rasterization based engines. I do know a general overview of OpenGL's pipeline and things like vertex and fragment shaders but I don't know how exactly the color is computed.

I'm guessing Rasterization based pipelines use the same procedure for shading i.e. A ray is shot from the pixel and then from the triangle that got projected on that pixel to the light source?

For indirect lighting they use things like ambient occlusion I guess. Am I correct?

If this is true then the only difference between rasterization and pathtracing would be the procedure for Visible Surface Determination and Indirect Illumination. Direct Illumination would be calculated the same way in both techniques?

If you can provide a brief overview of the current state of the art techniques used in these engines for things like reflection/shadows etc it'd be appreciated.

• Path tracing and rasterisation are quite different. Path tracing is basically sending out rays in world space from the camera position through its pixel and hoping for it to hit a light source while bouncing about the scene. Rasterisation is transforming all scene triangles into screen space and paint them on top of each other, using all the tricks in the trade to make it look real, including ambient occlusion. Reflections are easy to get correct with path tracing and "impossible" in rasterisation. Writing up how this is done in different commercially available game engines is a huge task... – beyond Jan 25 '19 at 12:36
• I know most of the things you said. I'm not talking about VSD but the shading/lighting part of both techniques which are somewhat similar I guess. For example in rasterization based engines we are still using BRDFs which require an outgoing and incoming direction. I can't think of any other way but using ray-casting to compute this BRDF. This means it's simialr to what we do in path tracing. And what you said -"impossible" in rasterization- This is not quite true. Reflections can be done in rasterization based engines it's just hard to do. – gallickgunner Jan 25 '19 at 12:42
• Well, I wrote "impossible" because it's almost true - I'm sure you get it. You can hack reflections so that it will fool most people while the trained eye will hurt. Even today getting reflections on a non-trivial sphere-flake is hard using rasterisation. Semi-glossy and refractive materials are also a difficult to (m/f)ake. – beyond Jan 25 '19 at 13:04

I'm guessing Rasterization based pipelines use the same procedure for shading i.e. A ray is shot from the pixel and then from the triangle that got projected on that pixel to the light source?

Close, but not quite. There's no actual tracing of rays but the underlying linear algebra is the same. It sounds like you already understand the first part of rasterisation, as far as getting some fragments which need to be shaded. Each fragment knows the attributes of the triangle it came from (the world position, normal, etc.). In traditional forward shading, the fragment shader runs once per light, and it also knows the attributes of the light, including the world-space direction from the shading point to the light: the direction you would trace a shadow ray.

Although no shadow ray is traced, the actual lighting calculation is exactly the same: you use the view and light directions and the surface normal in whatever BRDF you want. You just repeat the process for each light and sum the results. At the end, you have the same result that you'd get from a very simple ray-tracer, without any shadows, reflections, refractions, or indirect lighting.

How to add those missing features? Well, there are several possible techniques for each, but I'll briefly go over the simplest. A simple way of getting shadows is using shadow maps. Before you render your main scene, you do a render into an off-screen buffer, but using the light as a camera. (The projection and shape depend on the light, e.g. orthogonal projection for a directional/cylindrical light.) The trick is, you don't render all the colours: you only draw into the depth buffer. What you get is a picture of the distance from the light to the closest thing in each direction. Later on, when you're shading for that light's contribution in the fragment shader, you calculate the distance from the light to the shading point, and compare it to the value in the depth buffer. That tells you whether the shading point is the closest thing to the light along that ray, without having to run an intersection test.

Planar reflections and refractions are pretty simple too: just render the scene but with the camera transformed to produce the reflection or refraction image. For example, for a planar reflection in a lake or floor, you reflect the camera position about that plane to draw the whole scene upside-down. When you draw the real scene around it, any transparency shows that reflection.

• How do you get the view direction aka the ray from hit point to the eye in a raytracer? Do they just do view_dir = eye - hitpoint and light_dir = light_pos - hitpoint` to get these vectors and just plug in to the BRDF? – gallickgunner Jan 25 '19 at 20:07
• @gallickgunner In a ray-tracer, it's just the direction of the incoming ray. You don't need to calculate it again, you already have it stored. In a rasteriser where there is no incoming ray, it's calculated the way you describe. – Dan Hulme Jan 26 '19 at 10:59
• Yeah that's what I wanted to know. The direction of the rays are calculated so the process of direct lighting is almost the same as in raytracing. Thanks for the answer. – gallickgunner Jan 26 '19 at 11:36