# Screenspace Normals - Creation, Normal Maps, and Unpacking

I'm trying to condense my Deferred Rendering G-Buffer. So I have some questions about getting 2-component Screenspace Normals. I know Frostbite and Killzone (the only two AAA company's G-Buffers I could find) use them.

How are screenspace normals created, and is this step before or after using normal maps or bump maps? If it's done before using normal maps, how are normal maps going to be affected by the screenspace-ness of the normals, and if after, how can you justify using a Model-View matrix on all fragments rather on vertices? Isn't that a lot more calculations?

Finally, how are they unpacked? I realize you can get the blue component using pythagoras, but how are they returned to world space?

How are screenspace normals created, and is this step before or after using normal maps or bump maps?

They are created after using normal maps. In deferred rendering, you write to the various buffers (diffuse, normal, depth, etc.) in fragment shaders. By this time, the normal maps would have already been read.

If it's done before using normal maps, how are normal maps going to be affected by the screenspace-ness of the normals,...

If you are referring to normal maps on models, normal maps can stay the same. You can compress normal maps as well if you want, but this becomes irrelevant to what you do for writing to the normal buffer.

...and if after, how can you justify using a Model-View matrix on all fragments rather on vertices? Isn't that a lot more calculations?

Your model's normals are computed per vertex and interpolated through rasterization to be per fragment. The normal maps are processed per fragment, but these are just texture lookups, and these can be applied to the camera-space normals produced from the previous pipeline steps. Therefore, a view matrix is no longer necessary.

Finally, how are they unpacked? I realize you can get the blue component using pythagoras, but how are they returned to world space?

The idea behind 2-component normals versus 3-component normals is that the vector magnitude isn't often taken into account for normals. Also, normals facing away from the camera generally aren't useful. With two components, you can use model a hemisphere facing the camera. You unpack them the same way you pack them.

Think about normals as positions. When converting from world space (x, y, z) to screenspace (x, y, depth), but the depth does not effect where on the screen a mesh gets displayed. Convert the 3D vector to a vector in terms of screenspace (through view matrix multiplications).

Then you can discard the z-component (aligned with the camera) as long as you normals are unit vectors (because you can derive z from the other two vectors).

• Thanks Aces, but I'm still a bit confused. You multiply each fragment by the view Matrix after the usual normal mapping, pass the x and y to the deferred shader, and then multiply by inverse the view Matrix during the deferred shader? – Karim Abdel Hamid Sep 3 '16 at 3:37
• Yes, you can do that, but you can also just calculate the vertex normals through the vertex shader and apply the normal map from there. That way, you don't need to multiply each fragment by the View matrix. – aces Sep 3 '16 at 4:05

I'm trying to condense my Deferred Rendering G-Buffer. So I have some questions about getting 2-component Screenspace Normals. I know Frostbite and Killzone (the only two AAA company's G-Buffers I could find) use them.

I'm confused when you say "screenspace normals", Killzone uses view-space normals, they store the X & Y coordinate of the normal in FP16 format, and reconstruct Z using $z = sqrt(1.0 - Normal.x^2 - Normal.y2)$ a problem with that is that we lack the sign of Z, even in view-space the normals can point away from the camera, i.e. Z is not always positive.

How are screenspace normals created, and is this step before or after using normal maps or bump maps? If it's done before using normal maps, how are normal maps going to be affected by the screenspace-ness of the normals, and if after, how can you justify using a Model-View matrix on all fragments rather on vertices? Isn't that a lot more calculations?

After loading your normal maps, you construct a TBN matrix to transform from tangent space to e.g. world space before writing to the G-Buffer. So in essence, your normals in the G-Buffer are stored in world-space, not screen-space. With that you can apply lighting in world space, as far as I'm aware, Crytek & Epic do the lighting in world-space now.

I'm not sure what you mean by "screenspace-ness", but when writing any sort of data to the G-Buffer, you're essentially writing their current value, if the normals are in world-space, they will remain in world-space when written to the G-Buffer. When you sample the G-Buffer on the second pass, for each texel, you get an RGB color (assuming you're using RGB) that maps to your coordinate data.

• Cannot imagine a normal with negative Z in screen (view) space. It would require the culling turned off which itself is a bad idea from the performance perspective. – Grief Jun 17 '17 at 16:57