Image based lighting, tangent space coordinates, and optimization

Question

When I saw an implementation of normal mapping that computed the TBN matrix in the vertex shader and converted everything (in particular the view vector and light vector) to tangent space at that stage, it seemed like the proper way to do. This way there is no matrix-vector multiplication left to do in the fragment shader.

Enters image based lighting: for each fragment we want to fetch from an image, typically one or more cube map textures, the light intensity coming from certain directions. I assume this data is in world space.

My question is: is it possible to use IBL while retaining the vertex shader transform optimization mentioned above?

• It seems to me we still need to transform our fetch direction from tangent space to world space (or whichever space the cube map is in).
• Conversely, if we want to work in world space, like with deferred shading, we need to transform the normal for each fragment.

Does IBL necessarily imply at least one matrix vector multiplication per fragment and the optimization I mentioned is an uncommon case, or am I missing something?

Answer 1

Yes, you do need to transform the fetch direction into the space of the cubemap. If you could somehow figure out the fetch direction in the vertex shader, then you could do the transformation there instead, but that would produce worse lighting.

It also may be worth optimizing for a smaller number of interpolants between the vertex and pixel shaders (rather than a smaller number of matrix-vector multiplications in the pixel shader). As soon as you have one transformed view vector and two transformed light vectors that must be interpolated (which would otherwise be constant), you've matched the size of the tangent/bitangent/normal matrix that you would otherwise be interpolating. That has the potential to have more performance impact than doing extra matrix multiplications in the shader.