# Transforming fragment-to-camera to tangent space for parallax occlusion mapping

I'm trying to implement POM using this tutorial. As stated, I need to transform fragment-to-camera to tangent space and then pass it to a function.

• tangent space is computed using Assimp. Well I trusted it. For example tangent vectors look like that (Sponza model): • then in vertex I make transformation as in tutorial:

layout (location = 0) in vec3 vPosition;
layout (location = 1) in vec2 vTexCoord;
layout (location = 2) in vec3 vNormal;
layout (location = 3) in vec3 vTangent;
layout (location = 4) in vec3 vBitangent;
...
worldPos = matrices.modelMatrix * vec4(vPosition, 1.0);
//transform t, b, n from object to world (also called model) space?
vec3 t = normalize(mat3(matrices.modelMatrix) * vTangent);
vec3 b = normalize(mat3(matrices.modelMatrix) * vBitangent);
vec3 n = normalize(mat3(matrices.modelMatrix) * vNormal);
mTBN = transpose(mat3(t,b,n)); //transpose makes it from model to to tangent space?
tanViewPos = mTBN * cameraPos;
tanFragPos = mTBN * worldPos.xyz;

• then in fragment I pass parameters to function:

vec3 viewDir = normalize(tanViewPos - tanFragPos);
texCoord = steepParallaxMapping(vTexCoord0, viewDir, textureHeight);

vec2 steepParallaxMapping(vec2 texCoords, vec3 viewDir, sampler2D textureHeight)
{
float height_scale = 0.09;
// number of depth layers
const float numLayers = 30.0;
// calculate the size of each layer
float layerDepth = 1.0 / numLayers;
// depth of current layer
float currentLayerDepth = 0.0;
// the amount to shift the texture coordinates per layer (from vector P)
vec2 P = viewDir.xy * height_scale;
vec2 deltaTexCoords = P / numLayers;
// get initial values
vec2  currentTexCoords     = texCoords;
float currentDepthMapValue = texture2D(textureHeight, currentTexCoords).r;

while(currentLayerDepth < currentDepthMapValue)
{
// shift texture coordinates along direction of P
currentTexCoords -= deltaTexCoords;
// get depthmap value at current texture coordinates
currentDepthMapValue = texture2D(textureHeight, currentTexCoords).r;
// get depth of next layer
currentLayerDepth += layerDepth;
}

// get texture coordinates before collision (reverse operations)
vec2 prevTexCoords = currentTexCoords + deltaTexCoords;

// get depth after and before collision for linear interpolation
float afterDepth  = currentDepthMapValue - currentLayerDepth;
float beforeDepth = texture2D(textureHeight, prevTexCoords).r - currentLayerDepth + layerDepth;

// interpolation of texture coordinates
float weight = afterDepth / (afterDepth - beforeDepth);
return prevTexCoords * weight + currentTexCoords * (1.0 - weight);
}

Thats the result (simple displacement): Or other texture (complex): I'm doing something wrong. And I wonder why we multiply by 3x3 part of the model matrix?

EDIT:

I've found function which calculates tbn in fragment:

mat3 compute_tangent_frame(vec3 N, vec3 P, vec2 UV)
{
// get edge vectors of the pixel triangle
vec3 dp1 = dFdx( P );
vec3 dp2 = dFdy( P );
vec2 duv1 = dFdx( UV );
vec2 duv2 = dFdy( UV );

// solve the linear system
vec3 dp2perp = cross( dp2, N );
vec3 dp1perp = cross( N, dp1 );
vec3 T = dp2perp * duv1.x + dp1perp * duv2.x;
vec3 B = dp2perp * duv1.y + dp1perp * duv2.y;

// construct a scale-invariant frame
float invmax = inversesqrt( max( dot(T,T), dot(B,B) ) );
return mat3( T * invmax, B * invmax, N );
}


First Case: So if I use that function like that:

//in vertex
vFar = cameraPos - worldPos.xyz;
//in fragment
mat3 TBN = (compute_tangent_frame(vNormal0, normalize(vFar), vTexCoord0));
vec3 viewDir = (transpose(TBN)*normalize(cameraPos - worldPos.xyz));
vec2 texCoord = steepParallaxMapping(vTexCoord0, viewDir, textureHeight);
vNormalExtr = normalize(texture2D(textureNormal, texCoord).rgb*2.0 - 1.0);
normalMapColor = vec4(normalize((TBN) * vNormalExtr), 1.0);


parallax (all walls) and normal light seem to be ok: Second Case: But if I use my TBN from CPU:

//in vertex
mat4 model = ((matrices.modelMatrix));
vec3 t = normalize(mat3(model) * (vTangent));
vec3 b = normalize(mat3(model) * (vBitangent));
vec3 n = normalize(mat3(model) * (vNormal));
mTBN = ((mat3(t,b,n)));
//in fragment
mat3 TBN = mTBN;
vec3 viewDir = (transpose(TBN)*normalize(cameraPos - worldPos.xyz));
vec2 texCoord = steepParallaxMapping(vTexCoord0, viewDir, textureHeight);
vNormalExtr = normalize(texture2D(textureNormal, texCoord).rgb*2.0 - 1.0);
normalMapColor = vec4(normalize((TBN) * vNormalExtr), 1.0);


parallax is still twisted on all walls but normal light seem to be ok: If I dont transpose tbn (viewDir) (the correct way You said) in second case (mTBN)

vec3 viewDir = ((TBN)*normalize(cameraPos - worldPos.xyz));


then some walls seem to be correct, others not...

EDIT:

Adventures in tangent space... Is there a badge of longest question ever or biggest number of pictures?  • What are the walls that don't seem correct? How do they look? How do their N and T look? Oct 1, 2016 at 2:39

Without looking at the code, from the look of the pictures, my intuition would be that somewhere the code is using the inverse of the matrix it should.

Looking at the code seems to confirm so, but there are some other problems too.

mTBN = transpose(mat3(t,b,n)); //transpose makes it from model to to tangent space?


mat3(t,b,n) is a transform from world space to tangent space. The inverse of such a matrix is indeed its transpose, so mTBN transforms from tangent space to world space, which is not what you want.

tanViewPos = mTBN * cameraPos;
tanFragPos = mTBN * worldPos.xyz;


You cannot do something like that. mTBN is a 3x3 matrix: without a 4th column and a non-zero homogeneous coordinate, you can only transform directions, not positions. You need to first get the camera direction relative to the vertex, then transform that direction.

// Beware of the vector direction.
// Here viewDir points from the object toward the viewer.
vec3 worldViewDir = normalize(cameraPos - worldPos.xyz);
tanViewDir = mTBN * worldViewDir;