2
$\begingroup$

I'm trying to implement POM using this tutorial. As stated, I need to transform fragment-to-camera to tangent space and then pass it to a function.

  • tangent space is computed using Assimp. Well I trusted it. For example tangent vectors look like that (Sponza model):

enter image description here

  • then in vertex I make transformation as in tutorial:

    layout (location = 0) in vec3 vPosition;
    layout (location = 1) in vec2 vTexCoord;
    layout (location = 2) in vec3 vNormal;
    layout (location = 3) in vec3 vTangent;
    layout (location = 4) in vec3 vBitangent;
    ...
    worldPos = matrices.modelMatrix * vec4(vPosition, 1.0);
    //transform t, b, n from object to world (also called model) space?
    vec3 t = normalize(mat3(matrices.modelMatrix) * vTangent);
    vec3 b = normalize(mat3(matrices.modelMatrix) * vBitangent);
    vec3 n = normalize(mat3(matrices.modelMatrix) * vNormal);
    mTBN = transpose(mat3(t,b,n)); //transpose makes it from model to to tangent space?
    tanViewPos = mTBN * cameraPos;
    tanFragPos = mTBN * worldPos.xyz;

  • then in fragment I pass parameters to function:

    vec3 viewDir = normalize(tanViewPos - tanFragPos);
        texCoord = steepParallaxMapping(vTexCoord0, viewDir, textureHeight);

    vec2 steepParallaxMapping(vec2 texCoords, vec3 viewDir, sampler2D textureHeight)
    {
    float height_scale = 0.09;
    // number of depth layers
    const float numLayers = 30.0;
    // calculate the size of each layer
    float layerDepth = 1.0 / numLayers;
    // depth of current layer
    float currentLayerDepth = 0.0;
    // the amount to shift the texture coordinates per layer (from vector P)
    vec2 P = viewDir.xy * height_scale;
    vec2 deltaTexCoords = P / numLayers;
    // get initial values
    vec2  currentTexCoords     = texCoords;
    float currentDepthMapValue = texture2D(textureHeight, currentTexCoords).r;

    while(currentLayerDepth < currentDepthMapValue) { // shift texture coordinates along direction of P currentTexCoords -= deltaTexCoords; // get depthmap value at current texture coordinates currentDepthMapValue = texture2D(textureHeight, currentTexCoords).r; // get depth of next layer currentLayerDepth += layerDepth; }

    // get texture coordinates before collision (reverse operations) vec2 prevTexCoords = currentTexCoords + deltaTexCoords;

    // get depth after and before collision for linear interpolation float afterDepth = currentDepthMapValue - currentLayerDepth; float beforeDepth = texture2D(textureHeight, prevTexCoords).r - currentLayerDepth + layerDepth;

    // interpolation of texture coordinates float weight = afterDepth / (afterDepth - beforeDepth); return prevTexCoords * weight + currentTexCoords * (1.0 - weight); }

    Thats the result (simple displacement):

enter image description here

Or other texture (complex):

enter image description here

I'm doing something wrong. And I wonder why we multiply by 3x3 part of the model matrix?

EDIT:

I've found function which calculates tbn in fragment:

mat3 compute_tangent_frame(vec3 N, vec3 P, vec2 UV)
{
    // get edge vectors of the pixel triangle
    vec3 dp1 = dFdx( P );
    vec3 dp2 = dFdy( P );
    vec2 duv1 = dFdx( UV );
    vec2 duv2 = dFdy( UV );

    // solve the linear system
    vec3 dp2perp = cross( dp2, N );
    vec3 dp1perp = cross( N, dp1 );
    vec3 T = dp2perp * duv1.x + dp1perp * duv2.x;
    vec3 B = dp2perp * duv1.y + dp1perp * duv2.y;

    // construct a scale-invariant frame
    float invmax = inversesqrt( max( dot(T,T), dot(B,B) ) );
    return mat3( T * invmax, B * invmax, N );
}

First Case: So if I use that function like that:

//in vertex
vFar = cameraPos - worldPos.xyz; 
//in fragment
mat3 TBN = (compute_tangent_frame(vNormal0, normalize(vFar), vTexCoord0));
vec3 viewDir = (transpose(TBN)*normalize(cameraPos - worldPos.xyz));
vec2 texCoord = steepParallaxMapping(vTexCoord0, viewDir, textureHeight);
vNormalExtr = normalize(texture2D(textureNormal, texCoord).rgb*2.0 - 1.0);
normalMapColor = vec4(normalize((TBN) * vNormalExtr), 1.0);

parallax (all walls) and normal light seem to be ok:

enter image description here

Second Case: But if I use my TBN from CPU:

//in vertex
mat4 model = ((matrices.modelMatrix));
vec3 t = normalize(mat3(model) * (vTangent));
vec3 b = normalize(mat3(model) * (vBitangent));
vec3 n = normalize(mat3(model) * (vNormal));
mTBN = ((mat3(t,b,n)));
//in fragment
mat3 TBN = mTBN;
vec3 viewDir = (transpose(TBN)*normalize(cameraPos - worldPos.xyz));
vec2 texCoord = steepParallaxMapping(vTexCoord0, viewDir, textureHeight);
vNormalExtr = normalize(texture2D(textureNormal, texCoord).rgb*2.0 - 1.0);
normalMapColor = vec4(normalize((TBN) * vNormalExtr), 1.0);

parallax is still twisted on all walls but normal light seem to be ok:

enter image description here

If I dont transpose tbn (viewDir) (the correct way You said) in second case (mTBN)

vec3 viewDir = ((TBN)*normalize(cameraPos - worldPos.xyz));

then some walls seem to be correct, others not...

EDIT:

Adventures in tangent space... Is there a badge of longest question ever or biggest number of pictures?

enter image description here

enter image description here

$\endgroup$
  • $\begingroup$ What are the walls that don't seem correct? How do they look? How do their N and T look? $\endgroup$ – Julien Guertault Oct 1 '16 at 2:39
1
$\begingroup$

Without looking at the code, from the look of the pictures, my intuition would be that somewhere the code is using the inverse of the matrix it should.

Looking at the code seems to confirm so, but there are some other problems too.

mTBN = transpose(mat3(t,b,n)); //transpose makes it from model to to tangent space?

mat3(t,b,n) is a transform from world space to tangent space. The inverse of such a matrix is indeed its transpose, so mTBN transforms from tangent space to world space, which is not what you want.

tanViewPos = mTBN * cameraPos;
tanFragPos = mTBN * worldPos.xyz;

You cannot do something like that. mTBN is a 3x3 matrix: without a 4th column and a non-zero homogeneous coordinate, you can only transform directions, not positions. You need to first get the camera direction relative to the vertex, then transform that direction.

// Beware of the vector direction.
// Here viewDir points from the object toward the viewer.
vec3 worldViewDir = normalize(cameraPos - worldPos.xyz);
tanViewDir = mTBN * worldViewDir;

Let us know in comments if that helps solving your issue.

$\endgroup$
  • $\begingroup$ I made some edit $\endgroup$ – narthex Sep 30 '16 at 14:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.