THE SHORT VERSION
In my fragment shader I'm reading (using texelFetch) multiple times the same texel from a texture (created by another fragment shader) and write it to the output render buffer. After a few read operations on the texel a fault occurs.
The texture and my rendered image have a size of (n*256) x (m*192) pixel/texel. In the first instance (texel [0,...,255]x[0,...,191]) the texture contains a human.
Here is the minimal code of the fragment shader which should copy the human:
#version 450
// output render buffer
layout(location = 5) out float depth;
in vec4 gl_FragCoord;
uniform sampler2DRect GradientsTexture;
void main(){
ivec2 pos = ivec2(mod(gl_FragCoord.x,256), mod(gl_FragCoord.y,192));
vec4 gradient = texelFetch(GradientsTexture, pos);
depth = gradient.x;
}
After execution the output looks like this:
As you can see, there is an Problem after the third instance. Additionally I noticed, that the position of the black boxe/stripes varies randomly.
Do you have any idea how this problem could be fixed?
THE LONG VERSION
Basically my problem deals with texture access in OpenGL's Fragment Shader. First I'll describe the background of the problem and what I want to accomplish. Afterwards I'll explain the point at which my problem arises. And finally I'll present a few lines of code followed by my actual question. If you're not interested in the details you can skip directly to the problem. But maybe the detailed explanation is useful for others.
The Background
Basically I want to compute the partial derivatives of a rendered depth image according to equation (15) in this paper Wei, Xiaolin, Peizhao Zhang, and Jinxiang Chai. "Accurate realtime full-body motion capture using a single depth camera." ACM Transactions on Graphics (TOG) 31.6 (2012): 188.
Derivatives
In this paper there are two important derivatives which has to be computed:
1. The Parameter Gradient
The change of depth value $D$ with respect to a parameter $q_k$ which moves the skeleton: $$ \frac{\partial D_c}{\partial \vec{q}} = \left[\frac{\partial D_c}{\partial q_1},\; ...,\; \frac{\partial D_c}{\partial q_k},\;...\right] $$ But since I'm using multiple cameras, I've different depth images of the same model. In the example here two cameras are used ($c=1,2$) and therefore we have $D_1$ and $D_2$. This gradients can be seen in the image below. I'll refer to this image as Gradient-Texture $G$. Here we see multiple instances of the same model. The first instance (0) in the upper left corner is in contrast to the other instances not a derivative but the raw depth value $D_1$. The second instance is the depth change with respect to a rotation of the whole model around the x-axis (which is parameter $q_1$)($q_7$ is the angle of the upper leg). After all $\frac{\partial D_1}{\partial q_k}$ are computed we proceed with $c=2$ and see $D_2$ and so on.
Figure 1: The raw depth images and depth gradients with respect to parameters. (referred to as Gradient-Texture $G$ )
2. The Image Gradient
The image gradient is well known and has to be computed by applying the convolution operator $\left[-1/2, 0, +1/2 \right]$ to $D_1$ and $D_2$. By doing this in x and y direction we get:
$$ \nabla D_c = \left[ \frac{\partial D_c}{\partial x}, \frac{\partial D_c}{\partial y} \right] $$
Summation
In the end I have to sum up the two derivatives:
$$ \nabla D_c \frac{\partial \vec{x}}{\partial \vec{q}} + \frac{\partial D_c}{\partial \vec{q}} \;. \;\; (1)$$
Where $\frac{\partial \vec{x}}{\partial \vec{q}}$ is the change of the image coordinates of a projected point on the model with respect to a parameter. This becomes a constant for a orthographic projection as it is applied here.
The Problem
To compute the sum in equation (1) i need to add the gradient to each $\frac{\partial D_c}{\partial q_k}$. Therefore I need to access $D_c$ in each instance, compute the derivative and add it to the instance $\frac{\partial D_c}{\partial q_k}$.
This is done in a Fragment Shader which has as input the figure 1 (as GL_TEXTURE_RECTANGLE). Since each instance $i$ does not know where to find the raw depth image ($D_c$) in the texture, an additional input with the x- and y-offset from the current pixel/texel position to the according pixel/texel position of the raw depth image is provided: Let's call it $O_x$ and $O_y$. So to get the coordinates of the raw depth image $D_c$ in the Gradient-Texture $G$, we do: $$ x_D = x + O_x(x,y)\;\, \;\;\;\;\;\;\;\; \\ y_D = y + O_y(x,y)\;, \;\;\; (2)$$ where $x$, $y$ are the current pixel coordinates (gl_fragCoord) in the Fragment Shader and $x_D$ is the x-coordinate of the raw depth image $D_c$ assigned to the Gradient-Texture at $(x,y)$.
The Offset-Texture $O_x$ is shown in the image below.
Figure 2: The Offset-Texture which transforms the coordinates from a point of each instance in the Gradient-Texture to the coordinates of the raw depth image in the Gradient-Texture.
When I do the computation stated in equation (2) I obtain the Base-Coordinates $x_D$, as I called them. They are shown in the image below.
Figure 3: The Base-Position $x_D$ of the raw depth image for each instance.
And now when I access the values of the Gradient-Texture with the computed Base-Coordinates
$$D_c = G(x_D,y_D)$$
I get a fault. The result is shown in Figure 4 below. I would expect each instance to show it's assigned raw depth $D_c$. But as you see there are black artefacts. Afterwards I would have to compute the gradient image $\nabla D_c$ but I stopped here since there is obviously a fault.
Figure 4: Raw depth $D_c$ extracted from the Gradient-Texture $G$ using the computed Base-Coordinates
The Code
To provide useful feedback I assume you'll need the Code. In the following you'll find the Fragment Shader code.
The variable GradientsTexture contains $G$ and
GradientsOffset
contains $O_x$ and $O_y$.
#version 450
// locations of the output data
layout(location = 5) out float depth2;
in vec4 gl_FragCoord;
uniform sampler2DRect GradientsTexture;
uniform sampler2DRect GradientsOffset;
void main(){
ivec2 pos = ivec2(gl_FragCoord.x-0.5, gl_FragCoord.y-0.5);
ivec2 offset = ivec2(texelFetch(GradientsOffset, pos).xy);
ivec2 basePos = pos+offset;
vec4 gradient = texelFetch(GradientsTexture, basePos);
depth2 = gradient.x;
}
The Question
And finally my question: Does anyone knows where these artefacts come from? As you saw in the figures above, the steps before give corret results. Only the access to the Gradient-Texture with
vec4 gradient = texelFetch(GradientsTexture, basePos);
causes unexpected results as shown in figure 4.
Thanks in advance and sorry for the long explanation.
Cheers, Christian Bloch.
