THE SHORT VERSION

In my fragment shader I'm reading (using texelFetch) multiple times the same texel from a texture (created by another fragment shader) and write it to the output render buffer. After a few read operations on the texel a fault occurs.

The texture and my rendered image have a size of (n*256) x (m*192) pixel/texel. In the first instance (texel [0,...,255]x[0,...,191]) the texture contains a human.

Here is the minimal code of the fragment shader which should copy the human:

#version 450

// output render buffer
layout(location = 5) out float depth;

in      vec4    gl_FragCoord;

uniform sampler2DRect GradientsTexture;

void main(){

ivec2 pos       = ivec2(mod(gl_FragCoord.x,256),  mod(gl_FragCoord.y,192));
vec4  gradient  = texelFetch(GradientsTexture, pos);
depth           = gradient.x;
}


After execution the output looks like this:

As you can see, there is an Problem after the third instance. Additionally I noticed, that the position of the black boxe/stripes varies randomly.

Do you have any idea how this problem could be fixed?

THE LONG VERSION

Basically my problem deals with texture access in OpenGL's Fragment Shader. First I'll describe the background of the problem and what I want to accomplish. Afterwards I'll explain the point at which my problem arises. And finally I'll present a few lines of code followed by my actual question. If you're not interested in the details you can skip directly to the problem. But maybe the detailed explanation is useful for others.

The Background

Basically I want to compute the partial derivatives of a rendered depth image according to equation (15) in this paper Wei, Xiaolin, Peizhao Zhang, and Jinxiang Chai. "Accurate realtime full-body motion capture using a single depth camera." ACM Transactions on Graphics (TOG) 31.6 (2012): 188.

Derivatives

In this paper there are two important derivatives which has to be computed:

1. The Parameter Gradient

The change of depth value $D$ with respect to a parameter $q_k$ which moves the skeleton: $$\frac{\partial D_c}{\partial \vec{q}} = \left[\frac{\partial D_c}{\partial q_1},\; ...,\; \frac{\partial D_c}{\partial q_k},\;...\right]$$ But since I'm using multiple cameras, I've different depth images of the same model. In the example here two cameras are used ($c=1,2$) and therefore we have $D_1$ and $D_2$. This gradients can be seen in the image below. I'll refer to this image as Gradient-Texture $G$. Here we see multiple instances of the same model. The first instance (0) in the upper left corner is in contrast to the other instances not a derivative but the raw depth value $D_1$. The second instance is the depth change with respect to a rotation of the whole model around the x-axis (which is parameter $q_1$)($q_7$ is the angle of the upper leg). After all $\frac{\partial D_1}{\partial q_k}$ are computed we proceed with $c=2$ and see $D_2$ and so on.

Figure 1: The raw depth images and depth gradients with respect to parameters. (referred to as Gradient-Texture $G$ )

2. The Image Gradient

The image gradient is well known and has to be computed by applying the convolution operator $\left[-1/2, 0, +1/2 \right]$ to $D_1$ and $D_2$. By doing this in x and y direction we get:

$$\nabla D_c = \left[ \frac{\partial D_c}{\partial x}, \frac{\partial D_c}{\partial y} \right]$$

Summation

In the end I have to sum up the two derivatives:

$$\nabla D_c \frac{\partial \vec{x}}{\partial \vec{q}} + \frac{\partial D_c}{\partial \vec{q}} \;. \;\; (1)$$

Where $\frac{\partial \vec{x}}{\partial \vec{q}}$ is the change of the image coordinates of a projected point on the model with respect to a parameter. This becomes a constant for a orthographic projection as it is applied here.

The Problem

To compute the sum in equation (1) i need to add the gradient to each $\frac{\partial D_c}{\partial q_k}$. Therefore I need to access $D_c$ in each instance, compute the derivative and add it to the instance $\frac{\partial D_c}{\partial q_k}$.

This is done in a Fragment Shader which has as input the figure 1 (as GL_TEXTURE_RECTANGLE). Since each instance $i$ does not know where to find the raw depth image ($D_c$) in the texture, an additional input with the x- and y-offset from the current pixel/texel position to the according pixel/texel position of the raw depth image is provided: Let's call it $O_x$ and $O_y$. So to get the coordinates of the raw depth image $D_c$ in the Gradient-Texture $G$, we do: $$x_D = x + O_x(x,y)\;\, \;\;\;\;\;\;\;\; \\ y_D = y + O_y(x,y)\;, \;\;\; (2)$$ where $x$, $y$ are the current pixel coordinates (gl_fragCoord) in the Fragment Shader and $x_D$ is the x-coordinate of the raw depth image $D_c$ assigned to the Gradient-Texture at $(x,y)$.

The Offset-Texture $O_x$ is shown in the image below.

Figure 2: The Offset-Texture which transforms the coordinates from a point of each instance in the Gradient-Texture to the coordinates of the raw depth image in the Gradient-Texture.

When I do the computation stated in equation (2) I obtain the Base-Coordinates $x_D$, as I called them. They are shown in the image below.

Figure 3: The Base-Position $x_D$ of the raw depth image for each instance.

And now when I access the values of the Gradient-Texture with the computed Base-Coordinates

$$D_c = G(x_D,y_D)$$

I get a fault. The result is shown in Figure 4 below. I would expect each instance to show it's assigned raw depth $D_c$. But as you see there are black artefacts. Afterwards I would have to compute the gradient image $\nabla D_c$ but I stopped here since there is obviously a fault.

Figure 4: Raw depth $D_c$ extracted from the Gradient-Texture $G$ using the computed Base-Coordinates

The Code

To provide useful feedback I assume you'll need the Code. In the following you'll find the Fragment Shader code.

The variable GradientsTexture contains $G$ and

GradientsOffset


contains $O_x$ and $O_y$.

#version 450

// locations of the output data
layout(location = 5) out float  depth2;

in      vec4    gl_FragCoord;

uniform sampler2DRect GradientsTexture;
uniform sampler2DRect GradientsOffset;

void main(){

ivec2 pos       = ivec2(gl_FragCoord.x-0.5,  gl_FragCoord.y-0.5);
ivec2 offset    = ivec2(texelFetch(GradientsOffset, pos).xy);
ivec2 basePos   = pos+offset;

vec4  gradient  = texelFetch(GradientsTexture, basePos);

depth2 = gradient.x;

}


The Question

And finally my question: Does anyone knows where these artefacts come from? As you saw in the figures above, the steps before give corret results. Only the access to the Gradient-Texture with

vec4  gradient  = texelFetch(GradientsTexture, basePos);


causes unexpected results as shown in figure 4.

Thanks in advance and sorry for the long explanation.

Cheers, Christian Bloch.

• Is your CPU code calling into GL on multiple threads? My first guess would be that there's nothing wrong with the shader and it's a cache invalidation issue caused by some non-thread-safe use of your intermediate textures. – Dan Hulme Jan 6 '17 at 14:54
• Thanks, and no, the CPU code is executed without multithreading. By the way: Now I tried the following: vec2 basePos = vec2(mod(pos.x,256),mod(pos.y,192)); where one instance is 256x192 and i still get this fault. It seems that there is a limitation on the number of reads you can perform on a texel? – Christian_B Jan 6 '17 at 15:27
• Additionally i found out, that the position of the black boxes varies randomly, when using the mod(...) version – Christian_B Jan 6 '17 at 15:34
• In your short version, the shaders seem fine. Garbage in a depth or stencil buffer can easily do this kind of thing (make sure depth testing is disabled). On the GL bug side, have you tried forcing a cpu renderer or a different vendor's GPU? – Daniel M Gessel Jan 9 '17 at 15:56
• Just one question your gradient texture is rectangular texture . Just wanted to know in your application code you have not accidently enabled texture filtering modes . Only min filter GL_TEXTURE_MIN_FILTER should be set to GL_NEAREST or GL_LINEAR . Also wrapping modes for each coordinate must be either GL_CLAMP_TO_EDGE or GL_CLAMP_TO_BORDER. Also see mipmap is not turned on . One more thing in your shader make sure texelFetch pos is not normalised. – PixelClear Apr 19 '17 at 19:19