WebGL - issues doing a simple texture downsample

Question

I'm working on an image processing app which uses WebGL to apply shaders to a bunch of input photos. At one point in the algorithm, I need to create some image pyramids where I downscale the input photos by a factor of two each time, until I reach the smallest possible level.

For my downscaling routine, I'm using a really simple process:

1. Create a new, blank texture using texImage2D which is half the size of the original.
2. Attach this texture to a frame buffer and use it as a render target (bindFrameBuffer). I also set the viewport to match the new texture's size.
3. Draw a flat quad whose vertices cover the screen-space coordinates (-1 -> 1) using a 'copy-paste' shader which samples the original (full-size) texture. Note that the texture parameters are always NEAREST and CLAMP_TO_EDGE.

Here's the source code for my fragment shader:

precision mediump float;

uniform sampler2D u_inputImage;  // Original full-size texture

varying vec2 v_texCoords;

void main()
{
    gl_FragColor = vec4(texture2D(u_inputImage, v_texCoords).rgb, 1);
}

The result looks sensible, but the quality of the downscaled texture is pretty poor. In the following example, you can see how it looks jagged compared to the original, particularly along the edges of the table on the bottom-right. There are also some glitchy pixels on the top of the chair, and under the door handle:

Does anyone know why this simple method doesn't seem to work? Is there a better way of creating a downsampled texture using WebGL/OpenGL?

Answer 1

The downscaled picture is not smooth because you use "nearest" filtering.

Each block of 2x2 pixels in the original image is transformed into a single pixel in the downscaled image. Put another way, each pixel in the downscaled image represents the contents of 4 original pixels. Your current solution considers just one of those 4 pixels and ignores the others. This is what produces blockiness in the downscaled picture.

To fix this, each downscaled pixel must contain information from more than one original pixel. The simplest way to do this is by using a "box filter" where each downscaled pixel is computed as the mean of the 4 original pixels it covers. This can be achieved in a single texture fetch with linear filtering and the proper sampling position:

+---+---+
| A | B |  // ABCD => original pixels
+---X---+  // X => sample position in the middle of the 4 pixels (texels)
| C | D |
+---+---+

Linear filtering sampled at position 0 will give (A+B+C+D) / 4 or the average value.

Another computer graphic term for what you're doing is "mipmap generation". There are other filters which give better quality than the simple box filter I described.

Answer 2

Thanks for the input everyone. The problem turned out to be an issue with texture coordinates. I was sampling right on the edge of each pixel in the full-size image, causing causing it to shift to the next pixel in line. To fix this, I translated the texture coordinates left/up by half a pixel and I get the intended result.