I'm trying to understand what causes the blobby edges when using alpha testing to create transparency with 1-bit alpha channels. The edges appear pixelated, but with rounded corners and diagonal lines. blobby edge effect on plant leaf [Image from http://wiki.polycount.com/wiki/File:Alphatest_8bit_vs_1bit.jpg]

My theory is that the fragment shader is sampling from the alpha channel using bicubic interpolation then comparing with the alpha threshold. (I would expect straighter edges with bilinear interpolation). Is this correct? Is something special done to prevent overshoot artifacts?

  • 3
    $\begingroup$ I think it's bilinear interpolation. You should check what interpolation you have chosen, the standard choices are nearest, linear, anisotropic. Try switching to nearest: khronos.org/opengl/wiki/Sampler_Object#Anisotropic_filtering $\endgroup$
    – lightxbulb
    Mar 11, 2022 at 10:18
  • $\begingroup$ This isn't for a project of mine, but rather to try to explain a common artifact in video games to my students. I first noticed it in Hitman 3, and switching from anisotropic to trilinear filtering seemed to keep it around. I'll see if I can replicate the issue myself to better understand it. $\endgroup$
    – TomKern
    Mar 12, 2022 at 16:53
  • $\begingroup$ I don't think this is an artifact, I believe it is intended, since most would consider the nearest neighbour version a lot more jarring. Trilinear is just linear interpolation between bilinearly interpolated samples on two mip levels, so it definitely will be present there too. As for reproducing it try to resample the above black and white image using bilinear interpolation. $\endgroup$
    – lightxbulb
    Mar 12, 2022 at 18:16

1 Answer 1


Let's consider the simplest case, where our texture is 2x2, with alpha values $a_{0,0}, a_{1,0}, a_{0,1}, a_{1,1}$. With bilinear filtering, the alpha at $uv$ coordinate $(x,y)$ is


which works out to:

lerp($x a_{1,0} + (1-x) a_{0,0}$, $x a_{1,1} + (1-x) a_{0,1}$,$y$)

= $y(x a_{1,1} + (1-x) a_{0,1}) + (1-y) (x a_{1,0} + (1-x) a_{0,0})$,

Expanding this out, it's a polynomial of the form:


Solving when this is equal to some threshold generally produces hyperbolas, but in degenerate cases can produce lines.

In the image below, the texture specifies the colors at each grid line intersection. Within each grid square we have an example of interpolating a 2x2 texture, and the edge produced by comparing with a threshold is a segment of line or hyperbola.

enter image description here

I had originally assumed the lerp of the lerps would produce a polynomial of the form $b+cx+dy$, which solving against a threshold can only produce lines, so it was surprising to see nonlinear curves in the result.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.