# What kind of anti-aliasing is used in font rasterization?

I'm writing my own TrueType font rasterizer as an exercise to improve my knowledge in computer graphics (along with a basic vector graphics engine). At this stage, I have implemented most of the TTF file parsing, decoding glyph outlines, scaling, and scan converting the outlines into a bitmap. (I'm not doing any font hinting/grid fitting for the time being).

When it comes to anti-aliasing the glyph bitmaps, what process or method is typically used (or best)? So far I have tried two methods. Bilinear filtering and setting the alpha value at the edges according to the pixel coverage of the outline.

Bilinear sampling is too blurry to be effective in font rendering, and using pixel coverage is not continuous enough. Here are some images as reference of a lowercase Arial 'c' being rendered at 24 points:

Raw bitmap with no anti-aliasing:

Bilinear filtering:

Here is the same character rendered at the same size with a library (stb):

And finally, here it is rendered as it would appear on OS X (using CoreText I presume):

I'd be happy with whatever method is used by the stb library for my purposes.

• The final image appears to be using subpixel rendering (note the coloured fringes to the left and right are different colours). Do you want to use this approach (which requires knowing the pixel geometry of the particular screen you are rendering to) or do you just want the best approach for pixel rendering, ignoring subpixels (which can be displayed on any screen)? – trichoplax Jul 30 '16 at 21:48
• @trichoplax I think for right now I'd be happy with the best approach for pixel rendering, ignoring subpixels (which sounds more complicated). One step at a time. :) – Chris Jul 30 '16 at 22:11
• That sounds like a good approach - I just wanted to have it stated explicitly so answerers don't get distracted into explaining the last image. – trichoplax Jul 30 '16 at 22:30
• Can you explain what you mean by bilinear filtering in this context? I don't understand how that concept applies to rasterizing the glyph shapes. Also, when you tried "setting the alpha value at the edges according to the pixel coverage of the outline", how did you accomplish that? Supersampled rasterization? – Nathan Reed Jul 30 '16 at 22:39
• @NathanReed Sure. For bilinear filtering, I sweep over the raw bitmap and for each pixel, look at the three pixels surrounding it (the one to the right, the one below, and the one diagonally down and to the right) and average them together to come up with the final intensity value. Pixels off the edge of the bitmap are treated as 0. – Chris Jul 31 '16 at 0:07

Sean Barrett, the author of the stb libraries, wrote an article on how stb_truetype's rasterization works. It's quite readable.

The article describes two methods of antialiasing, both based on scanline rasterization. In scanline rasterization, you iterate through the lines (rows) of pixels and keep track of which edges from the glyph outline are intersecting the scanline (considered as a mathematical zero-thickness line); then you can identify which pixel-centers are inside the outline by tracking enter/exit intersection points along that line.

You can do horizontal antialiasing by calculating partial pixel coverage, using the position of the intersection points relative to the pixel grid. For example, if you have an entering point at $x = 4.6$ on the current scanline, then pixel index 4 would be set to 60% empty and 40% covered. This sounds somewhat like what you tried to do. However, this doesn't help with vertical antialiasing since the scanline is still considered as a mathematical line with zero thickness. So, the article recommends to just supersample along the vertical axis: that is, process multiple "scanlines" spaced out vertically within each pixel row, and average the results together.

The second method performs both horizontal and vertical antialiasing at once, by evaluating the area of intersection of the glyph outline with each pixel square. It's still based on scanline rasterization, but now treats the scanline as a 1-pixel-tall rectangle and tracks all edges that touch the rectangle. To evaluate pixel coverage, the trapezoidal rule is used to calculate a signed area for each glyph edge—positive or negative depending on the edge's orientation—and these are summed up to give the total coverage area within the pixel square. (There are a lot more implementation details covered in the article, plus you can read the source.) This is somewhat faster and more accurate than the supersampling method, though more difficult to implement.

Finally, it's important to consider whether you want gamma correction when rendering text (as with rendering anything else). Theoretically, the antialiased coverage values shouldn't be used directly as pixel values; they should be treated as an alpha, and used for linear-color-space blending between the background color and text color, followed by conversion to output gamma. However, in practice almost no one seems to do this; we're accustomed to seeing non-gamma-correct text, and screen fonts are designed to look good rendered this way. (Print fonts are another story.) On the other hand, if the text is animated, or rotated or not aligned with the pixel grid for any reason, gamma correction will help it look more consistent regardless of its placement relative to the pixel grid.

So, the choice of whether to apply gamma correction comes down to personal preference and the needs of the application. Either way, you may see artifacts such as the font appearing to get heavier or lighter depending on color and font size.

• This is great, thank you! I have looked at the source of stb_truetype, but my knowledge in computer graphics isn't thorough enough to really know what was happening on a theoretical level. I was not aware of the article he posted on it though, so that's a great help and will help clarify the process he uses, I'm sure. I'm glad to hear that maybe I was on the right track with the pixel coverage method, but I just wasn't taking into account the vertical part. :) – Chris Aug 1 '16 at 3:38
• I've just asked Algorithms to “anti-alias” (or somehow improve) binary 1-bit drawings and fonts, any thoughts? – uhoh Nov 30 '17 at 6:21

This is how anti-aliasing works:

1. render the outline in bilevel rendering at four times the size
2. take the averages of each 4×4 block
3. blend them with correct gamma between the foreground and background
• Welcome to CGSE. The question wasn't about how anti-aliasing works but which techniques are used/most suitable for font rendering. – wychmaster Jun 23 '20 at 20:27