Upscaling screenshots with subpixel rendered text

Question

Upscaling a screenshot with subpixel antialiased text produces unpleasant color fringes, e.g. why does black text have orange and blue pixels This is especially annoying when trying to show an application on a projector, when you have to zoom in with a tool like the Windows Magnifier to show details to the audience.

There are some solutions available to exploit the subpixel geometry of a display when downscaling an image, e.g. https://computergraphics.stackexchange.com/a/1431/12022. Are there any solutions for the opposite, i.e. taking an image with subpixel rendering and producing an upscaled image without color artifacts? I imagine this task is much more difficult in the presence of both subpixel (text) and non-subpixel (the rest of the screenshot) content in the same image, though.

Answer 1

By simulating what the subpixels look like, and optionally converting that to greyscale, we can obtain the intended (even better: color-artifact-free) effect of the original subpixel rendering.

Someone please add this to a screen magnifier software to help read small text and inspect graphic renders.

Results for going from the top left to the bottom right image look nice:

→

      ↓                 ↑

→

The algorithm is:

• Create an image that simulates what the screen looks like at 3x magnification, i.e. render each subpixel as three dedicated pixels of only that primary hue (red, green, or blue).
• Optionally, convert to greyscale (and brighten 3x because the simulated image can't be bright without HDR).
• Display the result at a high zoom level, otherwise the subpixels of the displaying device would interfere a tiny bit. Compensating for that as well is a different story.

Python code:

import numpy as np
import matplotlib.pyplot as plt

def subpixel_render_and_save_all(image_path):
    # Load the image as an array and normalize to RGB values [0, 255]
    img = plt.imread(image_path)
    
    # Ignore alpha channel for now. TODO handle alpha channel
    if img.shape[2] == 4:
        img = img[:, :, :3]
    
    img = (img * 255).astype(np.uint8)
    
    height, width, _ = img.shape

    # Create a base for the subpixel-rendered image (3x larger dimensions)
    img_subpixels = np.zeros((height * 3, width * 3, 3), dtype=np.uint8)

    # Place each color channel in its respective vertical subpixel column
    img_subpixels[:, 0::3, 0] = np.repeat(img[:, :, 0], 3, axis=0)  # Red
    img_subpixels[:, 1::3, 1] = np.repeat(img[:, :, 1], 3, axis=0)  # Green
    img_subpixels[:, 2::3, 2] = np.repeat(img[:, :, 2], 3, axis=0)  # Blue
    
    # Convert to grayscale using luminance method
    # img_subpixels_grayscale = (0.2989 * img_subpixels[:, :, 0] +
                                # 0.5870 * img_subpixels[:, :, 1] +
                                # 0.1140 * img_subpixels[:, :, 2]).astype(np.uint8)

    # Convert to grayscale
    img_subpixels_grayscale = (img_subpixels[:, :, 0] +
                               img_subpixels[:, :, 1] +
                               img_subpixels[:, :, 2]).astype(np.uint8)

    # Downscale by averaging each 3x3 block to a single pixel
    img_subpixels_grayscale_downscaled = img_subpixels_grayscale.reshape(height, 3, width, 3).mean(axis=(1, 3)).astype(np.uint8)

    base_filename = image_path.rsplit('.', 1)[0]
    plt.imsave(f"{base_filename}_subpixels.png", img_subpixels)
    plt.imsave(f"{base_filename}_subpixels_grayscale.png", img_subpixels_grayscale, cmap="gray")
    plt.imsave(f"{base_filename}_subpixels_grayscale_downscaled.png", img_subpixels_grayscale_downscaled, cmap="gray")

# Example usage
subpixel_render_and_save_all("input_image.png")

This concrete method is similar to Simon F's mysterious comment.