How does a computer upscale 1024x768 resolution to 1920x1080?

Question

Without there being black bars I mean. 1080 isn't any multiple of 768 so is there some sort of data loss?

Answer 1

In essence a image is a group of point samples (read A pixel is not a little square³). When you transform or scale the image you need to resample it. So what you do, theoretically, is take the point samples and convert them into a continuous function. Then you sample that continuous function and reconstruct a signal. So, there are two or three different phases here.

1. Converting the samples to a continuous function, (function reconstruction).
2. (Transforming)
3. Re-sampling the signal

Note that none of these steps have a fixed form. In practice, when optimized, it's impossible to tell that there are steps. The transformation does not really have to be simple it could be mapping the shape into a spiral etc.

Image 1: A 1-D signal reconstructed by different filters.

In practice, quite a bit of lore is known about signal reconstruction in the signal processing field. Designing these filters and choosing the right one is a art form on its own. But, in essence, the choice of filter is a tradeoff between blurring and ringing. Of course the algorithm also has other qualities such as how many instructions it takes to implement and how fast and how much memory it needs etc. Which can be very important in realtime or embedded applications.

Image 2: Overview of entire process.

Answer 2

There are numerous upscaling and downscaling algorithms available to scale images from any resolution to any other arbitrary resolution. Each algorithm typically involves a trade-off between efficiency, smoothness and sharpness, with varying degrees of each trade-off for different algorithms.

Check out this Wikipedia article for such algorithms and examples of such algorithms.

The most popularly known (and used) algorithm is the Bicubic interpolation algorithm. It interpolates between 2D points on a rectangular grid. Using Cubic Splines (or Cubic Interpolation), It first interpolates on one dimension (finds the interpolated row/column), then interpolates the interpolated row/column in the other dimension.

Bilinear interpolation is similar to Bicubic interpolation, except the former interpolates using a linear function and can interpolate only between two values and the latter uses a cubic function and can interpolate between four values.

The simple function for Bicubic interpolation is as follows:

f(f(p00, p01, p02, p03, y),
  f(p10, p11, p12, p13, y),
  f(p20, p21, p22, p23, y),
  f(p30, p31, p32, p33, y),
  x)

where (x, y) is the interpolated position and p[][] is the 2d array representing the 4 * 4 grid.

Check this link for more information and example code, which really helps a lot!

Answer 3

While the other answers given are correct I'm not sure they entirely answer your question. In order to not get black bars you have 2 options if the destination is not the same shape (or aspect ratio) as the source:

1. Fill the destination leaving some pixels from the source out of the destination. In your example, the source is 1024x768 and the destination is 1920x1080. If you scale the original image up so the result is 1920x1440, you can then crop the top and/or bottom off to make it 1920x1080.
2. Stretch the destination image to fit. So you'd scale vertically by 1080/768 = 1.40625, and scale horizontally by 1920/1024 = 1.875. This is almost never what you want, and looks terrible in most cases. (This is what makes people look much wider than they should, and turns circles into ovals and rotations into shears.)

There's another option, but it doesn't fit your criteria - scale uniformly, but only until 1 dimension is as big as the destination. In this case you would scale both horizontally and vertically by 1.40625 to get a result of 1440x1080. This is usually referred to as "scale to fit".

For any of these options you can use whichever scaling algorithm meets your needs.