1
$\begingroup$

Could someone please provide me with/point me to a description of the process involved in displaying a digital image? Starting from reading the data from a file to displaying the image on the screen.

I am puzzled regarding which parts of this process are handled by the OS and which are handled by the GPU.

I am especially interested in the transition from the initial, device-independent representation of the image to controlling the actual physical pixels on the screen. For example, how does a 1920x1080 image eventually get displayed on screens with higher or lower display resolutions?

$\endgroup$
  • 1
    $\begingroup$ This is a fairly broad question. The problem is that the answer varies wildly depending on which hardware, OS, and application is doing the displaying, not to mention which format the image is in, etc. You should think about narrowing down the question because as it is it may attract down votes or close votes. $\endgroup$ – user1118321 Jun 26 '19 at 4:23
  • $\begingroup$ I realise it's broad and you're absolutely right. The reason (and I should probably have included this in the question) is that I am an educator and it's exactly this broad overview I want to convey to learners. I need the details to improve my own understanding and make sure I am accurate and I don't introduce misconceptions, but it's the general idea that interests me. $\endgroup$ – odysseas Jun 27 '19 at 7:27
2
$\begingroup$

Starting from reading the data from a file to displaying the image on the screen.

As was pointed out in the comments, this is an extremely broad question, and depends on what operating system you're using, and which tool sets for that OS you're using. That said, I think it is possible to talk generally about the process with much of it (if not all) being similar from machine to machine, and OS to OS.

I am puzzled regarding which parts of this process are handled by the OS and which are handled by the GPU.

I'll try to point out which parts are handled strictly by the OS, your program, and the graphics hardware, and I'll even provide C-like pseudo code. My pretend code will not be efficient, simply illustrative, and I'll be pretending to read this example image file.

For example, how does a 1920x1080 image eventually get displayed on screens with higher or lower display resolutions?

I think some of my pretend pseudo code will illustrate that, but the short answer is that you will manually resize your image to better fit the screen (or any other drawing area) inside your program.

The Long Answer

The first step is to open the image file. It doesn't matter what the format of the image is, from the view of your program it'll simply be a list of bytes. Some languages have tools and libraries that can be used to open files in a generic (OS independent) way, but often this step is OS specific. However, even though the task of opening and/or reading the file is dependent on the operating system, once your program has copied the content into memory it's just bytes for your program to deal with however you desire.

File imgFile = OS_OPEN_FILE("sample.ppm"); // Ask the OS for a file.
byte buffer[imgFile.size()];               // Set aside memory to hold the file's content.
for(int i = 0; i < imgFile.size(); ++i) {  // Start a reading loop.
    buffer[i] = OS_READ_BYTE(imgFile);     // Ask the OS for all bytes in the file, one by one.
}

Once you have access to all of the bytes in the entire image file it is up to you, the programmer, to know what each byte means, then to make your program read and interpret them correctly. For example, the portable bitmap format starts with a couple of bytes letting you know if the picture file is color, gray scale, or black and white. There are bytes that let you know what size the picture is, bytes that should be ignored because they're comments intended for humans to read, and finally bytes that are the dots of color to be sent to the screen. It is up to the program to sort through all of that.

int bufSize = imgFile.size();
for(int i = 0; i < bufSize; ++i) {  // loop through all the bytes read earlier.
    if(buffer[i] == 'P' && buffer[i+1] == '3') {  // Handle the image header.
        /* Do something */
    }
    else if(buffer[i] == '#') {  // Handle comments
        /* Do something */
    }
    else if(buffer[i] == /*...*/) {
        /* Do something */
    }
   /* ... etc. */
}

The next task is OS specific. You'll need to somehow ask the operating system to give your program access to a "drawing surface". Again, there are libraries such as SDL or SFML that provide surfaces in a generic way, but they are hiding the surface the system gave them behind an easy to use wrapper.

Often the drawing surface given by the OS is just a big block of bytes and you can simply copy data into it. Again, it is up to you to know which byte does what (like, which bytes represent red, green, or blue), but the task itself is usually just to copy from one chunk of memory into another. This is known as a Block Image Transfer, or a BLIT, and it's at this point that you may be handling memory inside the graphics card. Ultimately, it's up to the OS to decide where the drawing surface keeps the bytes that represent colors, but you can usually request that those bytes be on the GPU.

Surface sampleImage = OS_GET_GPU_DRAWING_SURFACE();
if(sampleImage.good()) {
    /* Copy in bytes from buffer[], sorting or converting data as needed. */
}

Depending on what tools you're using, and what task you're performing, a drawing surface you request from the OS can be the screen itself, and other times the screen is something that you can't touch directly but can only send surfaces to. Even when you can draw directly to the screen you will want to draw to a hidden surface, then blit the finished work over to the screen to prevent flickering and other visual problems. This is called double buffering, and mostly applies to the screen because it's always visible to the user.

Generally you will have many small drawing surfaces that represent things like, buttons, pieces of terrain in a game, letters, or whatever, and you'll use those to assemble a larger image on a bigger (usually screen sized) drawing surface that you will finally send to the screen as one big block.

Just to reiterate, these steps are all being done by your program, but the way your program has to do them is OS specific, and much of the memory being handled at this late stage is probably on the GPU.

// Ready the screen and a drawing area.
Surface * screen = OS_GET_SCREEN_SURFACE();
Surface * screenBuffer = OS_GET_GPU_DRAWING_SURFACE();

// Make the image 800% bigger!
sampleImage.resize(8.0f);
// Copy the image to the screen sized buffer, centering it.
// blitTo(surface, x, y)
// I'm assuming that in this pretend code images draw from their top left corner.
sampleImage.blitTo(screenBuffer,
                   screenBuffer.width()/2 - sampleImage.width()/2,
                   screenBuffer.height()/2 - sampleImage.height()/2);
/* ... etc. */
/* Do more image drawing work. */

// Finally, display all our complex behind the scenes work to the user.
// This is called a buffer swap.
Surface * temp = NULL;
temp = screen;
screen = screenBuffer;
screenBuffer = temp;
temp = NULL;

The Short Answer

In summary...

Task your program needs to do (what's primarily in charge):

  1. Open a file (OS)
  2. Read file (OS & Program)
  3. Interpret data (Program)
  4. Get a drawing surface (OS & GPU)
  5. Fill the drawing surface with image data. (Program & GPU)
  6. Assemble your final picture. (Program & GPU)
  7. Gain access to the screen (OS)
  8. Send your final picture to the screen. (OS and/or Program)
| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Thank you so much for your answer, it was extremely useful. $\endgroup$ – odysseas Jun 27 '19 at 8:11
  • $\begingroup$ You're very welcome. :-) $\endgroup$ – RoboticForest Jun 27 '19 at 16:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.