Decoding video will result in different pixel formats on different GPUs and drivers. I have an application which can render (using OpenGL), only the RGB8 pixel format, therefore I need to do color conversion from the decoded pixel format to RGB8. I can access the gl... functions in this program but I cannot edit the shader, so I'm stuck with RGB8.

This ffmpeg example demonstrates how to do hardware decoding: https://github.com/FFmpeg/FFmpeg/blob/release/4.2/doc/examples/hw_decode.c

At line 109 it does this:

/* retrieve data from GPU to CPU */
            if ((ret = av_hwframe_transfer_data(sw_frame, frame, 0)) < 0) {

I want to avoid this because it takes time. Therefore, I need a way to reuse that decoded video, which is in GPU memory, to redo color conversion.

Is it possible to create a shader program that takes input from the GPU memory, not CPU memory? If I want to pass image from CPU to my shader, I simply use glTexImage2D. Is there a similar way to do it but from GPU memory?

Can I do even deeper and render the color decoded video on screen without copying from GPU memory too?

If what I wanna do is possible, are there shader programs available to color convert pixels for some major graphics cards/drivers? I don't want to support all of them, but I'd like to support at least the Jetson Nano board from NVIDIA.

  • $\begingroup$ Can you be more specific? Are you looking to convert from something like Y'CbCr to RGB (or the opposite)? Or HSV or Lab? It's certainly possible to convert between color spaces on the GPU without going to the CPU, but it's not clear what you're asking. $\endgroup$ Jul 27, 2019 at 3:41
  • $\begingroup$ @user1118321 hi, thank you for your answer. Well, I heard that each GPU decodes to a different color format. So what I need is to convert from this unknown color format to RGB8. I know it's a hard task to do it for all color formats, but I plan to support, for now, only the format outputted from the Jetson Nano board from NVIDIA. I don't know its output because the ffmpeg support for this board is not ready yet but I guess it's NV12 (the color format for CUDA). Can you tell more about how to do these color conversions? $\endgroup$ Jul 27, 2019 at 4:03
  • $\begingroup$ This article on NV12 says, "The NV12 image format is commonly found as the native format from various machine vision, and other, video cameras." The article has CPU code for converting from NV12 to Y'CbCr and then to RGB. I don't see why you couldn't modify that code to work on the GPU. $\endgroup$ Jul 27, 2019 at 21:25
  • $\begingroup$ @user1118321 thanks, that's a starting point. But how should I do this conversion in GPU? I have no idea, and I searched about rendering from GPU memory but couldn't find anything. Could you give me a starting point? $\endgroup$ Jul 27, 2019 at 22:37
  • $\begingroup$ OK, I've added an answer that goes into more detail. $\endgroup$ Jul 27, 2019 at 23:23

1 Answer 1


So it sounds like the main issue you're having is how to render something when all resources are already on the GPU. I would do it like this:

  1. Create the destination RGB texture that's the proper width and height
  2. Create an FBO
  3. Set the texture created in step 1 as the output (write) texture of the FBO
  4. Bind the NV12 texture to the texture unit you'll use in your shader
  5. Draw a textured quad that covers the entire output
  6. Apply a shader that samples the NV12 texture to generate the RGB value at each fragment

For step 1 you use glGenTextures() and glTexImage2D().

For steps 2 and 3 you use glGenFramebuffers() and glBindFramebuffer() passing glBindFramebuffer() the texture created in step 1 to attach it to GL_COLOR_ATTACHMENT0. (See here for more details.)

For step 4 you use glBindTexture().

For step 5 you can simply draw a quad that goes from (-1, -1) to (1, 1). You don't need any model view matrix or anything.

For step 6 you'll need to calculate where the red, green, and blue samples that correspond to the current output pixel are in the input texture and then sample them using the texture() function. You'll need to do 3 different look-ups for each output pixel. This link has C code for figuring out where each sample is from.

Once you're done you can delete the frame buffer and use the texture created in step 1 in your next step (display it, process it, save it to disk, or whatever).

  • $\begingroup$ Thanks, I'm learning about frameBuffers and I kinda understood all you said. I think I can follow it after some days of study. There's just one thing that I didn't understand. After calling ret = avcodec_receive_frame(avctx, frame);, ffmpeg stores my decoded frame somewhere on GPU and has a pointer to it called frame, on CPU. How do I make this frame get into my OpenGL program? I believe this is what you meant by Bind the NV12 texture to the texture unit you'll use in your shader $\endgroup$ Jul 28, 2019 at 6:27
  • $\begingroup$ I've not used ffmpeg, so I don't know for sure, but if you want to avoid going from the CPU back to the GPU you'll need to get the texture ID from ffmpeg. Otherwise you can upload the texture from the CPU using glTexImage2D(). $\endgroup$ Jul 28, 2019 at 14:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.