Is it possible to do color conversion on decoded video without getting it from GPUmemory to CPU memory?

Question

Decoding video will result in different pixel formats on different GPUs and drivers. I have an application which can render (using OpenGL), only the RGB8 pixel format, therefore I need to do color conversion from the decoded pixel format to RGB8. I can access the gl... functions in this program but I cannot edit the shader, so I'm stuck with RGB8.

This ffmpeg example demonstrates how to do hardware decoding: https://github.com/FFmpeg/FFmpeg/blob/release/4.2/doc/examples/hw_decode.c

At line 109 it does this:

/* retrieve data from GPU to CPU */
            if ((ret = av_hwframe_transfer_data(sw_frame, frame, 0)) < 0) {

I want to avoid this because it takes time. Therefore, I need a way to reuse that decoded video, which is in GPU memory, to redo color conversion.

Is it possible to create a shader program that takes input from the GPU memory, not CPU memory? If I want to pass image from CPU to my shader, I simply use glTexImage2D. Is there a similar way to do it but from GPU memory?

Can I do even deeper and render the color decoded video on screen without copying from GPU memory too?

If what I wanna do is possible, are there shader programs available to color convert pixels for some major graphics cards/drivers? I don't want to support all of them, but I'd like to support at least the Jetson Nano board from NVIDIA.

Answer 1

So it sounds like the main issue you're having is how to render something when all resources are already on the GPU. I would do it like this:

1. Create the destination RGB texture that's the proper width and height
2. Create an FBO
3. Set the texture created in step 1 as the output (write) texture of the FBO
4. Bind the NV12 texture to the texture unit you'll use in your shader
5. Draw a textured quad that covers the entire output
6. Apply a shader that samples the NV12 texture to generate the RGB value at each fragment

For step 1 you use glGenTextures() and glTexImage2D().

For steps 2 and 3 you use glGenFramebuffers() and glBindFramebuffer() passing glBindFramebuffer() the texture created in step 1 to attach it to GL_COLOR_ATTACHMENT0. (See here for more details.)

For step 4 you use glBindTexture().

For step 5 you can simply draw a quad that goes from (-1, -1) to (1, 1). You don't need any model view matrix or anything.

For step 6 you'll need to calculate where the red, green, and blue samples that correspond to the current output pixel are in the input texture and then sample them using the texture() function. You'll need to do 3 different look-ups for each output pixel. This link has C code for figuring out where each sample is from.

Once you're done you can delete the frame buffer and use the texture created in step 1 in your next step (display it, process it, save it to disk, or whatever).