I'm trying to render multi-channel images (more than 4 channels, for the purposes of feeding it to a Neural Network). Since OpenGL doesn't support it natively, I have multiple 4-channel render buffers, into which I render a corresponding portion of channels.

For example, I need multi-channel image of size 512 x 512 x 16, in OpenGL I have 4 render buffers of size 512 x 512 x 4. Now the problem is that the Neural Network expects the data with strides 512 x 512 x 16, i.e. 16 values of channels of one pixel are followed by 16 values of channels from the next pixel. However currently I can efficiently read my 4 render buffers via 4 calls to glReadPixels, basically making the data having strides 4 x 512 x 512 x 4. Manual reordering of data on the client side will not suffice me as it's too slow.

Main question

I've got an idea to render to a single 4-channel render buffer of size 512 x 512*4 x 4, because stride-wise it's equivalent to 512 x 512 x 16, we just treat a combination of 4 pixels in a row as a single pixel of 16-channel output image. Let's call it an "interleaved rendering"

But this requires me to magically adjust my fragment shader, so that every group of consequent 4 fragments would have exactly the same interpolation of vertex attributes. Is there any way to do that?

This bad illustration with 1 render buffer of 512 x 1024 4-channel image, is an example of how it should be rendered. With that I can in 1 call glReadPixels extract the data with stride 512 x 512 x 8 enter image description here

EDIT: better pictures What I have now (4 render buffers) enter image description here

What I want to do natively in OpenGL (this image is done in Python offline) enter image description here

  • $\begingroup$ Have you looked into 64 bit images? 4 channels of 64 bits would give you the number of channels needed, but it leaves the packing and unpacking of the data up to the shaders/code. But writing a function to pack/unpack the channels shouldn't be to difficult. $\endgroup$
    – pmw1234
    Oct 14 '21 at 12:57
  • $\begingroup$ @pmw1234 do you mean making a texture with internal format GL_RGBA32UI and pack 4 one-byte values per each component, having a total of 16 values per pixel? I like this idea. But I worry if endianness of OpenGL is universal across versions and hardware $\endgroup$ Oct 14 '21 at 13:27
  • $\begingroup$ yes, I wouldn't expect byte ordering to be an issue since the code is essentially taking over control of the packing. Also the market is dominated by little endian machines but even if it comes up specializing for machine endianness is just a matter of having the appropriate functions which would be simple to write, especially once there is a functional algorithm. $\endgroup$
    – pmw1234
    Oct 14 '21 at 13:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.