1
$\begingroup$

I convert RGB input to YUV 420 semi-planar output using compute shader.Luma plane is straightforward as it maps 1:1 with RGB dimensions. As we know, NV12 packs Cb and Cr into one plane, which means the CbCr values must be calculating from downsampled (x4) source.And this is the part I am not completely sure I am doing right. I currently try subsampling with simple bileanar filtering.And it 'almost' looks right.I uses this article as reference.It is possible that my code is 100% correct and the aliasing stems from compression losness.

Here is my part of shader code that samples from RGB texture and writes U and V values:

ivec2 itexSize =textureSize(rbga_tex,0);
vec2 texSize =  vec2(itexSize);
vec2 texelSize = 1.0 / (texSize / 2.0);
vec2 pixel = uv * texSize + 0.5;
pixel = (floor(pixel) / texSize) - vec2(texelSize/2.0);

//bilinear filtering:
float sc = 4.0;
vec3 ts = texture(rbga_tex,pixel).rgb * 0.25; // center
vec3 tr = texture(rbga_tex,pixel + vec2(texelSize.x * sc, 0.0)).rgb  * 0.25;
vec3 bl = texture(rbga_tex,pixel + vec2(0.0, texelSize.y * sc)).rgb * 0.25;
vec3 br = texture(rbga_tex,pixel + vec2(texelSize.x,texelSize.y) * sc).rgb * 0.25;

vec3 tex1 = (ts + tr + bl + br) ;

// calculate and write U...

// calculate and write V...

Here is the original rgb image:

enter image description here

And here is the snapshot of the video frame after the conversion:

enter image description here

So here are my questions:

a)I would like to understand if there is something I am doing wrong, and if possible to get a concise explanation on how subsampling should be done when converting from rbg to yuv420 color space.

b)If this result is correct, are there ways to get smoother effect while keeping overall sharpness?I understand that using bicubic filter we can get smoother visuals but at cost of image sharpness.

Improve this question
$\endgroup$
6
  • 3
    $\begingroup$ As a minor optimisation, you should be able to compute the average of 4 texels using a single texture2D operation. If you sample from a half-pixel offset the sampling area will cover 4 texels equally. $\endgroup$
    – PaulHK
    Dec 19, 2017 at 4:24
  • 2
    $\begingroup$ Your results look exactly like what I'd expect, and what I see from the output of popular video editing applications by the major manufacturers. Due to the low resolution of the Cb and Cr channels, I don't think you're likely to get it any sharper. That's just how 420 subsampling is. It takes up less space because it throws out data, and it isn't careful about it at all! $\endgroup$
    – user1118321
    Dec 19, 2017 at 5:06
  • 1
    $\begingroup$ @PaulHK, do you mean the case where the texture has linear filtering performed by the hardware? $\endgroup$
    – Michael IV
    Dec 19, 2017 at 22:33
  • 1
    $\begingroup$ To answer OP's original question: I've seen implementations of this which use seperate CrCb and Y textures (called semiplanar on some platforms). By doing this you not only simplify texel coordinate computation but you also get independent filtering on both the Chroma and Luma channels. $\endgroup$
    – PaulHK
    Dec 20, 2017 at 4:10
  • 1
    $\begingroup$ I have also seen that. Done via fragments shader. This approach is suboptimal in terms of performance, due to higher bandwidth consumption. $\endgroup$
    – Michael IV
    Dec 20, 2017 at 8:02

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.