# YUV (NV12) Chroma subsampling while converting from RGB image

I convert RGB input to YUV 420 semi-planar output using compute shader.Luma plane is straightforward as it maps 1:1 with RGB dimensions. As we know, NV12 packs Cb and Cr into one plane, which means the CbCr values must be calculating from downsampled (x4) source.And this is the part I am not completely sure I am doing right. I currently try subsampling with simple bileanar filtering.And it 'almost' looks right.I uses this article as reference.It is possible that my code is 100% correct and the aliasing stems from compression losness.

Here is my part of shader code that samples from RGB texture and writes U and V values:

ivec2 itexSize =textureSize(rbga_tex,0);
vec2 texSize =  vec2(itexSize);
vec2 texelSize = 1.0 / (texSize / 2.0);
vec2 pixel = uv * texSize + 0.5;
pixel = (floor(pixel) / texSize) - vec2(texelSize/2.0);

//bilinear filtering:
float sc = 4.0;
vec3 ts = texture(rbga_tex,pixel).rgb * 0.25; // center
vec3 tr = texture(rbga_tex,pixel + vec2(texelSize.x * sc, 0.0)).rgb  * 0.25;
vec3 bl = texture(rbga_tex,pixel + vec2(0.0, texelSize.y * sc)).rgb * 0.25;
vec3 br = texture(rbga_tex,pixel + vec2(texelSize.x,texelSize.y) * sc).rgb * 0.25;

vec3 tex1 = (ts + tr + bl + br) ;

// calculate and write U...

// calculate and write V...


Here is the original rgb image:

And here is the snapshot of the video frame after the conversion:

So here are my questions:

a)I would like to understand if there is something I am doing wrong, and if possible to get a concise explanation on how subsampling should be done when converting from rbg to yuv420 color space.

b)If this result is correct, are there ways to get smoother effect while keeping overall sharpness?I understand that using bicubic filter we can get smoother visuals but at cost of image sharpness.

• As a minor optimisation, you should be able to compute the average of 4 texels using a single texture2D operation. If you sample from a half-pixel offset the sampling area will cover 4 texels equally. – PaulHK Dec 19 '17 at 4:24
• Your results look exactly like what I'd expect, and what I see from the output of popular video editing applications by the major manufacturers. Due to the low resolution of the Cb and Cr channels, I don't think you're likely to get it any sharper. That's just how 420 subsampling is. It takes up less space because it throws out data, and it isn't careful about it at all! – user1118321 Dec 19 '17 at 5:06
• @PaulHK, do you mean the case where the texture has linear filtering performed by the hardware? – Michael IV Dec 19 '17 at 22:33
• To answer OP's original question: I've seen implementations of this which use seperate CrCb and Y textures (called semiplanar on some platforms). By doing this you not only simplify texel coordinate computation but you also get independent filtering on both the Chroma and Luma channels. – PaulHK Dec 20 '17 at 4:10
• I have also seen that. Done via fragments shader. This approach is suboptimal in terms of performance, due to higher bandwidth consumption. – Michael IV Dec 20 '17 at 8:02