Are there any way to optimize this flow:
- The frame captured from camera (NV21 byte array)
- Convert this frame to I420 format
- Upload Y data to
y_tex
; U data tou_tex
; V data tov_tex
- Run the following fragment shader:
...
float greenValue(vec2 coord)
{
return texture2D(y_tex, coord).r
- 0.344 * (texture2D(u_tex, coord).r - 0.5)
- 0.714 * (texture2D(v_tex, coord).r - 0.5);
}
// some code .... if (current pixel is skin color)
vec2 blurCoordinates[20];
blurCoordinates[0] = interp_tc.xy + singleStepOffset * vec2(0.0, -10.0);
blurCoordinates[1] = interp_tc.xy + singleStepOffset * vec2(0.0, 10.0);
blurCoordinates[2] = interp_tc.xy + singleStepOffset * vec2(-10.0, 0.0);
blurCoordinates[3] = interp_tc.xy + singleStepOffset * vec2(10.0, 0.0);
blurCoordinates[4] = interp_tc.xy + singleStepOffset * vec2(5.0, -8.0);
blurCoordinates[5] = interp_tc.xy + singleStepOffset * vec2(5.0, 8.0);
blurCoordinates[6] = interp_tc.xy + singleStepOffset * vec2(-5.0, 8.0);
blurCoordinates[7] = interp_tc.xy + singleStepOffset * vec2(-5.0, -8.0);
blurCoordinates[8] = interp_tc.xy + singleStepOffset * vec2(8.0, -5.0);
blurCoordinates[9] = interp_tc.xy + singleStepOffset * vec2(8.0, 5.0);
blurCoordinates[10] = interp_tc.xy + singleStepOffset * vec2(-8.0, 5.0);
blurCoordinates[11] = interp_tc.xy + singleStepOffset * vec2(-8.0, -5.0);
blurCoordinates[12] = interp_tc.xy + singleStepOffset * vec2(0.0, -6.0);
blurCoordinates[13] = interp_tc.xy + singleStepOffset * vec2(0.0, 6.0);
blurCoordinates[14] = interp_tc.xy + singleStepOffset * vec2(6.0, 0.0);
blurCoordinates[15] = interp_tc.xy + singleStepOffset * vec2(-6.0, 0.0);
blurCoordinates[16] = interp_tc.xy + singleStepOffset * vec2(-4.0, -4.0);
blurCoordinates[17] = interp_tc.xy + singleStepOffset * vec2(-4.0, 4.0);
blurCoordinates[18] = interp_tc.xy + singleStepOffset * vec2(4.0, -4.0);
blurCoordinates[19] = interp_tc.xy + singleStepOffset * vec2(4.0, 4.0);
// some code ....
float sampleColor = centralColor.g * 20.0;
sampleColor += greenValue(blurCoordinates[0]);
sampleColor += greenValue(blurCoordinates[1]);
sampleColor += greenValue(blurCoordinates[2]);
sampleColor += greenValue(blurCoordinates[3]);
sampleColor += greenValue(blurCoordinates[4]);
sampleColor += greenValue(blurCoordinates[5]);
sampleColor += greenValue(blurCoordinates[6]);
sampleColor += greenValue(blurCoordinates[7]);
sampleColor += greenValue(blurCoordinates[8]);
sampleColor += greenValue(blurCoordinates[9]);
sampleColor += greenValue(blurCoordinates[10]);
sampleColor += greenValue(blurCoordinates[11]);
sampleColor += greenValue(blurCoordinates[12]) * 2.0;
sampleColor += greenValue(blurCoordinates[13]) * 2.0;
sampleColor += greenValue(blurCoordinates[14]) * 2.0;
sampleColor += greenValue(blurCoordinates[15]) * 2.0;
sampleColor += greenValue(blurCoordinates[16]) * 2.0;
sampleColor += greenValue(blurCoordinates[17]) * 2.0;
sampleColor += greenValue(blurCoordinates[18]) * 2.0;
sampleColor += greenValue(blurCoordinates[19]) * 2.0;
Problem:
I tested on weak device (galaxy A6) and capture with HD resolution frame (1280x720). It take ~100ms/frame. The main reason of this bottle neck is the large number of texture2D()
call. Each frame make ~60,000,000 call texture2D()
.
Idea:
When I replace (u_tex
, v_tex
) by y_tex
for test, it's very fast. So, I think performance maybe improved by use only one texture instead of three textures (y_tex, u_tex, v_tex
).
Question:
Does my approach is feasible? Are there any way to optimize this shader or improve entire this flow?