want bokeh blur with custom shapes

Question

I have a 2d texture and a shape matte, the matte could be any shape, like a star, heart, and so on, and I want to use this matte to generate different-shaped bokeh blur, how can I do it? and normally, creating bokeh blur needs iterations, how to cope with the efficiency problem. hopefully, realize it in glsl

Answer 1

Here is a basic implementation for a rasterized scene.

1. Bind and write the depth buffer as usual.
2. Bind and write the stencil buffer with the shapes that need to be masked out. A simple 1 or 0 will do here.
3. Generate the color buffer
4. Run the bokeh filter as a full screen quad post processing effect.

The order here isn't terribly important the only step dependent on the other 3 is doing the actual blur.

In the fragment shader:

1. read the stencil and color buffers. If the stencil value is 1 return the unmodified color.
2. read the depth buffer.
3. Run the blur kernel.

---

Here is a basic fragment shader implementation: uBlurRadius is a uniform passed in to set the blur kernel size and uResolution is the width and height of the color buffer.

float stencil = texture(stencilBuffer, uv).r;

if (stencil == 1.0) {
  // Keep the pixel in focus.
  gl_FragColor = texture(colorBuffer, uv);
  return;
}

// Compute the depth of field mask.
float focusDepth = texture( depthBuffer, uv);
float blurDepth = focusDepth + uBlurRadius;
float blurAmount = smoothstep(focusDepth, blurDepth, depth);

// Apply the bokeh blur effect.
vec4 color = vec4(0.0);
float totalWeight = 0.0;
for (float i = -uBlurRadius; i <= uBlurRadius; i++) {
  for (float j = -uBlurRadius; j <= uBlurRadius; j++) {
    vec2 offset = vec2(i, j) / uResolution;
    float weight = exp(-length(offset) * length(offset) / 2.0);
    totalWeight += weight;
    color += texture(colorBuffer, uv + offset * blurAmount) * weight;
  }
}
gl_FragColor = color / totalWeight;

The biggest issue with this code is that fragments that are in focus can bleed/smear into those that are out of focus inside the blur kernel. If you implement this and look closely at the edges around scene elements that are in focus you will be able to see the effect it has. But the code is simple and relatively fast. The blur kernel size can be used to limit blur and improve performance which can be an easy way to make this work on a variety of hardware.