The blog post that you talked about, is not about generating bokeh for a computer generated image. It is instead about generating a believable depth of field effect from an image captured by a smartphone camera, as the effect is desired for portraits to make the subject stand out. It generally works by splitting the image in to two parts. One part is the subject, which should not be blurred. The other part is the background, which should be blurred but the pixels of the subject should have no influence and would thus have a weight of zero. Slightly more sophisticated algorithms may make the blur kernel size variable across an image to create a smoother transition, but the subject's pixels would still be ignored.
In games and CGI, post process depth of field is done differently. There are multiple ways for it. A simple google search shows multiple. You could do depth of field like you have, and simply ignore the pixels which should not be blurred to remove the ghosting. However, this is not true to how depth of field occurs inside of a camera. So, in order to get a good algorithm to simulate depth of field, I will first try to explain how you get the depth of field effect in an actual camera.
Let us say, that we have an object, a single lens and an image sensor. We look at them from the side.
Light scatters in many different directions when hitting the object. To simplify we only take one point. From this point, I will show the light that would hit the lens as a gray solid.
The lens bends the light towards a single point. That is the focal point.
You can see that the focal point is in front of the sensor. This causes the light from that single point on the object, to fall on a large portion of the sensor. If the focal point is behind the sensor, the same would happen. But, if the focal point lies on the sensor, there would only be one point on the sensor where the light falls on. This area where the light falls on the sensor, is called the circle of confusion (CoC). This is also the cause of the depth of field effect.
One important thing to know, is that when the point on the object moves around, the focal point also moves around. If the point moves closer to the lens, the focal point moves closer to the lens, which would alter the size of the circle of confusion. This gives you the gradual change in blurriness that you have with depth of field, since the circle of confusion gradually becomes larger when an object moves away from the area in focus.
If we add in an aperture we simply block out a part of the light, which causes the circle of confusion to be smaller. This allows us to determine how much light to let in (to make the image brighter or darker), and how strong the depth of field effect is.
From this, we can conclude a few things on how we should implement depth of field.
- Every point whose light enters the camera lens, has a circle of confusion.
- The circle of confusion is the area where the light, from that point, falls on.
- The circle of confusion changes size when the point moves closer to or further away from the lens, or when the aperture changes size.
- The area of the circle of confusion has no falloff. Every point inside the circle of confusion has the same weight. This results in a box blur.
- If a point is in focus, its circle of confusion would be either zero or something so small, that it is not noticed.
The algorithm is quite simple. For each pixel in the image, we use the depth to calculate the circle of confusion. Then, we add the colour of that pixel to all the other pixels that lie in the circle of confusion. However, if that other pixel is closer to the camera (a lower depth value), it would block the light from our pixel, and we should not add the colour of our pixel to the blocking pixel.
In pseudo code it would look something like this;
// for each pixel in the all-focus image
for(x = 0; x < width; x++){
for(y = 0; y < height; y++){
// How much blur? Zero or close to it if it should be in focus.
float CoCRadius = calcCoCRadius(depth(x, y));
// For every pixel in the square with the size of CoCRadius.
for(i = x - CoCRadius; i <= x + CoCRadius; i++){
for(j = y - CoCRadius; j <= y + CoCRadius; j++){
// Discard pixels outside of the shape. Square -> Circle
if(isInBokehShape(i - x, j - y)){
// Discard pixels that would block it.
if(depth(i, j) >= depth(x, y){
// Add the colour.
newColour[i, j] += colour(x, y);
// Add the weight, for normalizing.
weight[i, j] += 1.0f;
}
}
}
}
}
}
//Normalize the values with the weight.
for(x = 0; x < width; x++){
for(y = 0; y < height; y++){
newColour[x, y] /= weight[x, y];
}
}
This would give you a depth of field effect that is quite true, and does not give you the ghosting effect. $calcCoCRadius$ is a method that calculates the radius in pixels of the circle of confusion. It could be anything, but if you want it as true to an actual camera, then you can use the formula from this Wikipedia page. $isInBokehShape$ is a method which returns a boolean on whether the point is inside the bokeh shape. With the two for loops we just get a square, but if we want to get a circle shaped bokeh, we need to discard the pixels that would lie outside of the circle. This discarding is also the reason why we keep track of the weights and normalize afterwards with the weight, since we do not know how many pixels we need to discard. The weight is also useful if you do want a falloff in the circle of confusion, for example if you want a gaussian blur.
Another important thing to know, is that the implementation listed here, is not very optimal. The problem is basically that you write to multiple different pixels. This causes race conditions. Instead, it is better to rework the algorithm so that you only need one write. Basically, you loop through every pixel and you then you check all other pixels (or pixels inside of a maximum CoC radius) whether their CoC radius is large enough that they contribute to the pixel's colour. This way, you can also put the normalizing of the values inside the first loop over all the pixels instead of having to do that later.
This should be a fairy good way of implementing depth of field as a post processing effect. There are also other ways, but this is one.
