Ambient occlusion depth buffer normalization

Been trying to follow a shader from A Comparative Study of Screen-Space Ambient Occlusion Methods

(shader in appendix) specifically the Alchemy AO implementation. I am getting some odd results, can't seem to figure out what might be causing it. I feel it could do with the depth buffer since I was not sure what "ndc_linear_depth" meant. I was told to implement a

normalised device coordinate linear depth

Which confused me since Vulkan depth is already 0-1 and NDC is -1:1 which would produce inaccuracy for the depth? Regardless, I implemented what I thought would be correct but that did not seem to produce the right result for Alchemy AO. I was curious to know if the depth buffer is what was causing the issue. Would appreciate any insight. I am using Vulkan and GLSL

layout(binding = 3) uniform sampler2D texNoise;
layout(binding = 6) uniform sampler2D depthMap;
layout(binding = 2) uniform sampler2D gNormal;

layout(location = 1) in vec2 uvCoords;

layout(binding = 5) uniform UniformBufferObject {
mat4 model;
mat4 view;
mat4 proj;
float time;
}camera;

layout(binding = 4) uniform KernelSample {
vec3 samples[64];
mat4 projection;
vec4 camera_eye;
vec4 camera_center;
float z_far;
}kernelsamples;

layout(location = 0) out vec4 outColor;

vec4 ambient_occlusion;

const vec2 window = vec2(1920.0, 1080.0);

float near = 0.1;
float zfar = 1000;

float tc_depth(in vec2 tc)
{
return texture(depthMap, vec2(1.0 - uvCoords.x, uvCoords.y)).x;
}

float LinearizeDepth(in vec2 tc) {
float buffer_z = texture(depthMap, tc).x;
return (2.0 * near) / (zfar + near - buffer_z * (near - zfar));
}

float ndcDepth(in vec2 tc) {
float depth = texture(depthMap, tc).x;

return camera.proj[3][2] / (camera.proj[2][2] + depth);
}

float ec_depth(in vec2 tc)
{
float buffer_z = texture(depthMap, tc).x;
return camera.proj[3][2] / (-2.0 * buffer_z + 1.0 - camera.proj[2][2]);
}

void main()
{
vec2 tc_depths = gl_FragCoord.xy / window;

// Normals
vec3 normals = normalize(texture(gNormal, tc_depths).rgb * 2.0 - 1.0);

// normals in world coordinates
vec3 wc_normal = normalize(transpose(mat3(camera.view)) * normals);

float ndc_linear_depth = ndcDepth(tc_depths);

// camera_center(target) - camera_eye = world coordinate camera ray direction
vec3 wc_position_current_frag = kernelsamples.camera_eye.xyz + normalize((kernelsamples.camera_center.xyz - kernelsamples.camera_eye.xyz)) * ndc_linear_depth;

ambient_occlusion.a = 0.0;

const int base_samples = 0;
const int min_samples = 64;
const float projection_factor = 0.75;
const float bias = 1.000;
const float sigma = 2.0;
const float epsilon = 0.0001;

int samples = max(int(base_samples / (1.0 + base_samples * ndc_linear_depth)), min_samples);

mat4 inverse_view_projection_matrix = inverse(camera.view);

vec2 inverted_random_texture_size = 1.0 / vec2(textureSize(texNoise, 0));
vec2 tc_random_texture = gl_FragCoord.xy * inverted_random_texture_size;

// scale texture back to [-1: 1] range
vec3 random_direction = texture(texNoise, tc_random_texture).xyz;
random_direction = normalize(random_direction * 2.0 - 1.0);

for(int i = 0; i < samples; i++)
{
// select random point
vec2 sample_random_direction = texture(texNoise,  vec2(float(i) * inverted_random_texture_size.x, float(i / textureSize(texNoise, 0).x) * inverted_random_texture_size.y)).xy;
sample_random_direction = sample_random_direction * 2.0 - 1.0; //to screen coordinates

//read position and depth for sample
vec3 tc_sample;
tc_sample.xy = tc_depths + sample_random_direction * projected_radius;
tc_sample.z = tc_depth(tc_sample.xy);

vec3 ndc_sample = tc_sample * 2.0 - 1.0;
vec4 temporary = inverse_view_projection_matrix * vec4(ndc_sample, 1.0);
vec3 wc_sample = temporary.xyz / temporary.w;

vec3 v = wc_sample - wc_position_current_frag;

ambient_occlusion.a += max(0.0, dot(v, wc_normal) - bias) / (dot(v, v) + epsilon);
}

ambient_occlusion.a = max(0.0, 1.0 - 2.0 * sigma / float(samples) * ambient_occlusion.a);

outColor = vec4(ambient_occlusion.a, ambient_occlusion.a, ambient_occlusion.a, 1.0);

}


• Would be helpful to know what the "odd results" actually are you are talking about. Nov 26 '21 at 7:43

I feel it could do with the depth buffer since I was not sure what "ndc_linear_depth" meant. I was told to implement a

normalised device coordinate linear depth

Which confused me since Vulkan depth is already 0-1 and NDC is -1:1 which would produce inaccuracy for the depth?

NDC (normalized device coordinates) is just a coordinate system with the value range -1 to 1 on all its axis. If you have your values in the range 0 to 1, the simple transformation would be:

$$2 \cdot \mathbf{v} - 1$$

where $$\mathbf{v}$$ is your coordinate vector or depth value. It basically "just doubles the distances" between your depth values but the relations remain the same. So if one pixel is 4 times closer to the camera plane than another one in your original coordinate system, it will still be 4 times closer after the transformation. Since we are using floating-point numbers here, the only inaccuracies you should get are the usual floating-point math errors (most likely negligible). However, if you are storing values only in the range 0 to 1 in a buffer that supports values from -1 to 1, you are using just half of its value range and effectively reducing your available precision by 50%.

Now let's get to the "linear" part. In a linear depth buffer, the difference between two depth values always translates to the same distance. So if your NDC depth buffer covers an area of $$10$$ meters, the difference between the values $$-0.3$$ and $$-0.1$$ would be $$1$$ meter as it would be for the values $$0.8$$ and $$1.0$$.

Now consider, that your depth buffer has a limited set of values determined by the number of available bits. The problem with a linear depth buffer for rendering purposes is that you "waste bits" on far-away objects. You can not really tell if an object $$1$$km away is closer by $$1$$cm than another object. If both objects are just a meter away, it is more obvious. The closer object gets the smaller is the noticeable distance difference. However, at some point, you hit the minimal representable distance difference of your depth buffer. This leads to undesired effects like an object that is farther away being rendered in front of a closer one just because they share the same depth buffer value due to the limited number of values.

To fix this problem, one can apply a non-linear function to the depth values so that you can distinguish much smaller distances for closer objects than for far-away objects. In this non-linear depth buffer, assuming your camera resides at the depth value $$-1$$, the covered distance between the values $$-0.3$$ and $$-0.1$$ might be just a few meters while the distance between the values $$0.8$$ and $$1.0$$ is now more than a kilometer. A better explanation of this can be found here (see section Depth value precision)

So to come back to your problem: If you use non-NDC values, non-linear values, or both in an equation that expects linear NDC values, you will most likely get the wrong results. This might be what causes your "odd results"

Here is a linearize depth function that expects the non-linear values to be between [0,1] and works in Vulkan.

float LinearizeDepth(float d)
{
return zNear * zFar / (zFar + d * (zNear - zFar));
}


I recommend breaking the texture lookup and the linearization into seperate functions to help keep the code base clean/readable and have less bugs.

Also note that if this doesn't work then the issues with the code run deeper then just depth linearization.