I'm trying to implement an hybrid renderer based on Vulkan extension VK_NV_ray_tracing. Basically, the rasterizer compute the albedo and depth map of the scene. Then I try to compute the distance from the depth buffer so that I can send a shadow ray from a raygen program to compute direct lighting. I linearize the depth to obtain a correct distance value from the depth buffer. What I am doing is this:
const vec2 texcoord = vec2(gl_LaunchIDNV) / vec2(gl_LaunchSizeNV);
const double z_b = texture(depthBuffer, texcoord).x;
const double depth = (2.0 * near * far) / (far + near - z_b * (far - near));
I am then computing the ray position and direction
const vec2 pixelCenter = vec2(gl_LaunchIDNV.xy) + vec2(0.5);
const vec2 inUV = pixelCenter / vec2(gl_LaunchSizeNV.xy);
const vec2 d = inUV * 2.0 - 1.0;
const vec4 camFront = cam.viewInverse * vec4(0, 0, 1, 0);
const vec4 camPos = cam.viewInverse * vec4(0, 0, 0, 1);
const vec4 camTarget = cam.projInverse * vec4(d.x, d.y, 1, 1);
const vec4 camDir = cam.viewInverse * vec4(normalize(camTarget.xyz), 0);
And the final ray position is then set using the depth I just computed.
const vec3 pos = camPos.xyz + camDir.xyz * float(depth);
The problem here is that the depth seems incorrect because shadows tend to curve at the edges of the screen and do not fit the geometry.
I think this might be because of the depth buffer value that does not represent the distance from the origin of the camera but from the near plane, which would explain the problem, but I am not sure of that.
Any idea?