I'm using the ZED 2 camera, and although the API provides a means of getting a point cloud position from a specific pixel, for my project I need to be able to perform this calculation myself from the depth pass alone. According to the documentation:
"Depth maps captured by the ZED store a distance value (Z) for each pixel (X, Y) in the image. The distance is expressed in metric units (meters for example) and calculated from the back of the left eye of the camera to the scene object."
So my question is, given a pixel index (therefore UV value) and a depth value, how can I work out the world location of the pixel/point? I suspect this has something to do with inverting a matrix (camera? projection?), and I have the following calibration values for the left sensor of the camera:
fx: 1057.1 fy: 1056.71 cx: 979.01 cy: 531.934 k1: -0.0412 k2: 0.0095 k3: -0.0047 p1: -0.0005 p2: -0.0002
But matrices are far from my strong suit so I'm struggling to understand how to turn the above information into a matrix which I could use to extrapolate the position. Can anyone help me?