How to calculate position from depth pass of the ZED camera?

I'm using the ZED 2 camera, and although the API provides a means of getting a point cloud position from a specific pixel, for my project I need to be able to perform this calculation myself from the depth pass alone. According to the documentation:

"Depth maps captured by the ZED store a distance value (Z) for each pixel (X, Y) in the image. The distance is expressed in metric units (meters for example) and calculated from the back of the left eye of the camera to the scene object."

So my question is, given a pixel index (therefore UV value) and a depth value, how can I work out the world location of the pixel/point? I suspect this has something to do with inverting a matrix (camera? projection?), and I have the following calibration values for the left sensor of the camera:

fx: 1057.1   fy: 1056.71
cx: 979.01   cy: 531.934
k1: -0.0412   k2: 0.0095   k3: -0.0047
p1: -0.0005   p2: -0.0002


But matrices are far from my strong suit so I'm struggling to understand how to turn the above information into a matrix which I could use to extrapolate the position. Can anyone help me?