Get normalized device coordinate of an image pixel

Question

I have an image rendered to the screen, and based on the normalized device coordinates of the cursor I would like to pick the pixel of the image under the cursor.

Say the image has four channels, I then define a pixel struct:

struct Pixel {
  unsigned char r, g, b, a;
};

I can then get the closest index of the pixel under cursor like this:

int getPixel(float normalX, float normalY, float imageWidth, float imageHeight) {
  return (int)(normalY*imHeight)*imageWidth + normalX*imageWidth;
}

Now how would I do it the other way around, basically given the pixel index get its normalized device coordinates?

Answer 1

You can modulus by row length to retrieve the x coordinate (think about the x coordinate "wrapping around" after each line), and division by row length to get the y coordinate (inverting the multiply you have by imageWidth). We subtract one from the width and height to ensure the last pixel in the row/column maps to 1.0. Note that this isn't "Normalized Device Coordinates", since that implies a 3D quantity which also stores depth, and that all coordinates lie in [-1,1].

struct Point2D { float x; float y; };

Point2D GetNormalizedPixelCoords(int pixelIdx, int imageWidth, int imageHeight) {
    Point2D point;
    point.x = (pixelIdx % imageWidth) / (float)(imageWidth-1);
    point.y = (pixelIdx / imageWidth) / (float)(imageHeight-1);
    return point;
}

Answer 2

The first step to achieving what you are asking for would be to find the normalized texture coordinates a.k.a UV-coordinates of the pixel. This can be done as described by AJ Weeks in his answer. However, I wouldn't divide by width-1/height-1, but instead, I would add 0.5 to the x- and y-coordinate and divide by the full width and height. This should give you the "center" of each pixel. As an example, say you have an 8x8 texture and index 10. This would yield x=1 and y=2. After the addition, you get x=1.5 and y=2.5. Dividing each of those by 8 gives you the normalized texture coordinates of u=0.1875 and v=0,3125.

Depending on how you render your image, the next step might be a little bit tricky. In general, if your texture is mapped onto an arbitrary mesh, you need to find the triangle that includes the UV-coordinates of the pixel by performing a 2d-point-in-triangle test against the triangles' vertices UV-coordinates. Notice, that depending on your mesh's texture coordinates, there might be more than one triangle or none if you use the texture repetitively or only a certain area of it. If this is not the case, there still might be multiple triangles that contain it, if the point lies on a boundary or vertex. In this case, pick an arbitrary triangle, since the result should be the same.

Now if you have found the triangle determine the barycentric coordinates of the pixel from its UV-coordinates and the vertices of the triangle. Then use those coordinates and multiply them with the vertices 3d coordinates to get the actual 3d coordinates of the pixel in model space.

Finally, perform the usual transformations with your transformation matrices -> model to world -> world to camera -> perspective projection. Then you have your NDC coordinates of the pixel.

The determination of the 3d coordinates in model space can be simplified if your model is just a flat rectangular surface described by 4 vertices that has the one and only purpose to display the whole undistorted texture without repetitions, offsets or scaling (means the corner point vertices UV coordinates are either 0 or 1). Then you can perform a linear interpolation of the 3d coordinates in x and y direction where the UV-coordinate values of the pixel are the interpolation weights in the corresponding direction.

It gets even easier if you render your texture as a full-screen image. Then you only need to subtract 0.5 from your UV-coordinates and multiply the result by 2. This transforms the UV-range [0,1] to the NDC range [-1,1]. The depth value can be chosen arbitrarily.