I'm having some trouble trying to figure out how to go from a coordinate on the screen, to 3D world coordinates on a plane.
My case is as seen on the picture further down. World origin is in the center with the grid representing the X-Y plane.
I want to be able to click/touch on the screen, and transform the screen coordinates to the correct 3D world coordinates on the X-Y plane.
I first tried to figure out the maths myself by simple geometry, but got stuck...
I then tried the function
glm::unProject() after checking out this post.
The screen has dimensions
y are the screen coordinates where origin is in the top left corner. Positive x to the right, positive y down.
projM is the projection matrix.
Third parameter is the model matrix, which I believe in my case only needs to be an identity matrix.
I get the world coordinates X and Y (Z is already known as it is on the plane) by using these parameters in
glm::vec3 posVec = glm::unProject( glm::vec3(x, float(mWindowHeight) - y, 1.0f), glm::mat4(1.0f), projM, glm::vec4(0.0f, 0.0f, float(mWindowWidth), float(mWindowHeight)) );
What happens is that if I have screen coordinates near world origin, the world coordinates seems correct. But If I use screen coordinates near the edges and corners of the screen, I get a certain offset (see image below).
Where is this offset coming from, and what am I misunderstanding?
As mentioned in the comments, the second parameter should be the "model-view" matrix. I tried to switch
viewM (view matrix). The effect is still the same, if not with even more offset.
I also tried to get the
z buffer depth value by using:
(Got it from this tutorial)
float z; glReadPixels(int(x), int(mWindowHeight) - int(y), 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &z);
This results in very small values of X and Y world coordinates, and essentially where ever I touch, the resulting coordinates are near the origin.