# Can pixels on the image plane be mapped to world space coordinates?

I'm trying to learn the fundamentals of 3d computer graphics by writing a toy framework in WebGL.

I'm currently working on mouse based camera controls. There are plenty of good resources in the ACM SIGGRAPH library to be found, but there are some aspects of this area that I can't find answers to.

A lot of orbit controls simply use the origin as the pivot point, but other implementations can orbit around any arbitrary point in a scene and the camera stays 'still'. I'm trying to implement the latter.

It seems that a good way to do this would be to simply get the cross product of the two vectors made by the pivot point and 3d coords of the corresponding pixels on the image plane, but I can't anything like this approach anywhere which leads me to believe I've misunderstood some fundamental aspect of image projection/image planes.

Is the image plane an abstract construct that doesn't really have 3d coords or can pixels in the image plane actually be mapped to world coords?

• "other implementations can orbit around any arbitrary point in a scene and the camera stays 'still'." - please elaborate. – lightxbulb Sep 8 '19 at 21:52
• Imagine several objects in a scene and all those objects are placed around the origin. The camera looks at the origin (which is in the center of the image plane). Normally, the camera orbits the origin (which is also the pivot point), but by picking a different pivot point, the camera remains looking at the origin, but all the objects rotate around the pivot point. – oorst Sep 9 '19 at 0:07
• So you want a collection of objects revolving around some point in space? – lightxbulb Sep 9 '19 at 8:01
• Amongst other things, yes. But I want to experiment with this stuff to help me understand. I guess what I'm really after is a way to determine the vector from a pixel on the image plane to any point in the scene and to know if that's even possible. – oorst Sep 9 '19 at 22:07
• A collection of objects rotating around another object, can be done through hierarchical transformations. In your case T_pivot * R_pivot * T_model * pos. Transforming screen-space coords to world coords is an entirely different problem. There are many articles on this, here's one that includes it: mynameismjp.wordpress.com/2010/09/05/position-from-depth-3 – lightxbulb Sep 11 '19 at 8:29