Finding Camera projection matrix from known input and output

Lets say I have a rectangle with known coordinates in 3D scene, and I'm projecting it using pin point camera to the plane (screen).

You can imagine it deforms on the screen per camera transformation and projection.

And I know the resulting project 2D coordinates on the screen.

I wonder if I can recover camera parameters (projection matrix, transformation, field of view, distance from the screen and from the object) from known coordinates of the rectangle in 3d and know coordinates/results for 2d projection.

Thank you.

PS: if this task has infinite many solutions or unsolvable - what a minimum extra information I could gather or add to actually solve it ?

UPDATE (08/08/22): Found this lecture notes that seems to describe similar problem and call it Camera Calibration - https://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/EPSRC_SSAZ/node5.html#SECTION00050000000000000000

Similar questions were discussed at StackOverflow: https://stackoverflow.com/questions/64613719/recover-a-projection-matrix-with-known-3d-and-2d-points

And here is module that seems to solve this problem - https://docs.opencv.org/3.4/d9/d0c/group__calib3d.html

I'll try it out, for now would consider this to be a solved question.

• Camera has 7 degrees of freedom you have 8 points with 2 degrees each. So yeah should be possible. Aug 4, 2022 at 19:00
• I think the near/far planes of the projection matrix could be a problem, nevertheless the 2d coordinate will always be the same... Aug 4, 2022 at 21:22