I am trying to build a VR tracking system with a laptop webcam, and I have succeeded in identifying, and tracking paper markers I put in front of my webcam. For context I am using OpenCV with the latest OpenCVSharp wrapper in .Net 5.
Currently I can track all 4 corners of an object (ordered, so orientation is accounted for) as a 2d quad on the webcam's image, so long as I am provided with a reference image before hand. I know the exact measurements of the reference object, and now I want to use both the 4 corners I calculate, and the measurements of the object in order to determine the position and rotation of that object relative to my webcam.
In the future I would also like to add additional cameras to the setup to allow for more precision and reducing dead zones. They would know the positions relative to the other sensors for the sake of simplicity.
How would I go about calculating both the position and rotation of an object given its' measurements, and 4 ordered corners on an image?