I'll start with the coordinate system I'm trying to use: If I were standing on the ground, facing north, the x axis is a line moving from the west to the east (positive X is east). Positive y is north. Positive Z is the line from my feet up toward my head.

I have an image that represents a flat plane, upon which I might be standing - the Z coordinate is 0 for the whole image; it extends along the XY plane. My image is x by y pixels (we'll say 10000 by 5000) and that represents some real world unit - I don't care that much about the scale. The center of the image is located at the origin, so the top-left corner of the image (0, 0 pixel) is at (-5000, 2500).

I have a camera in this hypothetical, and I know its intrinsic matrix - we can just stick to the pinhole model for now, I can worry about distortion later. I also know the extrinsic matrix of this camera, because I know exactly where I want to position it in world space.

Given this information, I want to warp the image of the plane into how my hypothetical camera would see it.

With this goal in mind, I have a few questions:

  1. I am assuming that an initial rotation matrix of I_(3x3) would be equivalent to the camera looking straight down. Is this correct? If not, what does the identity rotation matrix correspond to?

  2. I am currently calculating a homography to use with cv2.warpPerspective. To calculate this homography, I am doing:

translationVector = np.matrix([[xTranslation], [yTranslation], [zTranslation]])
extrinsic = np.block([R_3x3,-R_3x3]@translationVector])
intrinsic = np.matrix([[f, 0, outputWidth/2],
                       [0, f, outputHeight/2],
                       [0, 0,   1]])
composite = extrinsic[:, [0, 1, 3]]
homography = intrinsic[:, :3] @ composite

When I use this homography matrix (using x and y translation to bring the camera to the center of the image), however, I get a weird... duplication of the source image, where it appears on the top and the bottom of the image. I've provided an example of what I mean - in this example, there is a positive Z offset, so only the portion of the image on the bottom should exist. (Note: in this case my 3x3 rotation matrix has rotation of +90.0001 degrees in X - more on this in question 3).


2A) Why does that "duplicate" appear, and how do I get rid of it?

2B) It seems like my image is rotated oddly. If I add 180 degrees rotation in Z it looks how I expect: is this expected or have I done something wrong? Is this, perhaps, because in the pinhole model the image gets inverted in x and y?

  1. If I input a rotation angle of exactly 90 degrees in X, a flickering white line appears horizontally along the center of the image with some z translation values and not with others. I assume this is some mathematic instability - is there any way to get rid of this, or is using "90.00001" degrees the best I can manage?

  2. Is there a better/faster/more direct way to warp the image - which always represents a flat XY plane - into the perspective of a camera with known intrinsics/extrinsics?

Thank you!

  • $\begingroup$ Welcome to CG. I know that it sometimes seems to make sense to ask several questions regarding one topic but we prefer one question per thread here. This way your question(s) are easier to find and understand for others in the future who might have similar ones. $\endgroup$
    – wychmaster
    Nov 25, 2022 at 10:14
  • $\begingroup$ @wychmaster That makes sense - I think we can consider basically all of this supporting information to the question of "is my approach reasonable, and if so, why do I have this weird double in the provided picture?" If you'd like me to edit it relative to that, I can do so. $\endgroup$
    – Helpful
    Nov 26, 2022 at 5:11


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.