Where exactly is the image plane / plane of projection

Question

Perspective projection matrices we use map 3D coords onto some imaginary image plane in front of the cameras pinhole position by casting a ray directly from the eye position to that 3d coordinate and seeing where it intersects the image plane.

But if you take your average 4x4 perspective projection matrix (assume no reversed z or infinite far plane at the moment), where exactly is this thing? As in, if you drew a diagram, where would you draw this "world space image plane" and all the contents in it?

It must be one of the planes orthogonal to the view direction. All points projected onto any of these planes will produce the same image only differing in scale. Yet presumably in one of them the ray along the top left of the view frustum of the camera will strike the image plane at the coordinate [-1,1] (depending on your choice of clip space coordinates), and the depth value of this intersection is disregarded (instead the distance of the point from the camera receives a custom remapping function for various reasons) and so on for the other frustum edges.

But I'm having a hard time figuring out how this can be with cameras that don't have a 1:1 aspect ratio. If I'm drawing a classical view frustum on paper and check some intersections on the frustum corners, it's never at [x,y] with |x| = |y|.

Answer 1

where exactly is this thing?

Nowhere. It's imaginary; it doesn't exist and doesn't have a position in space.

For a given point in camera space, the projection matrix defines a line between that point and the camera. You could consider the plane of projection to be any plane along that line (perpendicular to the camera view), with the point being projected defined by the intersection of that line with the plane.

The math doesn't need a projection plane. The idea is solely for the benefit of imagining how it would look.

But I'm having a hard time figuring out how this can be with cameras that don't have a 1:1 aspect ratio.

This is simulated by applying a non-uniform scale to the X and Y of the projection matrices.