Camera model for rendering the viewport as a window frame?

Question

I want to render as if someone is looking at a landscape through a window, adjusting for the viewer's head position. The landscape is easy enough (just a distant plane), but I'm not sure how to approach the window frame.

It seems like I want perspective within the window frame but for the frame itself to be fixed to the four corners of the viewport. I'm not sure what I want matches a frustum model, because the view I want isn't rectangular? Or maybe at really far distances and low FOV it'll get close?

Here's an example of rendering from a distan camera with a small FOV, one head-on and one from a side view. The head-on is fine, but in the side-view the frame has depth but is no longer aligned with the viewport.

I'm using three.js but not tied to that and can rewrite the shaders if need be. I have a demonstration of a stylized frame with a distant camera, high zoom, and exaggerated lateral movement, which sort of gets what I'm after, but I'm not confident in the approach.

Answer 1

The magic phrases for what I'm trying to achieve are "oblique frustum", "asymmetric projection", or "off-axis camera". I haven't implemented it in three.js but achieved the right effect with this unity plugin. But the general concept is to use the more esoteric fields in the projection matrix to create a skewed view volume.

• https://docs.unity3d.com/Manual/ObliqueFrustum.html
• https://jsantell.com/portals-with-asymmetric-projection/
• https://assetstore.unity.com/packages/tools/camera/offaxiscamera-98991