I want to render as if someone is looking at a landscape through a window, adjusting for the viewer's head position. The landscape is easy enough (just a distant plane), but I'm not sure how to approach the window frame.
It seems like I want perspective within the window frame but for the frame itself to be fixed to the four corners of the viewport. I'm not sure what I want matches a frustum model, because the view I want isn't rectangular? Or maybe at really far distances and low FOV it'll get close?
Here's an example of rendering from a distan camera with a small FOV, one head-on and one from a side view. The head-on is fine, but in the side-view the frame has depth but is no longer aligned with the viewport.
I'm using three.js but not tied to that and can rewrite the shaders if need be. I have a demonstration of a stylized frame with a distant camera, high zoom, and exaggerated lateral movement, which sort of gets what I'm after, but I'm not confident in the approach.