I am implementing foveated rendering. My FBO is divided into 2 layers which are concentric. The first layer covers 1/3 of the Framebuffer whereas the second one covers the rest of the screen. Now I want to avoid rendering meshes in the outer layer which are already in the inner layer.
I have implemented the following culling algorithm which is only concerned with culling between the layers:
- I create rectangles with min and max corners from the radii of the 2 layers. Now these coordinates are in NDC.
I have the min and max corner of each mesh in the local coordinate system. So I transform them into NDC using:
vec4 min = mvp * vec4(min_corner, 1.0); min = min.xyzw / min.w;
I create a bounding box using min and max corners.
- I check whether 2 rectangles overlap or not.
Now, when I transform into clip space, the x or y coordinates of the min are sometimes greater than the max corner. The same issue occurs when I divide them by w. What's going wrong here?
EDIT: Okay. I found the problem. Now I iterate over all the vertices, transform each of them into NDC using MVP and then calculate the min and max corners which works perfect. But this is much more overhead for each frame. Is there any optimization I can do here?