I have a projection calculation which maps points from 3D space to pixel (image) space, and I want to use it as instead of the standard MVP projections in a vertex shader.

In the image below the lower blue flow is how I understand the pipeline works in GLSL, and the upper green arc is what I want to use instead.

Since GLSL's projection matrix doesn't produce pixel coordinates, but a homogenous vec4 that allows to calculate pixel coords and depth estimation, I need to make some adjustments:

My idea is to take the vec2 pixel coordinates from my calculation, map them to clip space and let openGL finish the conversion to pixel space by itself

Only problem is I'm not entirely sure how to pull this off, as you can see Im missing some essential data about what happens after Clip space

Any ideas?

EDIT: The GLSL implementation of the 3D to 2D projection function can be found here https://www.shadertoy.com/view/fltfW2

• Can you add some information about the way you calculate the pixel position? Sep 19, 2022 at 7:50
• At the first "???" there the 4D vector will be divided by its "w" component. The geometry will be clipped to range [-1, 1] in all dimensions (z value range can be clipped on a different range depending on the graphics library you are using).... Sep 19, 2022 at 9:29
• Hi @Thomas, thanks for helping! Unfortunately I cannot disclose the calculations or matrices but I can say its been tested to take a point in 3D space and return it's projected 2D pixel coordinate (image / screen size is taken into account of course). it has been tested both in python and in GLSL (shadertoy) Sep 19, 2022 at 9:41
• when having a triangle (3 vertices) is the surface spanning this triangle a triangle in your calculated space as well? Sep 19, 2022 at 9:49
• the relationship between the vertices doesn't change so a triangle will remain a triangle. Sep 19, 2022 at 11:08