0
$\begingroup$

I am trying to perform quad-based tessellation using GLSL on Unity to be able to render polynomial-based parametric surfaces on a Unity quad. The quad-based tessellation using GLSL seems to be incorrectly choosing the vertices for the patch to be tessellated. I am using this shader (https://github.com/przemyslawzaworski/Unity3D-CG-programming/blob/master/GLSL/tessellation.shader) as a reference. However, modifying it to use quads in the domain shader stage messes up the ordering.

Shader "Custom/Tessellation"
{
    Properties
    {
        _MainTex ("Texture", 2D) = "white" {}
        _factor("Tessellation scale",Range(1.0,64.0)) = 1.0
    }
    SubShader
    {
        Cull Off
        Pass
        {
            GLSLPROGRAM
            #version 460     
            uniform float _factor;

            #ifdef VERTEX
                in  vec4 in_POSITION0;
                void main()
                {
                    gl_Position =  gl_ModelViewProjectionMatrix * in_POSITION0;
                }
            #endif

            #ifdef HULL          //GLSL Tessellation Control Shader

                layout (vertices = 4) out;
                void main()
                {
                    if (gl_InvocationID == 0)
                    {
                        gl_TessLevelInner[0] = _factor;   //Inside tessellation factor
                        gl_TessLevelInner[1] = _factor;   //Inside tessellation factor

                        gl_TessLevelOuter[0] = _factor;   //Edge tessellation factor
                        gl_TessLevelOuter[1] = _factor;   //Edge tessellation factor
                        gl_TessLevelOuter[2] = _factor;   //Edge tessellation factor
                        gl_TessLevelOuter[3] = _factor;   //Edge tessellation factor
                    } 
                    gl_out[gl_InvocationID].gl_Position = gl_in[gl_InvocationID].gl_Position;
                }
            #endif

            #ifdef DOMAIN        //GLSL Tessellation Evaluation Shader
                layout (quads) in;
                void main()
                {   
                    vec4 p1 = mix(gl_in[0].gl_Position, gl_in[1].gl_Position, gl_TessCoord.x);
                    vec4 p2 = mix(gl_in[2].gl_Position, gl_in[3].gl_Position, gl_TessCoord.x);
                    gl_Position = mix(p1, p2, gl_TessCoord.y);      
                }
            #endif

            #ifdef GEOMETRY      //geometry shader for rendering wireframe
                layout(triangles) in;
                layout(line_strip, max_vertices = 3) out;
                void main()
                {
                    for(int i = 0; i < gl_in.length(); ++i)
                    {
                        gl_Position = gl_in[i].gl_Position;
                        EmitVertex();
                    }
                    gl_Position = gl_in[0].gl_Position;
                    EmitVertex();  
                    EndPrimitive();
                }    
            #endif

            #ifdef FRAGMENT
                out vec4 color;
                void main()
                {
                    color = vec4(1,1,1,1);
                }
            #endif

            ENDGLSL
        }
    }
}

Please help me in identifying what modifications are incorrect. I would appreciate any and all the help. The following are the outputs both from the triangle-based shader as given in the link to the Github page above, and the quad-based shader with modifications for quads made by me.

Triangle-based Tessellation Working Properly

Quad-based Tessellation Generating Incorrect Results

$\endgroup$
  • $\begingroup$ I've not looked at the code, but visually it looks to me that you're supplying it with, say, vertices in the order "ABCD" when it perhaps expects "ABDC" (or vice versa). $\endgroup$ – Simon F Feb 7 at 9:26
  • 1
    $\begingroup$ Oh yes @SimonF it does seem to get fixed on reordering the vertices in domain shader. However, an ordering only works with one model type on Unity for me. For instance if ABCD works for Planes, then maybe a different ordering like ADBC works for Cubes. Is there any way to have a consistent ordering across objects? $\endgroup$ – Bhavyansh Mishra Feb 8 at 4:30
  • $\begingroup$ I don't know unity, but I'd be surprised that vertex ordering would change between a flat quad and a cube... but perhaps there is an inconsistency in the system. I know of some Unity people on twitter, but I don't know if they also have a presence here. $\endgroup$ – Simon F Feb 11 at 9:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.