OpenGL task/mesh shader extension NV_mesh_shader not behaving as I expected

Question

I am trying to get up to speed with the NV_mesh_shader extension from NVidia (on my RTX2080). Samples are rare, so I tried to come up with my own small sample.

I am rendering a sphere (20 triangles) using an index buffer and a vertex buffer. I can render it the "usual" way using "vertex pulling" with two shader storage buffers, one for vertices, one for indices, (like I want to do it with a mesh shader) just fine.

When using a mesh shader (no task shader so far) I get the following problem: calling glDrawMeshTaskNV(0,1)with the mesh shader set to layout(local_size_x=20) in I only get a single triangle rendered. It looks like the mesh shader is not being invoked 20 times as I think it should? If I call glDrawMeshTaskNV(0,20) and set layout(local_size_x=1) in it renders just fine with nothing else changed in the shader. I use gl_GlobalInvocationID.x to pull the proper vertices for a triangle.

Here's the complete shader for the non-working variant:

#version 450
#extension GL_NV_mesh_shader : require

#define GROUP_SIZE 20
// mesh shader setup
layout(local_size_x=GROUP_SIZE) in;

layout(max_vertices=3, max_primitives=1) out;
layout(triangles) out;

// vertex/index data setup
struct Vertex
{
    vec4 position;
    vec4 uv;
};

layout(std430, binding=1) readonly buffer vertex_data
{
    Vertex vertex[];
};

layout(std430, binding=2) readonly buffer index_data
{
    uint indices[];
};

uniform mat4 model;
uniform mat4 view;
uniform mat4 projection;

// mesh output
out gl_MeshPerVertexNV {
    vec4 gl_Position;
} gl_MeshVerticesNV[];


out uint gl_PrimitiveCountNV;
out uint gl_PrimitiveIndicesNV[];

void main()
{
    // get starting index
    const uint primitive_start_index = gl_GlobalInvocationID.x * 3;

    uint index_A = indices[primitive_start_index];
    uint index_B = indices[primitive_start_index+1];
    uint index_C = indices[primitive_start_index+2];

    // get vertices
    vec3 A = vertex[index_A].position.xyz;
    vec3 B = vertex[index_B].position.xyz;
    vec3 C = vertex[index_C].position.xyz;

    gl_MeshVerticesNV[0].gl_Position = projection * (view * (model * vec4(A, 1.0)));
    gl_MeshVerticesNV[1].gl_Position = projection * (view * (model * vec4(B, 1.0)));
    gl_MeshVerticesNV[2].gl_Position = projection * (view * (model * vec4(C, 1.0)));

    gl_PrimitiveIndicesNV[0] = 0;
    gl_PrimitiveIndicesNV[1] = 1;
    gl_PrimitiveIndicesNV[2] = 2;

    gl_PrimitiveCountNV += 1;
}

Keep in mind this is only a learning sample, I know this is maybe overkill for a mesh shader :)

Thanks for any hints. pettersson

Answer 1

I managed to figure it out. A coworker told me that I was misinterpreting the documentation. layout(max_vertices=3, max_primitives=1) out; actually means max_vertices and max_primitives per workgroup, not per shader invocation. So I was always only outputting a single triangle, regardless of what I set local_size_x to.

Changing the values to proper numbers resulted in a proper rendering.