I am trying to get up to speed with the NV_mesh_shader extension from NVidia (on my RTX2080). Samples are rare, so I tried to come up with my own small sample.

I am rendering a sphere (20 triangles) using an index buffer and a vertex buffer. I can render it the "usual" way using "vertex pulling" with two shader storage buffers, one for vertices, one for indices, (like I want to do it with a mesh shader) just fine.

When using a mesh shader (no task shader so far) I get the following problem: calling glDrawMeshTaskNV(0,1)with the mesh shader set to layout(local_size_x=20) in I only get a single triangle rendered. It looks like the mesh shader is not being invoked 20 times as I think it should? If I call glDrawMeshTaskNV(0,20) and set layout(local_size_x=1) in it renders just fine with nothing else changed in the shader. I use gl_GlobalInvocationID.x to pull the proper vertices for a triangle.

Here's the complete shader for the non-working variant:

#version 450

#define GROUP_SIZE 20
layout(local_size_x=GROUP_SIZE) in;

layout(max_vertices=3, max_primitives=1) out;
layout(triangles) out;

// vertex/index data setup
struct Vertex
{
vec4 position;
vec4 uv;
};

{
Vertex vertex[];
};

{
uint indices[];
};

uniform mat4 model;
uniform mat4 view;
uniform mat4 projection;

// mesh output
out gl_MeshPerVertexNV {
vec4 gl_Position;
} gl_MeshVerticesNV[];

out uint gl_PrimitiveCountNV;
out uint gl_PrimitiveIndicesNV[];

void main()
{
// get starting index
const uint primitive_start_index = gl_GlobalInvocationID.x * 3;

uint index_A = indices[primitive_start_index];
uint index_B = indices[primitive_start_index+1];
uint index_C = indices[primitive_start_index+2];

// get vertices
vec3 A = vertex[index_A].position.xyz;
vec3 B = vertex[index_B].position.xyz;
vec3 C = vertex[index_C].position.xyz;

gl_MeshVerticesNV[0].gl_Position = projection * (view * (model * vec4(A, 1.0)));
gl_MeshVerticesNV[1].gl_Position = projection * (view * (model * vec4(B, 1.0)));
gl_MeshVerticesNV[2].gl_Position = projection * (view * (model * vec4(C, 1.0)));

gl_PrimitiveIndicesNV[0] = 0;
gl_PrimitiveIndicesNV[1] = 1;
gl_PrimitiveIndicesNV[2] = 2;

gl_PrimitiveCountNV += 1;
}


Keep in mind this is only a learning sample, I know this is maybe overkill for a mesh shader :)

I managed to figure it out. A coworker told me that I was misinterpreting the documentation. layout(max_vertices=3, max_primitives=1) out; actually means max_vertices and max_primitives per workgroup, not per shader invocation. So I was always only outputting a single triangle, regardless of what I set local_size_x to.