The rule of thumb is not about the number of instances; it's about the size of each instance.
You don't want instances that are too small. You want instances that have a good number of vertices to them. In the hundreds to low thousands range. Quads only have 4 vertices, so they are a particularly bad choice for instancing.
The benefit of instancing quads is having less vertex data to read and upload. However, there are ways of achieving these benefits without actually having to use instancing.
Using indexed rendering means that you can use 4 positions instead of six. However, you also need an index buffer. Fortunately, you can use the same index buffer for each draw call, as the topology of a list of quads is the same regardless of their vertices. This buffer can use 16-bits indices for improved performance, and you can use multidraw rendering alongside the base vertex to render more than 16K quads from a single draw call.
For the other per-quad attributes, you can access an SSBO (or UBO if you don't have that much per-quad data) given each quad's index. To compute a quad's index, you need to use gl_VertexIndex
, which gives you the index of the vertex + its base vertex offset. gl_VertexIndex / 4
is the index of that quad, which you can then use to fetch data from your SSBO.
Technically, you could do this for the positions too. You would use no vertex buffers at all; using an index buffer as above to compute the quad index. And you would compute the positions of each vertex in the quad using gl_VertexIndex % 4
to figure out which vertex that VS invocation needs to compute.
However, it is difficult to ensure continuity if two quads share a common edge (this can be done via using fixed-point integer math in the scaling and offsetting). But if you're rendering a particle system, continuity is not all that important, so this is also an option. If continuity is important, you should just use 4 positions per-quad and use per-quad indices from SSBOs for the rest.