I am experimenting on a simple wave tessellation shader. I ported it from a working vertex shader. When my frequency is higher than 1, I get strange "saw-tooth" artifacting at peaks of the wave. Here are some images with a frequency of 3.

Wireframe artifacts


If I keep the frequency lower, it is smooth as butter. Any ideas what the problem could be?

#version 410 core
layout(quads, equal_spacing, ccw) in;
in vec2 teUv[];
out vec2 fUv;
out vec4 fWorld_pos;
uniform mat4 VP;
uniform mat4 M;
uniform sampler2D water_disp;
uniform float time;
const float M_PI = 3.1415926535;

float water_displacement(vec2 pos) {
    const float freq = 3.0; // <-- This is the problematic value.
    float d = length(pos);
    return (0.5 + 0.5 * sin(-M_PI * d * freq + time));

void main() {
    vec4 p1 = mix(gl_in[0].gl_Position,
    vec4 p2 = mix(gl_in[2].gl_Position,
    vec4 local_pos = mix(p1, p2, gl_TessCoord.y);
    fWorld_pos = M * local_pos;
    fWorld_pos.y += water_displacement(fWorld_pos.xz);

    gl_Position = VP * fWorld_pos;

    vec2 uv1 = mix(teUv[0], teUv[1], gl_TessCoord.x);
    vec2 uv2 = mix(teUv[2], teUv[3], gl_TessCoord.x);
    fUv = mix(uv2, uv1, gl_TessCoord.y);

And more examples:

Smooooth (freq = 0.5) Not so smooth (freq = 8)


1 Answer 1


Not enough triangles. The situation is analogous to the sampling theorem that states that you can not reconstruct a signal if your sample frequency is below a certain threshold. Although in this, case you're not attempting to rebuild the signal with a higher order filter so the result is even more dramatic. And it results in interpolation errors.

You can alleviate the problem with less points if your point wavefront follows your waves. So you may want to shift points towards the gradient maximum in other directions than UP.

  • $\begingroup$ I believe you're looking for the Nyquist rate. Also, he is very much attempting to rebuild a signal. That signal being a radial sine wave. This is very much a case of aliasing: the digital sampling of an analog signal with fewer samples than required to reconstruct it. $\endgroup$ Aug 5, 2017 at 15:09
  • $\begingroup$ Wow I never realized the nyquist limit applied to graphics (I'm an ex-sound engineer turned programmer) :) Can I ask more details (or a link) when you refer to shifting points toward the gradient maximum? $\endgroup$
    – scx
    Aug 5, 2017 at 15:35
  • $\begingroup$ @NicolBolas I meant he is just linear interpolating points not recosntructing the signal with a higher order fuction. $\endgroup$
    – joojaa
    Aug 5, 2017 at 17:19
  • 2
    $\begingroup$ @scx points in space are signals. Shader shards are signals. Antialiasing is exactly the same in sound as in images... etc. A lot of signal processing applies to images. Scaling, rotating etc. same things. $\endgroup$
    – joojaa
    Aug 5, 2017 at 17:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.