I am writing a polygon rasterization renderer. Some meshes render flawlessly, but some produce strange artefacts in the depth buffer. These artefacts do not just appear when I save the depth buffer for inspection but are clearly part of the buffer itself during runtime, as you can see it affecting the result of rendering very clearly. Does this look like some familiar problem? Do you have advice on how to diagnose this issue if not? I have attached two images, one of the render result and one of the saved depth buffer. I have also included a snippet of my code that handles depth testing:
if(mat_min(zs) < 0.0 || mat_max(zs) > 1.0)
{ return; }
// Exit if vertex is outside of depth range ^
...
double zpc_inv = bary.data[0] / zcs.data[0] + bary.data[1] / zcs.data[1] + bary.data[2] / zcs.data[2];
double zpc = 1.0 / zpc_inv;
double upc = bary.data[1] * zpc / zcs.data[1];
double vpc = bary.data[2] * zpc / zcs.data[2];
double wpc = 1.0 - upc - vpc;
bary = mat_row(3, wpc, upc, vpc);
// Calculate new perspective-correct barycentric coordinates ^
double z = mat_lincomb(zs, bary).data[0];
double buf_z = depthbuf[y * frame->width + x];
// Calculate depth from said coords ^
if(z > buf_z)
{
...
depthbuf[y * frame->width + x] = z;
}
// Draw if depth is suitable ^
EDIT: Based on an answer given here, I decided to do some experimentation with bounds. The artefacting is somehow caused by pixels which should have invalid bounds passing bounds checks. By changins some <= and >= into < and >, I tightened the bounds on what is considered a "valid" barycentric coord for shading and it helped significantly but did not eliminate the problem entirely.