I need to create a decaling into a surface like a mesh, and to avoid Z-fighting I've been using Stencil Buffer to Turn-off depth test and make the decaling over the surface the Stencil Op permits. In fact that decaling fails when we turn the camera in a such way that the viewer has more than one surface point "competing" for the same pixel. This happens when the surface isn't a plane, in our example we have a mesh representing a terrain. The problem is that we turn-off the depth test and the decaling object is always visible, even when we have a surface point closer to the camera. Is there someone here that had the same problem ? Do I have to quit stencil solution to avoid Z-fighting ?
It sounds like this code is using the stencil buffer to mask off the locations to draw the line. This isn't a "typical" decal approach. Normally the decal is draw directly into the scene and "depth bias" is turned on for the graphics state (OpenGL) or the graphics pipeline (Vulkan). Then turned off when done, again for OpenGL.
A solution that doesn't change the current rendering approach (which i guessed at) is to use a depth buffer when rendering the terrain. Then pass that depth buffer as an input to your decal shader, read the depth buffer and compare it to the depth of your current fragment. Then discard the fragment if the depth test fails. Basically, it is doing depth buffering by hand. It isn't to hard to set up and there are lots of tutorials out there to help get you going.
It also might be possible to just change the stencil buffer based on depth values this would be dependent on how the stencil is being generated. If it is regenerated each frame this could be a quick and easy solution.
On the other hand, if you are drawing lines or polygons directly, then you really should look into using depth bias settings. Assuming you are using OpenGL this is turned on before rendering the decal then turned off after rendering it. The hardware then computes an offset depending on how far the camera is away from the object up to some limit (which is also programable). This approach would allow the depth testing to be left on, it is view independent so it works from any direction, and it would allow the decal to be textured.
Also: The issue described here isn't really z fighting. Z fighting usually occurs when two fragments are being rendered to the same location and quantization causes the fragments to "fight" for visibility. As the camera moves one of the two fragments wins the "fight", but the winner keeps changing depending how much floating point error there is.