I'm using OpenGL's stencil test to carve out a semicircular hole in a transition frustum with a cylinder oriented perpendicularly. However as one can see below, while this objective is achieved, the cylinder is still visible through the frustum which shouldn't be the case.
I'm using the following code fragment that I found at https://en.wikibooks.org/wiki/OpenGL_Programming/Mini-Portal
glClear(GL_DEPTH_BUFFER_BIT);
glEnable(GL_STENCIL_TEST);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
glDepthMask(GL_FALSE);
glStencilFunc(GL_NEVER, 1, 0xFF);
glStencilOp(GL_REPLACE, GL_KEEP, GL_KEEP); // draw 1s on test fail (always)
// draw stencil pattern
glStencilMask(0xFF);
glClear(GL_STENCIL_BUFFER_BIT); // needs mask=0xFF
DRAW_CYLINDER ();
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
glDepthMask(GL_TRUE);
glStencilMask(0x00);
// draw where stencil's value is 0
glStencilFunc(GL_EQUAL, 0, 0xFF);
/* (nothing to draw) */
// draw only where stencil's value is 1
glStencilFunc(GL_EQUAL, 1, 0xFF);
DRAW_FRUSTUM();
glDisable(GL_STENCIL_TEST);
Moreover I've found that the results are insensitive to the parameters of the glStencilOp and DepthMask commands.
I seem to vaguely remember from several years ag that a multi-pass rendering may be required. If so, could someone point me to how to accomplish this with the proper set of command parameters?
thanks in advance.
This is an example of what I'm expecting - The cylinder should cut a hole through the frustum. I've rendered it here by trimming the black cylinder within the frustum so that it just just out past the frustum surface.