Stencil buffer appears to not be decrementing values correctly
Posted
by
Alex Ames
on Game Development
See other posts from Game Development
or by Alex Ames
Published on 2012-03-28T07:58:11Z
Indexed on
2012/03/28
17:47 UTC
Read the original article
Hit count: 329
I'm attempting to use the stencil buffer as a clipper for my UI system, but I'm having trouble debugging a problem I'm running in to.
This is what I'm doing: A widget can pass a rectangle to the the stencil clipper functions, which will increment the stencil buffer values that it covers. Then it will draw its children, which will only get drawn in the stencilled area (so that if they extend outside they'll be clipped). After a widget is done drawing its children, it pops that rectangle from the stack and in the process decrements the values in the stencil buffer that it has previously incremented.
The slightly simplified code is below:
static void drawStencil(Rect& rect, unsigned int ref)
{
// Save previous values of the color and depth masks
GLboolean colorMask[4];
GLboolean depthMask;
glGetBooleanv(GL_COLOR_WRITEMASK, colorMask);
glGetBooleanv(GL_DEPTH_WRITEMASK, &depthMask);
// Turn off drawing
glColorMask(0, 0, 0, 0);
glDepthMask(0);
// Draw vertices here
...
// Turn everything back on
glColorMask(colorMask[0], colorMask[1], colorMask[2], colorMask[3]);
glDepthMask(depthMask);
// Only render pixels in areas where the stencil buffer value == ref
glStencilFunc(GL_EQUAL, ref, 0xFF);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
}
void pushScissor(Rect rect)
{
// increment things only at the current stencil stack level
glStencilFunc(GL_EQUAL, s_scissorStack.size(), 0xFF);
glStencilOp(GL_KEEP, GL_INCR, GL_INCR);
s_scissorStack.push_back(rect);
drawStencil(rect, states, s_ScissorStack.size());
}
void popScissor()
{
// undo what was done in the previous push,
// decrement things only at the current stencil stack level
glStencilFunc(GL_EQUAL, s_scissorStack.size(), 0xFF);
glStencilOp(GL_KEEP, GL_DECR, GL_DECR);
Rect rect = s_scissorStack.back();
s_scissorStack.pop_back();
drawStencil(rect, states, s_scissorStack.size());
}
And this is how it's being used by the Widgets
if (m_clip)
pushScissor(m_rect);
drawInternal(target, states);
for (auto child : m_children)
target.draw(*child, states);
if (m_clip)
popScissor();
This is the result of the above code:
There are two things on the screen, a giant test button, and a window with some buttons and text areas on it. The text area scroll box is set to clip its children (so that the text doesn't extend outside the scroll box). The button is drawn after the window and should be on top of it completely. However, for some reason the text area is appearing on top of the button. The only reason I can think of that this would happen is if the stencil values were not getting decremented in the pop, and when it comes time to render the button, since those pixels don't have the right stencil value it doesn't draw over. But I can't figure out whats wrong with my code that would cause that to happen.
© Game Development or respective owner