Stencil Buffer Tutorial

I haven't been able to locate any stencil buffer tutorials that actually explain how to use it and what they are doing, just a bunch of tutorials that use it to achieve some effect or another, and assume you already are familiar with it, perhaps someone here knows of one?

OpenGL or DirectX?

OpenGL, sry bout that.

If I remember correctly, NeHe has a tutorial on that.

They have tutorials that use the stencil buffer, but require that you already know how to use it to understand it (which I dont :/)

that's because there's not a whole lot to understanding what the stencil buffer is, but I guess you missed the laymen's term explanation for what it is/what it is used for.

It's most basic function is to clip parts of the scene away. Say you want a bullet hole to appear in a wall. You draw a shape that looks like the hole where you want the hole to appear. You render the geometry to the stencil buffer, and for the pixels that the depth test passes (meaning it is visible) you set the stencil buffer to a value of 1. Then, when you go to draw that particular wall, you only draw where the stencil buffer is *not* equal to 1, effectively skipping where the whole would have been.

A cooler use is finding what points lay in a shadow volume, but you can't get that far until you understand what I said above.

EDIT:
I meant to say what *pixels* lay in shadow volume, not points (yes, there's a difference).

I'm reading book "OpenGL SuperBible" and I don't understand the stencil buffer thing.

So here is the example code from the book:

Code:

///////////////////////////////////////////////////////////
// Called to draw scene
void RenderScene(void)
    {
    GLdouble dRadius = 0.1; // Initial radius of spiral
    GLdouble dAngle;        // Looping variable
           
    // Clear blue window
    glClearColor(0.0f, 0.0f, 1.0f, 0.0f);
       
    // Use 0 for clear stencil, enable stencil test
    glClearStencil(0);
    glEnable(GL_STENCIL_TEST);

    // Clear color and stencil buffer
    glClear(GL_COLOR_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
               
    // All drawing commands fail the stencil test, and are not
    // drawn, but increment the value in the stencil buffer.
    glStencilFunc(GL_NEVER, 0x0, 0x0);
    glStencilOp(GL_INCR, GL_INCR, GL_INCR);

    // Spiral pattern will create stencil pattern
    // Draw the spiral pattern with white lines. We
    // make the lines  white to demonstrate that the
    // stencil function prevents them from being drawn
    glColor3f(1.0f, 1.0f, 1.0f);
    glBegin(GL_LINE_STRIP);
        for(dAngle = 0; dAngle < 400.0; dAngle += 0.1)
            {
            glVertex2d(dRadius * cos(dAngle), dRadius * sin(dAngle));
            dRadius *= 1.002;
            }
    glEnd();
           
    // Now, allow drawing, except where the stencil pattern is 0x1
    // and do not make any further changes to the stencil buffer
    glStencilFunc(GL_NOTEQUAL, 0x1, 0x1);
    glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
       
    // Now draw red bouncing square
    // (x and y) are modified by a timer function
    glColor3f(0.0f, 1.0f, 0.0f);
    glRectf(x, y, x + rsize, y - rsize);
   
    // All done, do the buffer swap
    glutSwapBuffers();
    }

Can you explain me the stencil buffer by this example.

Now I do understand it better, if I read this Silvercord explanation, but I still don't understand it completely.

For example what does meen (what it does) GL_INCR and GL_KEEP in glStencilOp function. This was an example, what I don't understand, and I don't yet understand the glStencilFunc arguments too (what they mean or do)

I am a beginner and spent a lot of time but I got it at last.
You understand the idea of Stencil buffers.
Now the question, how do you design the stencil pattern?
It is done as follows:
You tell OpenGL to fill the buffer with zeros.
Then you design an opaque stencil (nothing is drawn).
Then you tell OpenGL that whenever a pixel(form you image) is compared to the stencil buffer and it fails, increment the stencil pixel at this position by one.

When you do that and draw anything (a spiral in your example), what happens is that all the spiral is not drawn, and instead, every corresponding pixel in the stencil is incremented (becomes 1 instead of 0).

This is cutting this spiral from the stencil.

Now you change the stencil behavior. You ask him to compare pixel by pixel and keep the buffer as it is.

If you did not get the details ask me I can re explain.

Note: you asked your question before I even started learning OpenGL. May be you are now a professional graphics designer LOL

Originally Posted by joakim12

I'm reading book "OpenGL SuperBible" and I don't understand the stencil buffer thing.

So here is the example code from the book:

Code:

///////////////////////////////////////////////////////////
// Called to draw scene
void RenderScene(void)
    {
    GLdouble dRadius = 0.1; // Initial radius of spiral
    GLdouble dAngle;        // Looping variable
           
    // Clear blue window
    glClearColor(0.0f, 0.0f, 1.0f, 0.0f);
       
    // Use 0 for clear stencil, enable stencil test
    glClearStencil(0);
    glEnable(GL_STENCIL_TEST);

    // Clear color and stencil buffer
    glClear(GL_COLOR_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
               
    // All drawing commands fail the stencil test, and are not
    // drawn, but increment the value in the stencil buffer.
    glStencilFunc(GL_NEVER, 0x0, 0x0);
    glStencilOp(GL_INCR, GL_INCR, GL_INCR);

    // Spiral pattern will create stencil pattern
    // Draw the spiral pattern with white lines. We
    // make the lines  white to demonstrate that the
    // stencil function prevents them from being drawn
    glColor3f(1.0f, 1.0f, 1.0f);
    glBegin(GL_LINE_STRIP);
        for(dAngle = 0; dAngle < 400.0; dAngle += 0.1)
            {
            glVertex2d(dRadius * cos(dAngle), dRadius * sin(dAngle));
            dRadius *= 1.002;
            }
    glEnd();
           
    // Now, allow drawing, except where the stencil pattern is 0x1
    // and do not make any further changes to the stencil buffer
    glStencilFunc(GL_NOTEQUAL, 0x1, 0x1);
    glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
       
    // Now draw red bouncing square
    // (x and y) are modified by a timer function
    glColor3f(0.0f, 1.0f, 0.0f);
    glRectf(x, y, x + rsize, y - rsize);
   
    // All done, do the buffer swap
    glutSwapBuffers();
    }

Can you explain me the stencil buffer by this example.

Now I do understand it better, if I read this Silvercord explanation, but I still don't understand it completely.

For example what does meen (what it does) GL_INCR and GL_KEEP in glStencilOp function. This was an example, what I don't understand, and I don't yet understand the glStencilFunc arguments too (what they mean or do)

Stencil buffer is commonly used for one type of shadowing technique. However it suffers from hard-edge shadows and no penumbra in the shadows. Shadow volumes use the stencil buffer but they also use other techniques to produce the final shadows.

Stencil buffers can be used, though, to mask off items such as mirrors, windows, etc.