SIGCHLD handler not executing dependably

Hi, I'm trying to figure out how to use a SIGCHLD handler to execute reliably when a child worker process exits for use in larger projects, but this simple test isn't working. Since sigchld++ is executed every time the sig handler is called, and it fork()'s twice, I would expect it to print 2 every time, but it doesn't. Usually it's 2, but maybe 1/5 times it prints 1.

Code:

---

#include <iostream>

#include <signal.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>

using namespace std;

int sigchld = 0;
void sigchld_handler(int signal){
    sigchld++;
}

int main(){
    signal(SIGCHLD, sigchld_handler);
    if(fork() & fork()){
        unsigned int leftover = 1;
        while((leftover = sleep(leftover))); // The sleep is to allow both children time to exit.
        cout << sigchld << endl;
    }
    return 0;
}

---

Here is your code, with some additional information.

Code:

---

#include <iostream>
 
#include <signal.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
 
using namespace std;
 
int sigchld = 0;
void sigchld_handler(int signal){
    write(1,"Boo\n",4);
    sigchld++;
}
 
int main(){
    signal(SIGCHLD, sigchld_handler);
    if(fork() & fork()){
        cout << "Me=" << getpid() << endl;
        unsigned int leftover = 1;
        while((leftover = sleep(leftover))); // The sleep is to allow both children time to exit.
        cout << sigchld << endl;
        int status;
        pid_t pid;
        while ( (pid=waitpid(-1,&status,WNOHANG)) != -1 ) {
            if ( pid != 0 ) {
                cout << "Reaped child=" << pid << " with exit status=" << WEXITSTATUS(status) << endl;
            }
        }
    } else {
        pid_t me = getpid();
        cout << "Child=" << me << " of parent=" << getppid() << endl;
        return me % 10;
    }
    return 0;
}

---

Here are some results.
[/code]
$ g++ foo.cpp
$ ./a.out
Me=2999
Child=3001 of parent=2999
Child=3000 of parent=2999
Child=3002 of parent=3000
Boo
Boo
2
Reaped child=3000 with exit status=0
Reaped child=3001 with exit status=1
$ ./a.out
Me=3003
Child=3005 of parent=3003
Child=3004 of parent=3003
Child=3006 of parent=3004
Boo
1
Reaped child=3004 with exit status=4
Reaped child=3005 with exit status=5
[/code]

Some things to note.
1. There are 3 children created, but one of them is a grandchild (not a child). This is because fork() & fork() is evaluated once in a child as well (but you have no signal handler set in the child).
2. Using waitpid() in a loop is a more reliable way of finding which children have exited.

What you need to know is that signals are NOT placed in a queue. If the signal arrives whilst in the middle of a signal handler, it is lost.

Quote:

---

Originally Posted by Salem

Some things to note.
1. There are 3 children created, but one of them is a grandchild (not a child). This is because fork() & fork() is evaluated once in a child as well (but you have no signal handler set in the child).
2. Using waitpid() in a loop is a more reliable way of finding which children have exited.

What you need to know is that signals are NOT placed in a queue. If the signal arrives whilst in the middle of a signal handler, it is lost.

---

1. Yes, I meant to use && and take advantage of logical short circuiting.

3. What?! That is stupid beyond belief. Why would they want to limit signals to something less that built-in hardware (PIC) on IBM compatible machines even in the 80's?

That explains it. Thanks.

Quote:

---

Originally Posted by User Name:

What?! That is stupid beyond belief. Why would they want to limit signals to something less that built-in hardware (PIC) on IBM compatible machines even in the 80's?

---

Backwards compatibility. In Linux and most POSIX-like systems you can use "realtime" signals (SIGRTMIN to SIGRTMAX), in Linux about 30 available real-time signals. They are almost like the old signals, except these are queued, and will be caught in the order (from kernel's viewpoint) they were raised in. You can even use sigqueue() and pthread_sigqueue() to include a single int or a pointer as payload or message with the signal.

However, the thing I want to point out, is that waitpid() is an async-signal-safe function; you can call it inside a signal handler. Therefore, your signal handler should really be

Code:

---

volatile int  sigchld = 0;

void sigchld_handler(int signal)
{
    int saved_errno;
    pid_t result;

    /* Save errno, so that the thread we interrupted does
    * not suddenly experience errno from changing under them. */
    saved_errno = errno;

    while (1) {
        result = waitpid((pid_t)-1, NULL, WNOHANG);
        if (result == (pid_t)0) {
            /* Children exist, just have not exited yet */
            break;
        } else
        if (result == (pid_t)-1) {
            if (errno == EINTR)
                continue; /* Interrupted by another signal */

            /* errno must == ECHILD, no children left.
              Let's be defensive and handle all other errno codes,
              too, even if they don't/shouldn't ever occur.
            */
            break;
        }

        /* (pid_t) referred to a positive pid, a child that has died. */
        sigchld++;
    }

    /* Because errno is a per-thread variable, and no sane program
    * tries to get its address, only the original thread this signal interrupted
    * could possibly see errno. Because we saved the original value
    * after the original thread was interrupted, and restore it here,
    * no-one will ever notice any change in it.
    * Even if this signal handler took ages and ages to complete. */
    errno = saved_errno;
}

---

There is also a very small race window: a child process may exit after the last check, but before the signal handler has exited, so last child (or childs if they all exit within that same time window) might stay unreaped. It (they) will be reaped when the next child exits, though.

To avoid that race, you should add a loop just before reporting the exited child process count, which makes sure all children have exited and have been reaped. (The waitpid(-1,..,WNOHANG) call will return 0 if there are children that have not yet exited. With the WNOHANG flag, the call never blocks (waits for a child to exit), it always returns immediately.)

Also note that since sigchld is an int, you can only safely modify it in the signal handler. (Or in main, if you have no other threads and you block/disable the signal handler for the duration of the modifications.)

If you want to run the same loop in your main() or some other function, you really should use sig_atomic_t sigchld;. Unfortunately, it is limited to range 0..127 only in portable code. It's usually an alias for int, but sometimes just an 8-bit signed char.

I personally prefer compiler-provided atomic built-in functions (__sync...() built-ins for now, __atomic_...() when compilers catch up), to access the counter; this also allows accessing and modifying it from multiple threads concurrently. It's not really standardized in C, but these functions tend to be available in most C compilers anyway (at least GCC, Intel CC, Pathscale, and Portland Group had them, the last time I checked). (Although.. mixing threads and child processes correctly is a separate, complex issue -- not too difficult to get right, but there are some common stumbling blocks. Just to warn you.)

I'd recommend something like (untested):

Code:

---

pid_t  p;

while (1) {
    p = waitpid((pid_t)-1, NULL, 0);
    if (p == (pid_t)-1) {
        if (errno == EINTR)
            continue;
        /* It must be errno == ECHILD, no children left. */
        break;
    } else
    if (p == (pid_t)0)
        continue; /* Should never happen (since we did not use WNOHANG flag),
                    but let's be careful anyway. */

    /* It was a positive child process ID we just reaped. */
    sigchld++;
}

---

just before reporting the number of child processes that have exited.