How can I fork and exec an Ice process?

Under Unix, if you have an Ice process (client or server) and want to create a new process, you will have to call fork and exec. The basic code pattern looks something like the following:

C++
pid_t pid = fork();
switch(pid)
{
    case -1:
    {
        throw "cannot fork";
    }
    case 0:
    {
        // Child

        // ...

        const char* exe = ...;
        const char** argv = ...;
        execv(exe, argv);
    }
    default:
    {
        // Parent
    }
}

This looks harmless enough but, unless you do things right, chances are that your process might hang, crash, or do something else unexpected. Here are a few simple rules to make sure things work as intended.

  1. Close open file descriptors before calling exec.
  2. Only call async-signal-safe system calls in the child.
  3. Do not call Ice-related functions in the child.
  4. If the parent uses asynchronous signal handlers, disable signal delivery before calling fork.
  5. If the parent uses Ice::Application  or Ice::CtrlCHandler, and the child process needs the default behavior for SIGHUP, SIGINT, and SIGTERM, reset these signals to their default behavior in the child before calling exec.
  6. If exec fails, call _exit.

Closing open file descriptors in the child is important because not doing so wastes kernel resources and can also interfere with the parent (for example, preventing connection closure when the parent closes a socket that is held open by the child).

Once fork has succeeded, the code must only call async-signal-safe system calls. (The Unix attributes(5) main page provides a list of these system calls.) Making any other system call can potentially crash the child.

You must not call Ice-related APIs in the child before calling exec. To understand why this is necessary, consider how fork works for a threaded process. In essence, fork duplicates the entire virtual memory image of the parent and arranges for fork to return zero in the child process. In addition, if the parent is threaded, the parent threads are not cloned in the child; instead, fork creates a single thread in the child (which is the thread that returns from the call). However, because the child has a memory image that is identical to that of the parent, any thread-related data structures will simply be in the state they were in when the parent called fork and the kernel made a snapshot of the parent's memory. Among other things, this means that mutexes may remain locked in the child, and data structures may be in an inconsistent state because other threads may have been inside a critical region at the time the parent called fork.

If you call any Ice-related function in the child before calling exec, things can go badly wrong because the function may attempt to lock a mutex that was already locked at the time the parent called fork, causing the child to deadlock. Similarly, the function might call a library function that is not async-signal-safe, causing the child to crash.

If your application installs signal handlers, you need to take extra care. After a fork, the child process has the same signal disposition as the parent: signals that are caught and handled by the parent are also caught and handled by the child. It is possible that a signal is delivered to the child before the child can call exec. In this case, if the parent handles the signal, so will the child. Depending on what the signal handler does, things can go badly wrong. For one, the signal handler cannot make system calls that are not async-signal-safe — doing so can crash either parent or child. But, even if the signal handler is async-signal-safe, it may have side-effects that are detrimental if the signal arrives in the child before the exec. If so, you need to block signal delivery before the parent calls fork, and unblock it again in the parent after fork returns.

If you use the Ice::Application or the IceUtil::CtrlCHandler helper classes to handle signals, there is no problem. The Ice run time does not install any signal handlers. Instead, the helper classes block delivery of SIGHUP, SIGINT, and SIGTERM and use a dedicated thread that calls sigwait to synchronously accept signals. In turn, this means that your callback functions (set with Application::callbackOnInterrupt or CtrlCHandler::setCallback) can safely call into the Ice run time, and can safely call functions that are not async-signal-safe. However, if you do use these helper classes and call fork, the child process will block SIGHUP, SIGINT, and SIGTERM. If you need the default behavior for these signals in the child, you need to unblock them before calling exec. (For more information, see Does Ice catch any signals?)

Finally, if the exec fails for any reason, you must call _exit (not exit). The difference between the two calls is that _exit terminates the process immediately and does not perform any clean-up actions (such as calling atexit handlers). In turn, this means that the destructors of C++ global and static objects are not called when you call _exit (whereas, if you call exit, they are called). Preventing destructors from running if exec fails is important because, if destructors were to run, they could fail because of the same inconsistent data structures that may be encountered by a signal handler. (Ice uses a few global objects internally, so this rule applies even if you do not have any global objects in your own code.)

So, here is an outline of the code needed to correctly fork and exec:

C++
// Set up a pipe so the child can report errors.
int fds[2];
if(pipe(fds) == -1)
{
    throw "cannot create pipe";
}

// Set close-on-exec on write end of pipe.
int flags = fcntl(fds[1], F_GETFD);
if(flags == -1)
{
    throw "cannot get fcntl flags";
}
flags |= FD_CLOEXEC;
if(fcntl(fds[1], F_SETFD, flags) == -1)
{
    throw "cannot set close-on-exec";
}

// If the parent uses signal handlers, block signal delivery here.

pid_t pid = fork();
switch(pid)
{
    case -1:
    {
        throw "cannot fork";
    }
    case 0:
    {
        // Child

        // If the parent uses Ice::Application or IceUtil::CtrlCHandler,
        // and the child requires the default behavior for SIGHUP, SIGINT,
        // and SIGTERM, reset these signals to the default behavior here.

        // Close all open file descriptors.
        int maxFd = static_cast<int>(sysconf(_SC_OPEN_MAX));
        for(int fd = 0; fd < maxFd; ++fd)
        {
            if(fd != fds[1]) // Don't close write end of pipe.
            {
                close(fd);
            }
        }
    
        const char* exe = ...;
        char* const argv[] = ...;
        execv(exe, argv);

        const char msg[] = "exec failed";
        write(fds[1], msg, sizeof(msg) - 1);
        _exit(1);
    }
    default:
    {
        // Parent

        // Close the write end of the pipe.
        close(fds[1]);

        // Wait for child to write error message or exec successfully.
        stringstream err;
        char c;
        while(read(fds[0], &c, 1) > 0)
        {
            err << c;
        }
        close(fds[0]);
        string msg = err.str();

        // If the parent uses signal handlers,
        // restore signal delivery here.

        if(!msg.empty())
        {
            throw msg;
        }
    }
}

Note that this code will most likely need fleshing out for your application. For example, it simply closes all file descriptors, including stdin, stdout, and stderr. It is likely that you will instead want to connect these descriptors to a file or terminal or, if you do not need them, re-open them to /dev/null. (Leaving the standard file descriptors closed is bad practice because third-party libraries sometimes fail if these descriptors do not work.) You may also want to perform additional actions, such as copying the parent's environment variables for the child, changing the working directory, setting the process group, or similar. For more details on how to do this, you can consult a Unix book such as Advanced Programming in the Unix Environment.

Also note that, if exec fails, the preceding code reports the error instead of having the child exit silently. A common way to implement this (and used by the preceding code) is to call pipe before forking to create a pipe between parent and child and to set the close-on-exec flag for the writing end of the pipe. The child writes to the pipe if something goes wrong, and the parent reads the error message from the pipe; the parent's call to read either succeeds and reads the error message or returns with an error if the child called exec successfully because, in that case, the kernel closes the writing end of the pipe.

See Also