How can I maximize the number of threads in my C++ application?

A program that creates many threads is generally restrained by the availability of virtual memory because the operating system allocates a stack for each new thread. The default size of this stack varies by operating system; on Windows and Linux, the default size is 1MB. This value is typically sufficient for most needs, but we clearly cannot expect it to be appropriate for every application. If a 32-bit program is creating hundreds or thousands of threads, it's going to hit the virtual memory limit quite soon if it accepts the default stack size.

The thread's stack is consumed by local variables, nested function calls, and recursion, therefore selecting a smaller stack size should only be done with careful consideration (and a lot of testing!). You, as the developer, are the best judge of a suitable size for the stack.

If you decide to change the stack size, you have several options. For example, many linkers allow you to specify a different default stack size, which affects every thread the program creates. You can also change the size more selectively by specifying it individually for each thread. The IceUtil::Thread  class allows you to supply a different size for the stack when calling the start method, as shown below:

IceUtil::ThreadPtr t = ...
t->start(32 * 1024); // 32K stack

In Ice applications, the Ice run time uses configuration properties to determine the stack size of the threads it creates. For example, the default client and server thread pools use the StackSize property. If you've configured an object adapter MyAdapter with its own thread pool, the stack size configuration property is named MyAdapter.ThreadPool.StackSize.

Finally, it's important to understand that it is rarely advantageous to create lots of threads. For compute-bound applications, it's best to limit the number of threads to the number of physical processors in the host machine; adding any more threads only increases context switches and reduces performance. Additional threads can improve responsiveness when threads can become blocked while waiting for the operating system to complete a task, such as a network or file operation. However, having an abundance of threads is not like having an abundance of beer: if performance is good with ten threads, it will not be as good with one thousand threads, as all of those threads are competing for processor time and will drag your system's performance to its knees.

See Also