Small revisit of C++ message passing (in a threaded program)

The post C/C++ Message Passing from June ’08 is one or the more popular ones, if Google Analytics is to be believed. Not only that, but every one who arrived there by Google search was looking for C++, not C.

Upon re-reading it, it occurs to me that it would be a good candidate for being a lock free structure.  With C++0x, this should be pretty simple to do, using the new atomic template.  For straight C though, I would have to use compiler specific extensions (or inline assembly), so there may be little reason not to make a C++ specific version of the code that is lock free.

I’ve also thought of trying to use C++0x to construct a purely functional data structure library.  In that case, C++0x would be chosen because of the new shared_ptr addition.  I might have to have a post soon exploring that idea a bit further.

So far, I’ve converted this to be a C++ class and to use templates instead of void*s. Otherwise, it still behaves the same and still uses pthread locking directly. Additional versions will be forthcoming. Here is the recent work.

Hopefully it will be of use to some of the searchers. Keep an eye out for future revisions.

C/C++ Message Passing

Note: For C++ users, find an update at http://blog.jdboyd.net/2010/11/small-revisit-of-cc-message-passing-in-a-threaded-program/

My favorite approach to threaded programming is message passing, holding the shared memory for things that need it for performance reasons.

eCos has a handy general purpose message passing system, but C/C++ do not. Posix does have some methods that can be used for this. There are pipes or System V message queues. My problem with these is that they are meant for interprocess work, and thus have un-needed overhead for talking between threads. Also, those two methods are byte stream oriented.

What eCos has and what I wanted was the ability to pass pointers to a messages struct. Basically, a thread safe queue that pushes void* in and pops void* out, without invoking syscalls, which sap performance. I typically use this with a struct. Either all of the structs in the queue are the same type, or else in every struct used, the first item in the struct is an int indicating what struct to coerce the pointer to.

Anyway, here are the files:

Tested on Linux with GCC 3.2 and Solaris with Sun C++ 5.5. On Solaris, the test program for the queue returns two warnings, but functions fine. The test program is C++ for reasons that aren’t really relevant. This should be fixed at some point.

This is meant to be written to be acceptable for use in realtime systems. Obviously, if those systems don’t support posix threads, one will need to substitute the OS’s form of Mutex’s and Conds.