C++ Message Passing (overview, between threads)

Having installed Google Analytics two months ago, I now know that the two most popular search phrases that people use to get here from Google are “elisp hex” and “c++ message passing”.  There isn’t much more to say about using hex in elisp, but I’m sure that there is more to say about C++ message passing.

The first thing that comes to mind is what does that phrase mean?  The obvious interpretations are passing messages between threads, passing messages between processes, passing messages on a network (local or Internet).

I previously mentioned that phrase in C/C++ Message Passing and Small Revisit of C++ Message Passing (in a Threaded Program) .  Both relate to passing messages between pthreads in a C++ program.  I rely heavily on that pattern because I prefer to try to treat my threads as shared nothing.  When possible, pass messages between threads instead of relying on shared data structures, and you will eliminate a lot of potential bugs.  If done correctly, it will also help eliminate bottle necks coming from lock contention.

In theory, I don’t like shared everything threading, but there are times when sharing things is required for performance, and there are times when parallization tools like OpenMP (or auto-parallelizers) are very valuable, and they required a shared every thing environment.

One of these days, I should try adjusting the implementation to use shared memory to enforce the seperation even more.  That, of course, would take us into messages between processes, which is a post still in progress.

Small revisit of C++ message passing (in a threaded program)

The post C/C++ Message Passing from June ’08 is one or the more popular ones, if Google Analytics is to be believed. Not only that, but every one who arrived there by Google search was looking for C++, not C.

Upon re-reading it, it occurs to me that it would be a good candidate for being a lock free structure.  With C++0x, this should be pretty simple to do, using the new atomic template.  For straight C though, I would have to use compiler specific extensions (or inline assembly), so there may be little reason not to make a C++ specific version of the code that is lock free.

I’ve also thought of trying to use C++0x to construct a purely functional data structure library.  In that case, C++0x would be chosen because of the new shared_ptr addition.  I might have to have a post soon exploring that idea a bit further.

So far, I’ve converted this to be a C++ class and to use templates instead of void*s. Otherwise, it still behaves the same and still uses pthread locking directly. Additional versions will be forthcoming. Here is the recent work.

Hopefully it will be of use to some of the searchers. Keep an eye out for future revisions.

C/C++ Message Passing

Note: For C++ users, find an update at http://blog.jdboyd.net/2010/11/small-revisit-of-cc-message-passing-in-a-threaded-program/

My favorite approach to threaded programming is message passing, holding the shared memory for things that need it for performance reasons.

eCos has a handy general purpose message passing system, but C/C++ do not. Posix does have some methods that can be used for this. There are pipes or System V message queues. My problem with these is that they are meant for interprocess work, and thus have un-needed overhead for talking between threads. Also, those two methods are byte stream oriented.

What eCos has and what I wanted was the ability to pass pointers to a messages struct. Basically, a thread safe queue that pushes void* in and pops void* out, without invoking syscalls, which sap performance. I typically use this with a struct. Either all of the structs in the queue are the same type, or else in every struct used, the first item in the struct is an int indicating what struct to coerce the pointer to.

Anyway, here are the files:

Tested on Linux with GCC 3.2 and Solaris with Sun C++ 5.5. On Solaris, the test program for the queue returns two warnings, but functions fine. The test program is C++ for reasons that aren’t really relevant. This should be fixed at some point.

This is meant to be written to be acceptable for use in realtime systems. Obviously, if those systems don’t support posix threads, one will need to substitute the OS’s form of Mutex’s and Conds.