Best solution for inter-thread communication?

[IceDane]

I am working on an application for the linux operating system, using posix threads, and this is completely new to me. I have searched high and low, and it seems there is no single recommendation for all inter-thread communication - I have a feeling this is because different scenarios call for different solutions.

My scenario is as follows: I have one main thread that starts two threads on start up that deal with reading and writing from/to the same socket, respectively.
The reader thread will read data from the socket when possible, and then relay it on to the main thread, which will parse it and call certain event handlers if the data received contained certain messages.(It's an IRC bot like I mentioned in my last post, and it supports modules in that the user can add modules to it that contain functions that are to be called when the wanted message is received from the server; e.g. module libmod.so is loaded, and the symbol privmsg_handler added to the event chain for PRIVMSG. Then it will be called every time a privmsg is received).

Herein lies my question: What is the best way to pass data between threads? My initial thought was pipes, but AFAIK, this is mostly used for IPC. Is there a reason I don't want to be using pipes for ITC?
Thing is, it would be optimal if I could come up with a solution that would allow the custom module functions provided by the user to read input directly, if they wanted. This would aid in the creation of things like file server(Stateful stuff? Wrong term?), where the first command might be !listfiles, and then it would be waiting for a filename after that.
AFAIK, if I used named pipes, I could accomplish this. Or do these pipes only two ends like in reality, and would opening it in a module thus close it in the main thread or something similar?

I got told about something called System V IPC, but as long as there's nothing bad about using pipes for ITC, I think I will stick with them.

I would really appreciate any input on this matter, even if you have a completely different method to accomplish this, as we are still in the design phase.

[MK27]

I've used "unix" (local) sockets to do IPC w/ linux, and that works great. They are sort of like named pipes but more versatile, methinks. They are not compatible with inet sockets but the setup is identical.

[Cactus_Hugger]

Certainly you could accomplish this with sockets/pipes, but it would seem hugely overkill to open a file descriptor just for this. Not to mention you're copying data. But...

They're threads. Each thread in a process can see all of the memory of other threads. If two threads hold a pointer to the same location in memory, then they can both access it. (Though, if a thread is writing to a spot in memory, then you must ensure that it is the only thread accessing that memory.)
Additionally, you can do other things, like tell a thread to "wake up" if it is waiting on you. If you're using pthreads, look up pthread_mutex_lock, pthread_mutex_unlock, pthread_cond_wait and pthread_cond_signal.

What you seem to have, at least between your reader and your main thread, is a producer-consumer relationship. You didn't seem to describe the relation between the main and the writer, but I'll assume it is much the same. You might look up producer-consumer, to get an understanding of what it is.

But: why not handle the message in the reader thread? Do you have to pass it to the main thread, and why?

Here's a 'basic' producer-consumer, in pthreads. (No guarenteed of correctness-this is off the top of my head. ^_^)

// Note: You should use better OO/encapsulation principles than I'm about to. ;-)
struct MessageQueue
{
    std::queue<std::string> msg_queue;
    pthread_mutex_t mu_queue;
    pthread_cond_t cond;
};

{
    // In a reader thread, far, far away...
    MessageQueue *mq = <a pointer to the same instance that the main thread has>;
    std::string msg = read_a_line_from_irc_or_whatever();
    pthread_mutex_lock(&mq->mu_queue);
    mq->msg_queue.push(msg);
    pthread_mutex_unlock(&mq->mu_queue);
    pthread_cond_signal(&mq->cond);
}

{
    // Main thread
    MessageQueue *mq = <a pointer to the same instance that the main thread has>;

    while(1)
    {
        pthread_mutex_lock(&mq->mu_queue);
        if(!mq->msg_queue.empty())
        {
            std::string s = mq->msg_queue.top();
            mq->msg_queue.pop();
            pthread_mutex_unlock(&mq->mu_queue);
            handle_that_string(s);
        }
        else
        {
            pthread_cond_wait(&mq->cond, &mq->mu_queue)
        }
    }

Edit: Most of this code should be encapsulated within the MessageQueue struct (which should then be a class)... really only needs two member funcs: push and pop.
Edit: You'll also need to initialize that pointer/mutex/condition, of course.

[IceDane]

Thank you for the replies.

I'm sorry I didn't explain the sender/writer thread. Basically, whenever anything needs to send something to the server, they would pass the data to the writer thread, by whatever means I end up using, and then signal the conditional variable it would be waiting on.

I understand what you are saying about the overkill - I realize they share address space, but the reason I was thinking about pipes is that I want to avoid global variables, and that was really the only way I came up with at the time, but I realize I can just pass it to the threads in a struct of some sort, that contains everything else the threads need to use.

Do you have any suggestions for implementing this taking into consideration that at some point in time, a function in a custom module might want to see the data received by the server?

Regarding why the reader doesn't just parse and handle the data, I guess that's fine, then the main thread would simply be waiting for the quit signal so it can subsequently tell the other threads to exit, then do all clean up.

Thanks again for the advice.

[edesign]

hey,

I am still a newbie to this stuff, but I'm working on similar kind of stuff for a few days and as far as I've captured the things msgq is the best way to do that...(it may not be ... it is what I've captured )

You can have a central msgq and reader/writer/controller post msgs to that and everybody will be waiting on the same Q for the msg... or another approach is to have separate Q for each thread and everybody will be waiting on its own Q for the msg...

-Edesign

[cyberfish]

The easiest way to do inter-threads communication is probably just using memory.

but I realize I can just pass it to the threads in a struct of some sort, that contains everything else the threads need to use.

That will do. Remember to use mutexes to protect your shared data.

IPC is a little more complicated because processes don't have shared memory. That's why they have to do it through OS facilities (like pipes).

[edit]or explicitly requesting shared memory from the OS.[/edit]

[CornedBee]

The easiest way to do inter-threads communication is probably just using memory.

I kind of disagree. Raw memory is too low-level to be the easiest way.

The easiest way in this case would be a pre-built inter-thread message queue, such as TBB's concurrent_queue template. It provides an extremely simple interface and handles all the gritty details.

[IceDane]

I kind of disagree. Raw memory is too low-level to be the easiest way.

The easiest way in this case would be a pre-built inter-thread message queue, such as TBB's concurrent_queue template. It provides an extremely simple interface and handles all the gritty details.

This would definitely be optimal, but unfortunately, this is being done in 100% raw C.

I guess learning C++ will be the next thing to do. The string class, as well as the STL will make things much easier. You should see the text parsing stuff in the bot - it's way too much code for too little functionality.