Best solution for inter-thread communication?

This is a discussion on Best solution for inter-thread communication? within the Linux Programming forums, part of the Platform Specific Boards category; I am working on an application for the linux operating system, using posix threads, and this is completely new to ...

  1. #1
    Ex scientia vera
    Join Date
    Sep 2007
    Posts
    478

    Best solution for inter-thread communication?

    I am working on an application for the linux operating system, using posix threads, and this is completely new to me. I have searched high and low, and it seems there is no single recommendation for all inter-thread communication - I have a feeling this is because different scenarios call for different solutions.

    My scenario is as follows: I have one main thread that starts two threads on start up that deal with reading and writing from/to the same socket, respectively.
    The reader thread will read data from the socket when possible, and then relay it on to the main thread, which will parse it and call certain event handlers if the data received contained certain messages.(It's an IRC bot like I mentioned in my last post, and it supports modules in that the user can add modules to it that contain functions that are to be called when the wanted message is received from the server; e.g. module libmod.so is loaded, and the symbol privmsg_handler added to the event chain for PRIVMSG. Then it will be called every time a privmsg is received).

    Herein lies my question: What is the best way to pass data between threads? My initial thought was pipes, but AFAIK, this is mostly used for IPC. Is there a reason I don't want to be using pipes for ITC?
    Thing is, it would be optimal if I could come up with a solution that would allow the custom module functions provided by the user to read input directly, if they wanted. This would aid in the creation of things like file server(Stateful stuff? Wrong term?), where the first command might be !listfiles, and then it would be waiting for a filename after that.
    AFAIK, if I used named pipes, I could accomplish this. Or do these pipes only two ends like in reality, and would opening it in a module thus close it in the main thread or something similar?

    I got told about something called System V IPC, but as long as there's nothing bad about using pipes for ITC, I think I will stick with them.

    I would really appreciate any input on this matter, even if you have a completely different method to accomplish this, as we are still in the design phase.
    "What's up, Doc?"
    "'Up' is a relative concept. It has no intrinsic value."

  2. #2
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    I've used "unix" (local) sockets to do IPC w/ linux, and that works great. They are sort of like named pipes but more versatile, methinks. They are not compatible with inet sockets but the setup is identical.
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  3. #3
    int x = *((int *) NULL); Cactus_Hugger's Avatar
    Join Date
    Jul 2003
    Location
    Banks of the River Styx
    Posts
    902
    Certainly you could accomplish this with sockets/pipes, but it would seem hugely overkill to open a file descriptor just for this. Not to mention you're copying data. But...

    They're threads. Each thread in a process can see all of the memory of other threads. If two threads hold a pointer to the same location in memory, then they can both access it. (Though, if a thread is writing to a spot in memory, then you must ensure that it is the only thread accessing that memory.)
    Additionally, you can do other things, like tell a thread to "wake up" if it is waiting on you. If you're using pthreads, look up pthread_mutex_lock, pthread_mutex_unlock, pthread_cond_wait and pthread_cond_signal.

    What you seem to have, at least between your reader and your main thread, is a producer-consumer relationship. You didn't seem to describe the relation between the main and the writer, but I'll assume it is much the same. You might look up producer-consumer, to get an understanding of what it is.

    But: why not handle the message in the reader thread? Do you have to pass it to the main thread, and why?

    Here's a 'basic' producer-consumer, in pthreads. (No guarenteed of correctness-this is off the top of my head. ^_^)
    Code:
    // Note: You should use better OO/encapsulation principles than I'm about to. ;-)
    struct MessageQueue
    {
        std::queue<std::string> msg_queue;
        pthread_mutex_t mu_queue;
        pthread_cond_t cond;
    };
    
    {
        // In a reader thread, far, far away...
        MessageQueue *mq = <a pointer to the same instance that the main thread has>;
        std::string msg = read_a_line_from_irc_or_whatever();
        pthread_mutex_lock(&mq->mu_queue);
        mq->msg_queue.push(msg);
        pthread_mutex_unlock(&mq->mu_queue);
        pthread_cond_signal(&mq->cond);
    }
    
    {
        // Main thread
        MessageQueue *mq = <a pointer to the same instance that the main thread has>;
    
        while(1)
        {
            pthread_mutex_lock(&mq->mu_queue);
            if(!mq->msg_queue.empty())
            {
                std::string s = mq->msg_queue.top();
                mq->msg_queue.pop();
                pthread_mutex_unlock(&mq->mu_queue);
                handle_that_string(s);
            }
            else
            {
                pthread_cond_wait(&mq->cond, &mq->mu_queue)
            }
        }
    Edit: Most of this code should be encapsulated within the MessageQueue struct (which should then be a class)... really only needs two member funcs: push and pop.
    Edit: You'll also need to initialize that pointer/mutex/condition, of course.
    Last edited by Cactus_Hugger; 05-24-2009 at 11:07 AM.
    long time; /* know C? */
    Unprecedented performance: Nothing ever ran this slow before.
    Any sufficiently advanced bug is indistinguishable from a feature.
    Real Programmers confuse Halloween and Christmas, because dec 25 == oct 31.
    The best way to accelerate an IBM is at 9.8 m/s/s.
    recursion (re - cur' - zhun) n. 1. (see recursion)

  4. #4
    Ex scientia vera
    Join Date
    Sep 2007
    Posts
    478
    Thank you for the replies.

    I'm sorry I didn't explain the sender/writer thread. Basically, whenever anything needs to send something to the server, they would pass the data to the writer thread, by whatever means I end up using, and then signal the conditional variable it would be waiting on.

    I understand what you are saying about the overkill - I realize they share address space, but the reason I was thinking about pipes is that I want to avoid global variables, and that was really the only way I came up with at the time, but I realize I can just pass it to the threads in a struct of some sort, that contains everything else the threads need to use.

    Do you have any suggestions for implementing this taking into consideration that at some point in time, a function in a custom module might want to see the data received by the server?

    Regarding why the reader doesn't just parse and handle the data, I guess that's fine, then the main thread would simply be waiting for the quit signal so it can subsequently tell the other threads to exit, then do all clean up.

    Thanks again for the advice.
    "What's up, Doc?"
    "'Up' is a relative concept. It has no intrinsic value."

  5. #5
    Registered User
    Join Date
    Feb 2008
    Posts
    146
    hey,

    I am still a newbie to this stuff, but I'm working on similar kind of stuff for a few days and as far as I've captured the things msgq is the best way to do that...(it may not be ... it is what I've captured )

    You can have a central msgq and reader/writer/controller post msgs to that and everybody will be waiting on the same Q for the msg... or another approach is to have separate Q for each thread and everybody will be waiting on its own Q for the msg...

    -Edesign

  6. #6
    Registered User
    Join Date
    Dec 2006
    Location
    Canada
    Posts
    3,146
    The easiest way to do inter-threads communication is probably just using memory.

    but I realize I can just pass it to the threads in a struct of some sort, that contains everything else the threads need to use.
    That will do. Remember to use mutexes to protect your shared data.

    IPC is a little more complicated because processes don't have shared memory. That's why they have to do it through OS facilities (like pipes).

    [edit]or explicitly requesting shared memory from the OS.[/edit]
    Last edited by cyberfish; 05-25-2009 at 03:59 AM.

  7. #7
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,892
    The easiest way to do inter-threads communication is probably just using memory.
    I kind of disagree. Raw memory is too low-level to be the easiest way.

    The easiest way in this case would be a pre-built inter-thread message queue, such as TBB's concurrent_queue template. It provides an extremely simple interface and handles all the gritty details.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  8. #8
    Ex scientia vera
    Join Date
    Sep 2007
    Posts
    478
    Quote Originally Posted by CornedBee View Post
    I kind of disagree. Raw memory is too low-level to be the easiest way.

    The easiest way in this case would be a pre-built inter-thread message queue, such as TBB's concurrent_queue template. It provides an extremely simple interface and handles all the gritty details.
    This would definitely be optimal, but unfortunately, this is being done in 100% raw C.

    I guess learning C++ will be the next thing to do. The string class, as well as the STL will make things much easier. You should see the text parsing stuff in the bot - it's way too much code for too little functionality.
    "What's up, Doc?"
    "'Up' is a relative concept. It has no intrinsic value."

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Thread Prog in C language (seg fault)
    By kumars in forum C Programming
    Replies: 22
    Last Post: 10-09-2008, 01:17 PM
  2. Shared memory implementation using thread
    By kumars in forum C Programming
    Replies: 5
    Last Post: 06-18-2008, 04:24 AM
  3. user thread library
    By Eran in forum C Programming
    Replies: 4
    Last Post: 06-17-2008, 01:44 AM
  4. accept() Hangs
    By Epo in forum Networking/Device Communication
    Replies: 14
    Last Post: 09-09-2005, 11:53 AM
  5. C++ Threading?
    By draggy in forum C++ Programming
    Replies: 5
    Last Post: 08-16-2005, 12:16 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21