Advice on inter-process communication

[erupter]

Hello.

As a project for my master thesis, I'm developing a multi-robot mapping system where the robots share data.
Now since this is all simulated, i run one simulator (player/stage under ubuntu 12.04) with multiple robots, and connect multiple processes (via sockets) to it to control each robot.

If it were a real situation with real robots, I could use a TCPIP networks and just use the local broadcast.
But given I'm confined to one machine, I need to devise a way to have multiple processes (each the same, I mean only one program executed multiple times with different command line parameters) communicate with each other.

A socket system would require each process to register a different socket, and a proxy process to be running and doing the mailing of the messages.

It could be a thesis by itself.
Are there any available solutions I could exploit to concentrate on the real matter of my work? (the mapping)
I am proficient with C, not that much with C++ although I'm familiar with OOP (been using .NET for quite some time).

[ledow]

Write one master program which spawns off X amount of threads (NOT processes), one for each child, and provides the intercommunication routines by reading/writing each thread's queue when asked?
Use shared memory to achieve the same effect with different processes?
Look into just about any other program that has to do this, e.g. Apache, Squid etc. and see how they do it?

Sorry, but for masters, I'd expect you to have already covered this, no matter what language. Hell, just a google on "shared memory" and any operating system name will provide you with resources in the first ten pages where it basically answers your question.

[erupter]

Write one master program which spawns off X amount of threads (NOT processes), one for each child, and provides the intercommunication routines by reading/writing each thread's queue when asked? Use shared memory to achieve the same effect with different processes?

The idea of a single executable that could be launched multiple times with different parameters was alluring.
This way I have to write a C++ program for sure (is thread spawning even doable with just C? I suspect the necessary library bindings for C don't exist) although it simplifies communication quite a bit.
I'm not familiar with the concept of shared memory though.

Look into just about any other program that has to do this, e.g. Apache, Squid etc. and see how they do it?

Are you meaning for me to go READ the source code of programs like Apache or Squid???
Are you serious? How much time do you anticipate it would take to begin to understand what does what in the Apace web server???

Sorry, but for masters, I'd expect you to have already covered this, no matter what language. Hell, just a google on "shared memory" and any operating system name will provide you with resources in the first ten pages where it basically answers your question.

Except that my first degree is in EE not in CS, and the advanced one is in Robotics. So I never covered such topics.
I kindly suggest you not to judge people you don't know.

[ledow]

- You can still do the single executable - shared memory. But threaded children is probably much more sensible and controllable.

- "Is thread-spawning even doable with just C" - Are you honestly serious? Where on earth would you get the idea that it's some magical C++-only feature? You're using it this second from C code, I guarantee you, especially if you're on Linux. Threading has *nothing* to do with the language involved, either. Any language can have threading functionality added (even if it's only a bodge) and any language can support it by default if it wanted to. Hell, there are FORTRAN standard libraries / compilers out there that support threading and all sorts of process interactions like shared memory. Look into POSIX (established UNIX-based standard).

- You don't need to read the source at all, I didn't say that. Just a tech document will do, or a design document on how they have multiple children and workers. E.g. Apache spawns child processes to handle connections, which can all talk to various bits of the other Apache processes, same as Squid does for its children and even things like Postfix do for email reception (Postfix has a SILLY number of extra processes for security design, apparently). And the lines of actual source code that spawn those children and let them intercommunicate are easily found with a quick jaunt through the code (hint: look for mention of IPC in the filenames, usually).

- Not to be insulting, but I don't particularly care what your degree is in. You're at Masters level. This stuff can be found with a quick Google or asking someone in the CS department. Masters is about finding things you've never covered before. And, again, I expect you to have already covered this anyway - I know the EE guys on my BSc courses did. It's what I would *expect* of a Masters course (so nothing personal against you, in that) and then being a Masters student (hell, being someone with access to Google who bothered to post in a forum) I'd expect you to be able to find some of this, with example code, in literally minutes.

[smokeyangel]

lol, aside from reading the Apache source...... that actually wouldn't be a bad idea if you actually wanted to design something. But you said:

Are there any available solutions I could exploit to concentrate on the real matter of my work? (the mapping)

Yes -- there almost certainly are libraries you can use. IPC libraries range from the simple to the insane, so try to have a good think about what you need to be able to do (having to switch libraries late in the project is not something you want to have to do....). Are the processes going to be broadcasting stuff that lots of other processes should receive? Or is it more exchanging messages between specific processes?

A socket system would require each process to register a different socket, and a proxy process to be running and doing the mailing of the messages.

It could be a thesis by itself.

For sure it's feasible for a competent programmer - but it'd not be a good thesis since lots of people have done it before .

You could start by googling for "linux ipc multicast". There are a couple of hits that look like they might be promising. I think D-BUS is meant to be fast and good but really complicated. On the other far end of the scale, there's KBUS which doesn't claim to be fast or efficient or featureful, but it probably really easy to learn to use. Kbus also has a C interface if you really prefer not to use C++. And everything in between.

Good luck

[erupter]

- "Is thread-spawning even doable with just C" - Are you honestly serious?

I'm not a linux programmer, hence I don't know what kind of support is provided for kernel libraries.
What if a library has only C++ headers?
It's wouldn't be the first.

- You don't need to read the source at all, I didn't say that. Just a tech document will do, or a design document on how they have multiple children and workers. E.g. Apache spawns child processes to handle connections, which can all talk to various bits of the other Apache processes, same as Squid does for its children and even things like Postfix do for email reception (Postfix has a SILLY number of extra processes for security design, apparently). And the lines of actual source code that spawn those children and let them intercommunicate are easily found with a quick jaunt through the code (hint: look for mention of IPC in the filenames, usually).

Ok I'll do.

- Not to be insulting, but I don't particularly care what your degree is in. You're at Masters level. This stuff can be found with a quick Google or asking someone in the CS department. Masters is about finding things you've never covered before. And, again, I expect you to have already covered this anyway - I know the EE guys on my BSc courses did. It's what I would *expect* of a Masters course (so nothing personal against you, in that) and then being a Masters student (hell, being someone with access to Google who bothered to post in a forum) I'd expect you to be able to find some of this, with example code, in literally minutes.

You are insulting as you are still assuming things.
If you strongly feel the need to complain about my education, you're free to address your complaint to the ministry of education.
Given that I'm perfectly on par with what has been required to grant me my titles, I don't need to justify why my 46 exams don't include nothing more than basic (c) programming.
I may possibly be wrong in calling my degree "Master".
To clarify it's the 2nd level degree you get after another 2yrs of study after the basic 3yrs.

lol, aside from reading the Apache source...... that actually wouldn't be a bad idea if you actually wanted to design something. But you said:

I know, but I'm a little bit short on time here

Yes -- there almost certainly are libraries you can use. IPC libraries range from the simple to the insane, so try to have a good think about what you need to be able to do (having to switch libraries late in the project is not something you want to have to do....). Are the processes going to be broadcasting stuff that lots of other processes should receive? Or is it more exchanging messages between specific processes?

I don't know as the idea is developing with me.
For starters I think about 200KBs per minute in a conceptual single message.
The exchange is just between my n processes, where n may vary from 1 to 5 tops.

For sure it's feasible for a competent programmer - but it'd not be a good thesis since lots of people have done it before .

You could start by googling for "linux ipc multicast". There are a couple of hits that look like they might be promising. I think D-BUS is meant to be fast and good but really complicated. On the other far end of the scale, there's KBUS which doesn't claim to be fast or efficient or featureful, but it probably really easy to learn to use. Kbus also has a C interface if you really prefer not to use C++. And everything in between.

Good luck

Thanks I'll have a look at your suggestions.

[Codeplug]

>> Are there any available solutions I could exploit to concentrate on the real matter of my work?
The Intelligent Transport Layer - zeromq
Read the guide. Fairly easy API that handles lots of different communications patterns.

gg

[brewbuck]

Since the "robots" are already all connected to the simulation server, why can't that be the broadcast mechanism? Just invent a new kind of message (or more than one) for "robot-to-robot" transmissions.

[erupter]

Since the "robots" are already all connected to the simulation server, why can't that be the broadcast mechanism? Just invent a new kind of message (or more than one) for "robot-to-robot" transmissions.

You mean using player's inner messaging system? I'm having a bit of problems with player itself, meaning they changed much of the APIs I was used to, and I'm still getting used to the new behaviour. Also to me the nested API system of Player has always given headache. May it's just me, but I find the language and usage a bit cumbersome. If there was a library I could load and use, it would be much better. I'll have a look anyway. Thanks for suggesting zeromq, it looks promising!