Advanced Piping Question!

Hi

I am creating a program that can support piping between any number of processes. For 2 processes, you create 1 pipe, and use it between them.

However, for 3 or more pipes, this method will not work because of writing to a pipe while you still need the old one. Example:

http://img8.imageshack.us/img8/3145/pipesf.png

If you write to pipe[1] again (the red part), it will overwrite the previously written information, won't it?

What is the proper way to do this?

-------------------------

I have found that you can just create 1000 pipes or something like in the example below:

Code:

http://www.cs.loyola.edu/~jglenn/702/S2005/Examples/dup2.html

  int pipes[4];
  pipe(pipes); // sets up 1st pipe
  pipe(pipes + 2); // sets up 2nd pipe

  // we now have 4 fds:
  // pipes[0] = read end of cat->grep pipe (read by grep)
  // pipes[1] = write end of cat->grep pipe (written by cat)
  // pipes[2] = read end of grep->cut pipe (read by cut)
  // pipes[3] = write end of grep->cut pipe (written by grep)

However, since I need to be able to support ANY number of pipes, and not just 2 or 3, I do not want to do this.

Does anyone know how this can be achieved properly?

Can I do something like this?

Code:

int old_write_pipe = mypipes[1]; // save the old write pipe
int new_write_pipe; // new write pipe

mypipes[1] = new_write_pipe; // replace the old write pipe in the array with a new file descriptor

pipe(mypipes);

Anyone have any insight?

I'm not sure exactly what it is you're trying to do. You can only have communication between processes with the same parent for piping. How are you proposing to share these?


Quzah.

Originally Posted by quzah

I'm not sure exactly what it is you're trying to do. You can only have communication between processes with the same parent for piping. How are you proposing to share these?


Quzah.

Check out the picture I posted. I already have the case with 2 processes working. To make it simple, the problem is what do I put for the red text in the picture?

How is it you think one process needs unlimited pipes to it?


Quzah.

Originally Posted by quzah

How is it you think one process needs unlimited pipes to it?

Quzah.

Not unlimited obviously, but I need to run this in a loop. The simple example is something like ls | wc | grep but my professor will test with an undetermined amount of commands, so it has to be dynamic, not just ia set number like int pipe[10].

Ok, specific question: Say I have int mypipes[2], and then pipe(mypipes).

Fork() process 1 - write to mypipes[1]
Fork() process 2 - read from mypipes[0], write to mypipes[1]

Will the data that process 1 writes get overwritten when process 2 writes into the pipe?

Why don't you try it and find out?


Quzah.

Originally Posted by quzah

Why don't you try it and find out?


Quzah.

Unfortunately, I won't be able to test this for a while due to the way my class is set up.

A simple yes or no as to if it will overwrite would be much appreciated. I understand if you don't want to help.

You apparently have a computer, why don't you grab a compiler and try it? I'm not sure how you're thinking you're going to do your homework without compiling it ever.


Quzah.

Originally Posted by quzah

You apparently have a computer, why don't you grab a compiler and try it? I'm not sure how you're thinking you're going to do your homework without compiling it ever.


Quzah.

If you can direct me towards a compiler for the G1 phone, I surely would

Anyway, you're obviously getting upset by this thread for some odd reason. Please I would appreciate it if you would just stop posting and leave it to the people that can actually answer the question.

Why do you get the impression I'm angry? Very odd. You're posting on a programming board, with no actual way to program. That just doesn't make much sense.

"Hey guys, I want to paint a picture, but I have no brush! Can you tell me how to make a sunset?"


Quzah.

Originally Posted by quzah

Why do you get the impression I'm angry? Very odd. You're posting on a programming board, with no actual way to program. That just doesn't make much sense.

"Hey guys, I want to paint a picture, but I have no brush! Can you tell me how to make a sunset?"


Quzah.

I'm posting on my phone. I am not home. I can not program on my phone.

I don't think you're understanding piping correctly anyway. That's what I keep trying to tell you. You can only use pipes between processes which have the same parent. I think you're confusing the pipe function, with piping from a shell.

ls | wc | grep

Your program isn't going to handle that. The shell will. Unless you're writing a shell, I'm not seeing why you think you need a bunch of pipes.


Quzah.

Originally Posted by quzah

Unless you're writing a shell,

We are. That is the assignment. (:

You're going to have to call pipe() once for each "link" between the processes.

Either do it all beforehand :

Code:

pipes = malloc(2 * sizeof(int) * (num_processes-1))

for (i = 0; i < 2*(num_processes-1); i+=2)
    pipe(pipes + i);

pipes[n] would be the read pipe, pipes[n+1] would be the write pipe for each even n value. Each process is only going to write to one pipe and read from another - the rest it should close. Kind of a pain to keep track of, but closest to what your first example is doing.

or each time you fork off another process :

Code:

int pipes[2];

while (1)
{
    pipe(pipes);
    fork();
    if (child)
    {
        dup() read pipe 
        close unused pipe
        if no more children to create, exec()
    }
    else if parent
    {
       dup() write pipe 
       close unused pipe
       exec()
    }
}

This is just the basic 2-process case wrapped in a loop. It's also untested pseudo-code, but hopefully it's close to accurate.

The main thing to keep in mind is that each call to pipe() is going to ignore and overwrite what was in the arguments when you passed them in. Copy them to local vars or use dup()/dup2() before calling pipe() if you need to save the results of the previous pipe() call.

Originally Posted by KCfromNC

You're going to have to call pipe() once for each "link" between the processes.

Ahhhhhh, thanks! I was trying to use the same pipe[2] for multiple processes. After the second, it would always dump out to stdout.

Originally Posted by KCfromNC

The main thing to keep in mind is that each call to pipe() is going to ignore and overwrite what was in the arguments when you passed them in. Copy them to local vars or use dup()/dup2() before calling pipe() if you need to save the results of the previous pipe() call.

Oh, good idea!