Advanced Piping Question!

This is a discussion on Advanced Piping Question! within the C Programming forums, part of the General Programming Boards category; Hi I am creating a program that can support piping between any number of processes. For 2 processes, you create ...

  1. #1
    Registered User
    Join Date
    Apr 2008
    Posts
    167

    Advanced Piping Question!

    Hi

    I am creating a program that can support piping between any number of processes. For 2 processes, you create 1 pipe, and use it between them.

    However, for 3 or more pipes, this method will not work because of writing to a pipe while you still need the old one. Example:

    http://img8.imageshack.us/img8/3145/pipesf.png

    If you write to pipe[1] again (the red part), it will overwrite the previously written information, won't it?

    What is the proper way to do this?

    -------------------------

    I have found that you can just create 1000 pipes or something like in the example below:

    Code:
    http://www.cs.loyola.edu/~jglenn/702/S2005/Examples/dup2.html
    
      int pipes[4];
      pipe(pipes); // sets up 1st pipe
      pipe(pipes + 2); // sets up 2nd pipe
    
      // we now have 4 fds:
      // pipes[0] = read end of cat->grep pipe (read by grep)
      // pipes[1] = write end of cat->grep pipe (written by cat)
      // pipes[2] = read end of grep->cut pipe (read by cut)
      // pipes[3] = write end of grep->cut pipe (written by grep)
    However, since I need to be able to support ANY number of pipes, and not just 2 or 3, I do not want to do this.

    Does anyone know how this can be achieved properly?

    Can I do something like this?

    Code:
    int old_write_pipe = mypipes[1]; // save the old write pipe
    int new_write_pipe; // new write pipe
    
    mypipes[1] = new_write_pipe; // replace the old write pipe in the array with a new file descriptor
    
    pipe(mypipes);
    Anyone have any insight?

  2. #2
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    I'm not sure exactly what it is you're trying to do. You can only have communication between processes with the same parent for piping. How are you proposing to share these?


    Quzah.
    Hope is the first step on the road to disappointment.

  3. #3
    Registered User
    Join Date
    Apr 2008
    Posts
    167
    Quote Originally Posted by quzah View Post
    I'm not sure exactly what it is you're trying to do. You can only have communication between processes with the same parent for piping. How are you proposing to share these?


    Quzah.
    Check out the picture I posted. I already have the case with 2 processes working. To make it simple, the problem is what do I put for the red text in the picture?

  4. #4
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    How is it you think one process needs unlimited pipes to it?


    Quzah.
    Hope is the first step on the road to disappointment.

  5. #5
    Registered User
    Join Date
    Apr 2008
    Posts
    167
    Quote Originally Posted by quzah View Post
    How is it you think one process needs unlimited pipes to it?

    Quzah.
    Not unlimited obviously, but I need to run this in a loop. The simple example is something like ls | wc | grep but my professor will test with an undetermined amount of commands, so it has to be dynamic, not just ia set number like int pipe[10].

    Ok, specific question: Say I have int mypipes[2], and then pipe(mypipes).

    Fork() process 1 - write to mypipes[1]
    Fork() process 2 - read from mypipes[0], write to mypipes[1]

    Will the data that process 1 writes get overwritten when process 2 writes into the pipe?

  6. #6
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    Why don't you try it and find out?


    Quzah.
    Hope is the first step on the road to disappointment.

  7. #7
    Registered User
    Join Date
    Apr 2008
    Posts
    167
    Quote Originally Posted by quzah View Post
    Why don't you try it and find out?


    Quzah.
    Unfortunately, I won't be able to test this for a while due to the way my class is set up.

    A simple yes or no as to if it will overwrite would be much appreciated. I understand if you don't want to help.

  8. #8
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    You apparently have a computer, why don't you grab a compiler and try it? I'm not sure how you're thinking you're going to do your homework without compiling it ever.


    Quzah.
    Hope is the first step on the road to disappointment.

  9. #9
    Registered User
    Join Date
    Apr 2008
    Posts
    167
    Quote Originally Posted by quzah View Post
    You apparently have a computer, why don't you grab a compiler and try it? I'm not sure how you're thinking you're going to do your homework without compiling it ever.


    Quzah.
    If you can direct me towards a compiler for the G1 phone, I surely would

    Anyway, you're obviously getting upset by this thread for some odd reason. Please I would appreciate it if you would just stop posting and leave it to the people that can actually answer the question.

  10. #10
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    Why do you get the impression I'm angry? Very odd. You're posting on a programming board, with no actual way to program. That just doesn't make much sense.

    "Hey guys, I want to paint a picture, but I have no brush! Can you tell me how to make a sunset?"


    Quzah.
    Hope is the first step on the road to disappointment.

  11. #11
    Registered User
    Join Date
    Apr 2008
    Posts
    167
    Quote Originally Posted by quzah View Post
    Why do you get the impression I'm angry? Very odd. You're posting on a programming board, with no actual way to program. That just doesn't make much sense.

    "Hey guys, I want to paint a picture, but I have no brush! Can you tell me how to make a sunset?"


    Quzah.
    I'm posting on my phone. I am not home. I can not program on my phone.

  12. #12
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    I don't think you're understanding piping correctly anyway. That's what I keep trying to tell you. You can only use pipes between processes which have the same parent. I think you're confusing the pipe function, with piping from a shell.
    ls | wc | grep
    Your program isn't going to handle that. The shell will. Unless you're writing a shell, I'm not seeing why you think you need a bunch of pipes.


    Quzah.
    Hope is the first step on the road to disappointment.

  13. #13
    Registered User
    Join Date
    Apr 2008
    Posts
    167
    Quote Originally Posted by quzah View Post
    Unless you're writing a shell,
    We are. That is the assignment. (:

  14. #14
    Registered User
    Join Date
    Mar 2009
    Posts
    344
    You're going to have to call pipe() once for each "link" between the processes.

    Either do it all beforehand :

    Code:
    pipes = malloc(2 * sizeof(int) * (num_processes-1))
    
    for (i = 0; i < 2*(num_processes-1); i+=2)
        pipe(pipes + i);
    pipes[n] would be the read pipe, pipes[n+1] would be the write pipe for each even n value. Each process is only going to write to one pipe and read from another - the rest it should close. Kind of a pain to keep track of, but closest to what your first example is doing.

    or each time you fork off another process :

    Code:
    int pipes[2];
    
    while (1)
    {
        pipe(pipes);
        fork();
        if (child)
        {
            dup() read pipe 
            close unused pipe
            if no more children to create, exec()
        }
        else if parent
        {
           dup() write pipe 
           close unused pipe
           exec()
        }
    }
    This is just the basic 2-process case wrapped in a loop. It's also untested pseudo-code, but hopefully it's close to accurate.

    The main thing to keep in mind is that each call to pipe() is going to ignore and overwrite what was in the arguments when you passed them in. Copy them to local vars or use dup()/dup2() before calling pipe() if you need to save the results of the previous pipe() call.
    Last edited by KCfromNC; 04-17-2009 at 09:26 AM.

  15. #15
    Registered User
    Join Date
    Apr 2008
    Posts
    167
    Quote Originally Posted by KCfromNC View Post
    You're going to have to call pipe() once for each "link" between the processes.
    Ahhhhhh, thanks! I was trying to use the same pipe[2] for multiple processes. After the second, it would always dump out to stdout.

    Quote Originally Posted by KCfromNC View Post
    The main thing to keep in mind is that each call to pipe() is going to ignore and overwrite what was in the arguments when you passed them in. Copy them to local vars or use dup()/dup2() before calling pipe() if you need to save the results of the previous pipe() call.
    Oh, good idea!

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Question about pointers #2
    By maxhavoc in forum C++ Programming
    Replies: 28
    Last Post: 06-21-2004, 12:52 PM
  2. Advanced C Question
    By Vinod Menon in forum C Programming
    Replies: 11
    Last Post: 05-28-2004, 08:43 AM
  3. Advanced but yet general
    By Rhodium in forum C Programming
    Replies: 6
    Last Post: 08-09-2003, 12:46 PM
  4. Question...
    By TechWins in forum A Brief History of Cprogramming.com
    Replies: 16
    Last Post: 07-28-2003, 09:47 PM
  5. opengl DC question
    By SAMSAM in forum Game Programming
    Replies: 6
    Last Post: 02-26-2003, 08:22 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21