Pipes question

This is a discussion on Pipes question within the C Programming forums, part of the General Programming Boards category; I am trying to write a C application, that utilizes a perl script. the Perl script i have reads from ...

  1. #1
    Registered User
    Join Date
    Dec 2006
    Posts
    7

    Pipes question

    I am trying to write a C application, that utilizes a perl script.

    the Perl script i have reads from standard input, modifies the input, and outputs the modified version to stdout.

    here is a code sample, it will do a lot more than this, but im just trying to get it to work before i really get into it.

    Code:
    while (<STDIN>) {
    
    chomp;
    my $name = $_;
    print "here is some modified output ".$name."\n";
    
    }
    What i would ultimately like to do, is have that perl script always running as a subprocess, using execv(), or similar. that way, whenever i need to modify a string in my C program, i can simply pipe it to that perl script, and somehow retrieve the output.

    Iv read a ton about two way pipes, but it seems like they all deal with programs that are fully written in C and can utilize both sides of the pipe. I learned about dup2() in college, but cant seem to get it to work right for whatever reason.. So far, iv successfully been able to pipe a string to the perl script, have it modify it, and print it to stdout like it should. my problem, is that i need to read the scripts output from the C program, and iv had a hell of a time trying to do that.

    Here is what iv come up with so far, and i just doesnt work very well. sorry for the messy code, this is only temporary and wasnt planning on making it public..

    Code:
    int readpipe[2];
    int writepipe[2];
    
    #define	PARENT_READ	readpipe[0]
    #define	CHILD_WRITE	   readpipe[1]
    #define CHILD_READ	    writepipe[0]
    #define PARENT_WRITE	writepipe[1]
    
      
       if (pipe (readpipe)){
               fprintf (stderr, "Pipe failed.\n");
               return EXIT_FAILURE;
        }
           
       if (pipe (writepipe)){
               fprintf (stderr, "Pipe failed.\n");
               return EXIT_FAILURE;
       }
        
       if ( (pid = fork()) < 0){
    	printf("could not create subprocess!\n");
       }
       else if ( pid == 0 )	{
    		
    	/* this is the child */
    
    	close(PARENT_READ);
    	close(PARENT_WRITE);
    	
    	dup2(CHILD_READ,  0);  		 close(CHILD_READ);
    	dup2(CHILD_WRITE, 1);		close(CHILD_WRITE); 
    
    	execv("./dirmap.pl", argv);    /* start the perl script
        }
       else{
    
    	/* this is the parent *
    	
    	char string[100];
    	char *newstring;
    	int nbytes = 100;
    	newstring = (char *) malloc (nbytes + 1);
    	int rc;
    	
    	close(CHILD_READ);
    	close(CHILD_WRITE);
    	
    	sprintf(string, "%s", "here is a test string");
    
    	while( rc = write(PARENT_WRITE, string, strlen(string)) ){
    
       /* EVERYTHING WORKS UP TO THIS POINT, BUT TRYING TO READ IN THE OUTPUT  IS GIVING ME TROUBLE */
    
    		getline(&newstring, &nbytes, PARENT_READ);
    		fprintf(stderr, "FROM C: %s\n", newstring);
             }
        }
    If someone could provide a logic outline of the best way to do this, or a code sample, i would be forever greatful.

    and i cant "just do the string modification in the C program" because i will be utilizing some pre-made perl modules .

    thanks!
    Last edited by LostShootinStar; 12-07-2006 at 08:21 PM.

  2. #2
    Registered User
    Join Date
    Nov 2006
    Posts
    176
    you're going about it the right way
    just let the child know when its time

    heres an exaple I wrote up....it takes input, feeds input into child echo with stdin, then prints output from stdout with parent

    Code:
    #include <stdio.h>
    #include <stdlib.h>
    #include <unistd.h>
    #include <string.h>
    
    void child_handler(int input_pipe[], int output_pipe[])
    {
            int ret_val;
            char buffer;
    
            /* close unneeded */
            close(input_pipe[1]);
            close(output_pipe[0]);
    
            ret_val = read(input_pipe[0], &buffer, 1);
    
            if (ret_val <= 0)
            {
                    perror("child read");
                    close(input_pipe[0]);
                    close(output_pipe[1]);
                    exit(1);
            }
    
            if (buffer != '-')
            {
                    printf("unknown data\n");
                    close(input_pipe[0]);
                    close(output_pipe[1]);
                    exit(1);
            }
    
            dup2(input_pipe[0],  0);
            dup2(output_pipe[1], 1);
    
            ret_val = execl("/bin/echo", (char *)0);
    
            if (ret_val == -1)
            {
                    perror("execl");
                    close(input_pipe[0]);
                    close(output_pipe[1]);
                    exit(1);
            }
    }
    
    void parent(int input_pipe[], int output_pipe[])
    {
            int ret_val;
            char buffer[32];
    
            close(input_pipe[1]);
            close(output_pipe[0]);
    
            printf("Enter line to echo (starting with - for valid input)\n>");
    
            fgets(buffer, sizeof(buffer), stdin);
    
            ret_val = write(output_pipe[1], &buffer, sizeof(buffer));
    
            if (ret_val <= 0)
            {
                    perror("parent write");
                    close(input_pipe[0]);
                    close(output_pipe[1]);
                    exit(1);
            }
    
            ret_val = read(input_pipe[0], &buffer, sizeof(buffer));
    
            if (ret_val <= 0)
            {
                    perror("parent read");
                    close(input_pipe[0]);
                    close(output_pipe[1]);
                    exit(1);
            }
    
            printf("Echo Back: %s", buffer);
    }
    
    int main(void)
    {
            int parent_to_child[2];
            int child_to_parent[2];
            int pid;
            int ret_val;
    
            ret_val = pipe(parent_to_child);
            if (ret_val == -1)
            {
                    perror("PIPE p2c");
                    exit(1);
            }
    
            ret_val = pipe(child_to_parent);
            if (ret_val == -1)
            {
                    perror("PIPE p2c");
                    exit(1);
            }
    
            pid = fork();
    
            switch (pid)
            {
                    case -1: perror("fork"); exit(1);
                    case 0: child_handler(parent_to_child, child_to_parent);
                    default: parent(child_to_parent, parent_to_child);
            }
    
            return 0;
    }
    output:
    Enter line to echo (starting with - for valid input)
    >invalid input
    unknown data
    parent read: Success

    2:
    Enter line to echo (starting with - for valid input)
    >-valid input
    Echo Back:
    valid input


    so basically changing echo to your script and using loops (carefull no deadlock) to stay open and take multiple lines of data

    ::edit:: woops forgot to close the pipes in functions, just visualize that I did
    Last edited by sl4nted; 12-07-2006 at 10:09 PM.

  3. #3
    Registered User
    Join Date
    Dec 2006
    Posts
    7
    thank you very much!

  4. #4
    Registered User
    Join Date
    Dec 2006
    Posts
    7
    I ALMOST have this working...it looks like my C program is blocking at the read() function.

    Code:
    ret_val = read(PARENT_READ, buffer, sizeof(buffer) + 1) ;
    
    /* WONT GET PAST THIS POINT */
    		printf("parent has read\n");
    
            if (ret_val <= 0)
            {
                    perror("parent read");
                    close(PARENT_READ);
                    close(PARENT_WRITE);
                    exit(1);
            }
    
            printf("Echo Back: %s", buffer);

    this is my perl script
    Code:
    while (<STDIN>) {
    chomp;
    print  "here is some modified output ".$_."\n";
    
    
    /*IF I PUT AN EXIT STATEMENT HERE, EVERYTHING WORKS */
    }
    I noticed if i put an 'exit' statement inside that while loop, everything works perfect, except for the fact that the script exits. without the 'exit' statement, the C program blocks at read(); i would really like to find a way to keep the script running, to avoid the overhead of creating a process every time i need to use it (this program will be handling 10,000+ file names that need to be modified.

    please help!
    Last edited by LostShootinStar; 12-08-2006 at 10:34 PM.

  5. #5
    Registered User
    Join Date
    Nov 2006
    Posts
    176
    I'm gonna need to see more, it looks like you've changed it a bit. If it's blocking there, its extremely likly that nothing is being writen into the pipe it is trying to read from. So it sits and waits to read. so maybe you have your pipes mixed up.

    ::edit

    ahhh I see now...it blocks on a 2nd pass.....If you are still using execl to start your perl script, that may be the problem...don't quote me on this but I think the exec function is transfering your scripts image with the childs image, this means, your script will run through..but the child doesn't go on beyond the exec. so it never produces 2nd output, and never writes that output to the pipe, so the parent never reads the non-ouput.
    if that makes any sense to you. exec functions may not be the way to go. but then again I may be totally wrong about this copying the process image over.

    ::edit edit
    nope I'm right, man execl, and I thaught I forgot most everything I learned in uni, that I don't still use
    Last edited by sl4nted; 12-08-2006 at 10:49 PM.

  6. #6
    Registered User
    Join Date
    Dec 2006
    Posts
    7
    here is the whole code.

    I KNOW for a fact that the perl script is both getting the input from the C program, and writing it back to the pipe the correct way. i know that, because when i do put the exit statement in the perls while loop, everything in the C program works perfectly. so its only when the perl script doesnt exit that the read() function blocks. which tells me that the read() function is missing some kind of EOF marker or something similar...i just cant figure it out.

    I think your code worked well because the 'echo' was exiting after it output, while this perl script is not.
    here is the code

    Code:
    GLOBAL int readpipe[2];
    GLOBAL int writepipe[2];
    
    #define	PARENT_READ		readpipe[0]
    #define	CHILD_WRITE		readpipe[1]
    #define 	CHILD_READ		writepipe[0]
    #define 	PARENT_WRITE	writepipe[1]
    
    void child_handler()
    {
            int ret_val;
            char buffer;
    
            /* close unneeded */
            close(PARENT_READ);
            close(PARENT_WRITE);
    
            dup2(CHILD_READ,  0);
            dup2(CHILD_WRITE, 1);
    
            ret_val = execl("./dirmap.pl", (char *)0);
    	
            if (ret_val == -1)
            {
                    perror("execl");
                    close(CHILD_READ);
                    close(CHILD_WRITE);
                    exit(1);
            }
    }
    
    void parent(char * dir_string){
    
            int ret_val;
            char buffer[100];
            
            sprintf(buffer,"%s\n", dir_string);
    
    		printf("Parent is writing %s\n", buffer);
            ret_val = write(PARENT_WRITE, &buffer, sizeof(buffer));
            
            if (ret_val <= 0)
            {
                    perror("parent write error\n");
                    close(PARENT_READ);
                    close(PARENT_WRITE);
                    exit(1);
            }
    		
    		printf("parent is reading\n");
    		
            ret_val = read(PARENT_READ, buffer, sizeof(buffer) + 1) ;
    
    		printf("parent has read\n");
    
            if (ret_val <= 0)
            {
                    perror("parent read");
                    close(PARENT_READ);
                    close(PARENT_WRITE);
                    exit(1);
            }
    
            printf("Echo Back: %s", buffer);
    }
    
    if (pipe (readpipe)){
    		fprintf (stderr, "Pipe failed.\n");
    		return EXIT_FAILURE;
    	}
    	
    	if (pipe (writepipe)){
    		fprintf (stderr, "Pipe failed.\n");
    		return EXIT_FAILURE;
    	}
    	
    	/* pipes started */
    	    
    	/* fork the child process for dirmap.pl */
    	pid = fork();
    	
    	if( pid < 0){
    		printf("Could not create dirmap subprocess!\n");
    		return EXIT_FAILURE;
    	}
    	else if ( pid == 0 )	/* in the child */
    	{
    		
    		child_handler();
    	}
    	else{
    	
            close(CHILD_READ);
            close(CHILD_WRITE);
            
    	//char *dp = "This is a test\n";
        create_archive ();
        name_close ();
        	parent("This is a test");
        	sleep(1);
        	parent("This is another test");
     }
    Last edited by LostShootinStar; 12-08-2006 at 10:51 PM.

  7. #7
    Registered User
    Join Date
    Nov 2006
    Posts
    176
    I donno, got me stumped....maybe someone else can figure it out, or I'll take a look again in the morning with a fresh mind.
    I was thinking though...if you want it on all the time to do this. why not change the perl script and make it a server that just sits there and reads input from a socket and writes back.
    then the c program just loops connects to the server when it needs to, writes data to the socket, then reads the reply

  8. #8
    Registered User
    Join Date
    Dec 2006
    Posts
    7
    Quote Originally Posted by sl4nted
    I donno, got me stumped....maybe someone else can figure it out, or I'll take a look again in the morning with a fresh mind.
    I was thinking though...if you want it on all the time to do this. why not change the perl script and make it a server that just sits there and reads input from a socket and writes back.
    then the c program just loops connects to the server when it needs to, writes data to the socket, then reads the reply
    not a bad idea...ill give it a shot. i have a decent amount of experience with C sockets, but not with perl. hopefully they are easy to connect.

  9. #9
    Registered User
    Join Date
    Nov 2006
    Posts
    176
    k, I had to change my first example a bit...since your script is to stay on. I've made a little program that echos

    Code:
    #include <stdio.h>
    
    int main(void)
    {
            char buffer[50];
    
            while (1)
            {
                    fgets(buffer, sizeof(buffer), stdin);
                    printf("%s", buffer);
                    fflush(stdout);
            }
    
            return 0;
    }
    which is run with the main program

    Code:
    #include <stdio.h>
    #include <stdlib.h>
    #include <unistd.h>
    #include <string.h>
    
    #define BUFFSIZE 256
    
    void pipe_error(int input, int output, const char *msg)
    {
            perror(msg);
            close(input);
            close(output);
            exit(1);
    }
    
    void child_handler(int input_pipe[], int output_pipe[])
    {
            /* close unneeded */
            close(input_pipe[1]);
            close(output_pipe[0]);
    
            /* change stdin, stdout */
            dup2(input_pipe[0],  0);
            dup2(output_pipe[1], 1);
    
            close(input_pipe[0]);
            close(output_pipe[1]);
    
            execl("PATH TO ECHO GOES HERE", "myecho", (char *)0);
    
            exit(0);
    }
    
    void parent(int input_pipe[], int output_pipe[])
    {
            int ret_val;
            char buffer[BUFFSIZE], *ptr;
    
            close(input_pipe[1]);
            close(output_pipe[0]);
    
            while (1)
            {
                    bzero(buffer, BUFFSIZE);
    
                    printf("Enter line to echo\n>");
                    fgets(buffer, sizeof(buffer), stdin);
    
                    ret_val = write(output_pipe[1], &buffer, strlen(buffer));
                    if (ret_val <= 0)
                            pipe_error(input_pipe[0], output_pipe[1], "parent write");
    
                    ret_val = read(input_pipe[0], &buffer, sizeof(buffer));
                    if (ret_val <= 0)
                            pipe_error(input_pipe[0], output_pipe[1], "parent read");
    
                    printf("Got Back: %s\n", buffer);
    
            }
    
            close(input_pipe[0]);
            close(output_pipe[1]);
            exit(0);
    }
    
    int main(void)
    {
            int parent_to_child[2];
            int child_to_parent[2];
            int pid;
            int ret_val;
    
            ret_val = pipe(parent_to_child);
            if (ret_val == -1)
            {
                    perror("PIPE p2c");
                    exit(1);
            }
    
            ret_val = pipe(child_to_parent);
            if (ret_val == -1)
            {
                    perror("PIPE c2p");
                    exit(1);
            }
    
            pid = fork();
    
            switch (pid)
            {
                    case -1: perror("fork"); exit(1);
                    case 0: child_handler(parent_to_child, child_to_parent);
                    default: parent(child_to_parent, parent_to_child);
            }
    
            return 0;
    }
    works fine from what I can see...and is nearly exactly what your attempting

    output:

    Enter line to echo
    >first line
    Got Back: first line

    Enter line to echo
    >second line
    Got Back: second line

    Enter line to echo
    >here's a third line
    Got Back: here's a third line

    Enter line to echo
    >seems to be working
    Got Back: seems to be working

    Enter line to echo
    >done testing
    Got Back: done testing

    Enter line to echo
    >no way to exit
    Got Back: no way to exit

    Enter line to echo
    >ctrl+c
    Got Back: ctrl+c

    Enter line to echo
    >

    this should help I hope

  10. #10
    Registered User
    Join Date
    Dec 2006
    Posts
    7
    That works wonderfully, except that when i replace the echo program with the perl script, it does the exact same thing

    so i guess its some kind of perl problem. It only works when the script exits. ill try and ask on a perl forum...

    thank you very much for the help thus far..

  11. #11
    Frequently Quite Prolix dwks's Avatar
    Join Date
    Apr 2005
    Location
    Canada
    Posts
    8,046
    The input to your Perl script is likely buffered. This means that the output from your C program is queued until enough of it has been received, so that that larger chunk of data can be sent instead of a bunch of smaller chunks (larger chunks are more efficient). So your Perl script is waiting for data, even though your C program has already supplied the data.

    The solution to this is to get the C program to flush the stream, possibly with fflush(). Supplying an EOF flushes the buffer automatically, but ends the perl script.

    I may have misread your question but I think that's what's happening.

    BTW, there's a good perl forum on DaniWeb (see my signature).
    dwk

    Seek and ye shall find. quaere et invenies.

    "Simplicity does not precede complexity, but follows it." -- Alan Perlis
    "Testing can only prove the presence of bugs, not their absence." -- Edsger Dijkstra
    "The only real mistake is the one from which we learn nothing." -- John Powell


    Other boards: DaniWeb, TPS
    Unofficial Wiki FAQ: cpwiki.sf.net

    My website: http://dwks.theprogrammingsite.com/
    Projects: codeform, xuni, atlantis, nort, etc.

  12. #12
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,434
    You could try putting
    Code:
    $| = 1;
    In your perl script to make it flush the output buffer every time it does a write.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  13. #13
    Registered User
    Join Date
    Dec 2006
    Posts
    7
    Quote Originally Posted by Salem
    You could try putting
    Code:
    $| = 1;
    In your perl script to make it flush the output buffer every time it does a write.
    That did the trick. thank you!

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Pipes sometimes fail
    By EVOEx in forum Linux Programming
    Replies: 2
    Last Post: 05-02-2009, 01:47 PM
  2. Alice....
    By Lurker in forum A Brief History of Cprogramming.com
    Replies: 16
    Last Post: 06-20-2005, 02:51 PM
  3. Question...
    By TechWins in forum A Brief History of Cprogramming.com
    Replies: 16
    Last Post: 07-28-2003, 09:47 PM
  4. Services and Pipes
    By nickname_changed in forum Windows Programming
    Replies: 0
    Last Post: 07-16-2003, 06:46 AM
  5. Question, question!
    By oskilian in forum A Brief History of Cprogramming.com
    Replies: 5
    Last Post: 12-24-2001, 12:47 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21