Thread: Piping OUTPUT from 1 script, and using it as INPUT for a different script

  1. #1
    Registered User
    Join Date
    Jun 2011
    Posts
    12

    Piping OUTPUT from 1 script, and using it as INPUT for a different script

    Hello world

    I have a simple question, taking output of one file, and using it as input for another. I've been googling around for a long time, found lots of code but it doesn't do what I'm looking for, probably because I'm searching wrong.. I have 4 different scripts to simplify my program, but I need the output from them all.

    E.g. (this is not my real program, just a small example with same concept)

    ---------------- file1.c ------------------
    Code:
    int main(void) {
    char* charArray;
    fgets (charArray, 2, stdin); //get user chars input
    printf("%c", charArray);
    }

    ---------------- file2.c ------------------
    Code:
    int main(int argc,char *argv[]) { //this program takes input from command line
    printf ("%s", argv[1])
    }



    So yeah, that is quick example.
    Running file1.c, at fgets, input, you enter: abc .. And it prints it out on terminal
    Running file2.c, ./file2.o abc

    So the output of file1.o, and automatically use it in file2.o's input argument, without me having to do 2 separate executions on 2 files.

    So now I will explain my proper situation I'm in, (I've got all the code working for the inside of the file (there is a lot of code in each file so I need to keep them separate or I get confused like hell), however I want to automate all the executions instead of me manually typing 4 separate commands into CLI. So after waiting until a file has executed, then using the output of that file, as the arg (input) for the next file):

    ./file4.o > output.txt //copies entire OUTPUT block of text into file "output.txt"
    ./file3.o .. reads file "output.txt", and at end printf string
    ./file2.o .. reads file3.o OUTPUT (string), and printf another string,
    ./file1.o .. reads file2.o OUTPUT (string), and does what I need it to do


    P.s. in my current code, by OUTPUT I mean printf, but I'm guessing you guys will not printf the OUTPUT, instead pipe it into the next files args somehow

    P.s.s. I can't directly pipe file4.o OUTPUT into file3.o INPUT, as the output.txt it is very very big, and my file3.o is coded with using a text file.

    Sorry for the long winded question. I hope you understand it
    Last edited by astral; 06-15-2011 at 05:56 AM.

  2. #2
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    This is somewhat confusing as I think you are using the word "file" and "process" interchangeably.

    A file is something that is read or written to by a process.

    A process is something that is executed by another process (eg, when you start a program from the command line, the shell, a process, executes that process).

    You do not execute a file. You do not read or write a process.

    Maybe you want to look at exec() and popen()...
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  3. #3
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    ./file4.o > output.txt //copies entire OUTPUT block of text into file "output.txt"
    ./file3.o .. reads file "output.txt", and at end printf string
    ./file2.o .. reads file3.o OUTPUT (string), and printf another string,
    ./file1.o .. reads file2.o OUTPUT (string), and does what I need it to do

    So put these 4 lines into a text file and call it say doit.sh
    Make it executable, then just do

    ./doit.sh
    and sit back and enjoy.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  4. #4
    Registered User
    Join Date
    Jun 2011
    Posts
    12
    Quote Originally Posted by Salem View Post
    ./file4.o > output.txt //copies entire OUTPUT block of text into file "output.txt"
    ./file3.o .. reads file "output.txt", and at end printf string
    ./file2.o .. reads file3.o OUTPUT (string), and printf another string,
    ./file1.o .. reads file2.o OUTPUT (string), and does what I need it to do

    So put these 4 lines into a text file and call it say doit.sh
    Make it executable, then just do

    ./doit.sh
    and sit back and enjoy.
    I don't think you understand the question.

    That will not work, since each file won't receive the output, as a parameter for there input.

    The part you quoted, is what I haven't done and is what needs to completed.


    I will break it down even more with INPUT (command line argument), OUTPUT (needs to be fed into another file's argument)
    (The file2/file1 is where I mainly need help):

    ./file4.o > output.txt //(this creates a text file, with a lot of data in)
    INPUT:Nothing .. OUTPUT: output.txt

    ./file3.o //(searches entire output.txt file for a certain pattern)
    INPUT: output.txt .. OUTPUT: foundString

    ./file2.o foundString //(pattern is then decoded)
    INPUT:foundString .. OUTPUT:decodedString

    ./file1.o decodedString //(stuff..)
    INPUT:decodedString .. OUTPUT: (Final result)

    That is as simply as I can put it, I'm sure everyone can understand what I'm trying to do now. How do I pipe a files output into another files input?

    P.s. I tried exec, it runs the program from within a script, however all it does is run the program and printf the strings. I need those strings to be captured and placed in the next file's INPUT.
    Last edited by astral; 06-15-2011 at 07:54 AM.

  5. #5
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    So perhaps
    foundString=$(./file3.o output.txt)
    decodedString=$(./file2.o $foundString)
    ./file1.o $decodedString


    If you're using a shell other than bash, then replace $(command) with `command` (those are back-ticks if you're looking for them on the keyboard - they're seldom used).
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  6. #6
    Registered User
    Join Date
    May 2010
    Location
    Naypyidaw
    Posts
    1,314
    ./file3.o //(searches entire output.txt file for a certain pattern)
    INPUT: output.txt .. OUTPUT: foundString
    Stop using .o extension for executable. That makes me confused!
    If file3 can read from stdin, you could just use pipe.

    Code:
    prog4 | prog3
    I believe it's even possible to pipe to prog3 and at the same time redirect to file.(output.txt).

  7. #7
    Registered User ssharish2005's Avatar
    Join Date
    Sep 2005
    Location
    Cambridge, UK
    Posts
    1,732
    ./file4.o > output.txt //(this creates a text file, with a lot of data in)
    INPUT:Nothing .. OUTPUT: output.txt

    ./file3.o //(searches entire output.txt file for a certain pattern)
    INPUT: output.txt .. OUTPUT: foundString

    ./file2.o foundString //(pattern is then decoded)
    INPUT:foundString .. OUTPUT:decodedString

    ./file1.o decodedString //(stuff..)
    INPUT:decodedString .. OUTPUT: (Final result)
    From what you have explained I don’t really think, it good practice to do such thing so many different binaries. Rather i would just put them all in a project and call the appropriate functions (API's). Piping would be good idea, but quite not sure about performance tho, when it comes to rather piping a huge file. Of course piping like 'foundstring', 'decodestring' wouldn't be a big deal at all.

    An other alternative would be to use popen or 2 way pipes to exec the binaries you want to communicate between the processes.

    ssharish
    Life is like riding a bicycle. To keep your balance you must keep moving - Einstein

  8. #8
    Registered User
    Join Date
    May 2010
    Location
    Naypyidaw
    Posts
    1,314
    It's unix design.
    filter through pipe!

  9. #9
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,336
    I don't think we need to pipe prog4 to prog3, since prog3 reads from the file, right? I think your ./doit.sh file would just be
    Code:
    prog4
    prog3 | prog2 | prog1

  10. #10
    Registered User
    Join Date
    Jun 2011
    Posts
    12

    Thumbs up

    Done it guys, cheers for the help.

    Here's the single-lined code to do everything, which I ended up creating. Executes in around 1(s), awesome.

    This a "SHELL" command (any *NIX type OS will have a terminal or what-not), which I simply do by copying & pasting all of it into a "SHELL" and it executes everything in order
    Code:
    ./HTTPRequestSock.o GET > outputHTML; 
    ./search.o  > outputSTRING-S; 
    ./progDecodeX2-1KEY.o `cat outputSTRING-S`  > outputDECODED; 
    ./HTTPRequestSock.o POST `cat outputDECODED`
    Last edited by astral; 06-16-2011 at 05:13 AM. Reason: added description

  11. #11
    Registered User
    Join Date
    Apr 2011
    Posts
    308
    astral,

    Is that 1 second code you just posted c code or c++ or something else? Please add comments explaining that code.
    Last edited by jeremy duncan; 06-15-2011 at 06:17 PM.

  12. #12
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,336
    That is a command (i.e., the thing you type at a command-line prompt).

  13. #13
    Registered User
    Join Date
    Apr 2011
    Posts
    308
    I wish he would use C since I don't know command. Or is that command part of the compiling done in C?

  14. #14
    Registered User
    Join Date
    May 2009
    Posts
    4,183
    Quote Originally Posted by jeremy duncan View Post
    I wish he would use C since I don't know command. Or is that command part of the compiling done in C?
    This whole post was not really a C programming question; it was a question on how to invoke multiple C programs and use pipes and redirection during the command invokes.

    Tim S

  15. #15
    Banned
    Join Date
    Aug 2010
    Location
    Ontario Canada
    Posts
    9,547
    Quote Originally Posted by jeremy duncan View Post
    I wish he would use C since I don't know command. Or is that command part of the compiling done in C?
    You don't know how to use the DOS or Shell command lines?
    Best learn... it's an integral part of knowing what you're doing as a programmer.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. piping output input
    By Dr Saucie in forum C Programming
    Replies: 1
    Last Post: 04-20-2010, 10:03 PM
  2. Replies: 0
    Last Post: 02-05-2009, 06:59 PM
  3. Pausing script to read output
    By a_priebe47 in forum C Programming
    Replies: 1
    Last Post: 06-15-2004, 10:16 PM
  4. ~ User Input script help~
    By indy in forum C Programming
    Replies: 4
    Last Post: 12-02-2003, 06:01 AM
  5. Output from console-based script.
    By berto in forum Windows Programming
    Replies: 0
    Last Post: 05-26-2003, 03:06 PM

Tags for this Thread