delete line from text file

This is a discussion on delete line from text file within the Linux Programming forums, part of the Platform Specific Boards category; I'm working on doing some basic file sorting on a web server running unix. I'm basically trying to keep track ...

  1. #1
    PC Fixer-Upper Waldo2k2's Avatar
    Join Date
    May 2002
    Posts
    2,001

    delete line from text file

    I'm working on doing some basic file sorting on a web server running unix. I'm basically trying to keep track of a few basic things (hack attempts, what ip addresses connect and how often, how much they download, etc.) However, after sorting everything, I want to delete lines containing a certain ip address and/or the - character. However I just cannot figure out what command to use to accomplish this, here's what I have (assuming all log files are named access.log.x where x is a number):
    Code:
            cut -d ' ' -f1 access*>tf2
    	cut -d ' ' -f10 access*>tf1
    	echo "Bandwidth usage by file size:">usage.rpt
    	paste -d ' ' tf1 tf2 | sort -n -r  >>usage.rpt
                                             ^command to delete needed here
    basically the first field from the apache log files is the ip address, and the tenth field is either the size downloaded in bytes or in case of a 404 error a - character. After pasting and sorting by file size in descending order I need to chop out those lines. I have everything accomplished up until deleting those lines, at which point im lost. Any suggestions?? Thank you very much.
    PHP and XML
    Let's talk about SAX

  2. #2
    End Of Line Hammer's Avatar
    Join Date
    Apr 2002
    Posts
    6,231
    grep -v text_i_dont_want
    like so:
    Code:
    cut -d ' ' -f"1 10" access.log.* | grep -v "-" | sort -nr
    When all else fails, read the instructions.
    If you're posting code, use code tags: [code] /* insert code here */ [/code]

  3. #3
    PC Fixer-Upper Waldo2k2's Avatar
    Join Date
    May 2002
    Posts
    2,001
    thanks a bunch, I' used grep for some other things...but I guess I missed the -v in the man pages. Thanks again, I've really been beating my head against the wall on this one.
    PHP and XML
    Let's talk about SAX

  4. #4
    End Of Line Hammer's Avatar
    Join Date
    Apr 2002
    Posts
    6,231
    If you're into processing text files like this, you'll probably want to invest some time in awk.

    Here's a simple example. There may well be better ways to do what I've done, I am no awk expert!

    Code:
    >cat awk.cmds
    
    BEGIN { 
      print "My report..." 
    }
    
    #
    # No test here, this action happens for every line
    #
    {
      iplist[$1] += $10;
    }
    
    #
    # Numeric check on field 10,
    # Record bytes uploaded
    #
    ($10+0 == $10)  { 
      TotalBytes += $10; 
      printf ("IP: %-16s Bytes: %-10d\n", $1, $10);
    }
    
    #
    # Test for -, meaning no bytes uploaded
    #
    $10 == "-" {
      FailedRequests++;
    }
    
    #
    # This happens at the end of the run
    #
    END { 
      printf ("Total bytes processed: %d\n", TotalBytes); 
      printf ("Total failed requests: %d\n", FailedRequests); 
      printf ("\nIP Summary List Follows:\n");
      
      for ( i in iplist ) 
      {
        printf ("IP: %-16s TotalBytes: %-10d\n", i, iplist[i]);
      }
    }
    
    >awk -f awk.cmds access.log.*
    ....
    .... output here... run it and see....
    When all else fails, read the instructions.
    If you're posting code, use code tags: [code] /* insert code here */ [/code]

  5. #5
    PC Fixer-Upper Waldo2k2's Avatar
    Join Date
    May 2002
    Posts
    2,001
    I'm definetly going to use awk next time, that way I'll have some more control over all this crap. Thanks again.
    PHP and XML
    Let's talk about SAX

  6. #6
    Crazy Fool Perspective's Avatar
    Join Date
    Jan 2003
    Location
    Canada
    Posts
    2,640
    >>If you're into processing text files like this, you'll probably want to invest some time in awk.

    and sed... very handy for quick in-line content replacing among other things.

    Code:
    sed --in-place 's/old-text/new-text/' <file>

  7. #7
    Frequently Quite Prolix dwks's Avatar
    Join Date
    Apr 2005
    Location
    Canada
    Posts
    8,047
    I don't know about awk, but Perl is very handy for that kind of thing.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Inventory records
    By jsbeckton in forum C Programming
    Replies: 23
    Last Post: 06-28-2007, 04:14 AM
  2. reading a text file printing line number
    By bazzano in forum C Programming
    Replies: 4
    Last Post: 09-16-2005, 10:31 AM
  3. Possible circular definition with singleton objects
    By techrolla in forum C++ Programming
    Replies: 3
    Last Post: 12-26-2004, 09:46 AM
  4. simulate Grep command in Unix using C
    By laxmi in forum C Programming
    Replies: 6
    Last Post: 05-10-2002, 04:10 PM
  5. Validating the contents of a char buffer
    By mattz in forum C Programming
    Replies: 3
    Last Post: 12-09-2001, 05:21 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21