Like Tree3Likes

Awk and sed bash script help

This is a discussion on Awk and sed bash script help within the Linux Programming forums, part of the Platform Specific Boards category; Thanks Anduril for a push in the right direction for the numerical comparison. Got it working. It's fully functional and ...

  1. #16
    Registered User Annonymous's Avatar
    Join Date
    Apr 2011
    Location
    Jackson, New Jersey, United States
    Posts
    302
    Thanks Anduril for a push in the right direction for the numerical comparison. Got it working. It's fully functional and running.

    Final code:
    Code:
    #!/bin/bash
    
    netstat -ant | awk -F" " '{print $5}' | awk -F":" '{print $2}' | sed '/^$/d' | grep -E "0|1|2|3|4|5|6|7|8|9" > netstat.txt;
    
    awk '!x[$0]++' netstat.txt > netstat2.txt && cat netstat2.txt | sed '/^$/d' netstat2.txt;
    LINES=$(awk 'END {print NR}' netstat2.txt) 
    NLINES=`echo $LINES`;
    
    THREE=3;
    
    if [ $NLINES -lt $THREE ]; then
    echo "Your safe"
    
    	else 
    	echo "[danger]";	
    	/path/to/ui_executable;
    
    fi

  2. #17
    - - - - - - - - oogabooga's Avatar
    Join Date
    Jan 2008
    Posts
    2,808
    You don't need the sed calls to remove empty lines since the grep will do that (since it only matches lines with at least one digit). And you don't need the files, either. Also, the variables THREE and NLINES are both unnecessary. If the idea with THREE was to name a "magic number" then it really doesn't accomplish much! Maybe LIMIT, but I think a plain 3 is best.
    Code:
    #!/bin/bash
    
    LINES=$(
        netstat -ant           |
        awk '{print $5}'       |
        awk -F":" '{print $2}' |
        grep -E "[0-9]"        |
        awk '!x[$0]++'         |
        awk 'END {print NR}'   )
    
    if [ $LINES -lt 3 ]
    then
        echo "You're safe"
    else
        echo "[danger]"
        /path/to/ui_executable
    fi
    It's still pretty inefficient, 6 program invocations (4 awks!).

    A possibility that doesn't use awk is this (but I don't know what the cut columns should be since I'm not running Linux at the moment; so you'll have to adjust the 50-70 to cut the correct columns).
    Code:
    LINES=$(
        netstat -ant       |
        cut -c 50-70       |
        grep -Eo ":[0-9]+" |
        sort -u            |
        wc -l              )
    The cost of software maintenance increases with the square of the programmer's creativity. - Robert D. Bliss

  3. #18
    Registered User Annonymous's Avatar
    Join Date
    Apr 2011
    Location
    Jackson, New Jersey, United States
    Posts
    302
    Quote Originally Posted by oogabooga View Post
    You don't need the sed calls to remove empty lines since the grep will do that (since it only matches lines with at least one digit). And you don't need the files, either. Also, the variables THREE and NLINES are both unnecessary. If the idea with THREE was to name a "magic number" then it really doesn't accomplish much! Maybe LIMIT, but I think a plain 3 is best.
    Code:
    #!/bin/bash
    
    LINES=$(
        netstat -ant           |
        awk '{print $5}'       |
        awk -F":" '{print $2}' |
        grep -E "[0-9]"        |
        awk '!x[$0]++'         |
        awk 'END {print NR}'   )
    
    if [ $LINES -lt 3 ]
    then
        echo "You're safe"
    else
        echo "[danger]"
        /path/to/ui_executable
    fi
    It's still pretty inefficient, 6 program invocations (4 awks!).

    A possibility that doesn't use awk is this (but I don't know what the cut columns should be since I'm not running Linux at the moment; so you'll have to adjust the 50-70 to cut the correct columns).
    Code:
    LINES=$(
        netstat -ant       |
        cut -c 50-70       |
        grep -Eo ":[0-9]+" |
        sort -u            |
        wc -l              )

    Yeah I see that now. I removed the sed command, and removed the field separator in the first awk command.

    I am taking a different approach too. I was thinking, what if the internet something other than 443 and 80 are running. For instance, 443 and 1337? That fits the safe criteria of 2 lines. Though, it is actually quite the opposite! So, instead of searching for the number of lines, I used an inverted egrep search to parse out 443 and 80. Then if there are any remaining ports, launch the warning UI.

    Code:
    #!/bin/bash
    
    netstat -ant | awk '{print $5}' | awk -F":" '{print $2}' | awk '!x[$0]++' | grep -E "[0-9]" > netstat.txt; egrep -v "^(443|80)" netstat.txt > netstat2.txt;
    
    echo "[OPEN PORTS]"
    
    LINES=$(awk 'END {print NR}' netstat2.txt) 
    
    THREE=1;
    
    if [ $LINES -lt $THREE ]; then
    cat netstat.txt;
    exit 0;
    
    	else 
    	cat netstat.txt;
    	sudo /path/to/ui_executable
    
    fi
    I also removed the unecissary $NLINES variable as well.

    I WOULD LIKE TO ADD MORE TO THE GUI! Maybe a button to turn it off. Lets say I am downloading a torrent or using an ftp site; I wouldn't want an annoying pop up every minute. So I would make a button to act as a kill switch! What do you think?
    Last edited by Annonymous; 05-09-2012 at 09:37 PM.

  4. #19
    Registered User
    Join Date
    Nov 2010
    Location
    Long Beach, CA
    Posts
    5,451
    I like the idea of adding to the GUI and the functionality overall, but it's going to get complicated, at least if you want it to be reasonably useful. Since this is to alert you of unexpected/unauthorized network usage, you wont want to shut it all the way off while downloading a big torrent, which may take hours. Thus you would only want to ignore traffic to that port for that process. But then, what if that process forks for some reason (and ends up with a new pid). I understand this is just for personal use, but I'd say it's worth thinking it through a bit and coming up with a solid plan before doing much more coding. It will get much harder to do via a simple shell script as you add functionality, think about alternatives. Consider too checking more often than 60 seconds. I can send a lot of malicious data in under 60 seconds, that you wont detect.

    Here's a list of some common stuff that I could think of that you might not want to flag.

    • Torrents
    • Streaming music/media
    • SSH/SCP
    • SFTP, NFS
    • SMTP/POP
    • NetBIOS/Samba if you connect to Windows shares with your Linux machine
    • Instant message
    • VMWare
    • Video games

  5. #20
    Registered User Annonymous's Avatar
    Join Date
    Apr 2011
    Location
    Jackson, New Jersey, United States
    Posts
    302
    Quote Originally Posted by anduril462 View Post
    I like the idea of adding to the GUI and the functionality overall, but it's going to get complicated, at least if you want it to be reasonably useful. Since this is to alert you of unexpected/unauthorized network usage, you wont want to shut it all the way off while downloading a big torrent, which may take hours.
    Yeah, that makes sense. See, my reasoning was that if I were to download a movie from thepiratebay and have all kinds of high random ports open; i would be aware of this and not want a pesky pop up annoying the crap out of me every minute!

    Reading the very valid points you have made, have changed my outlook completely. I did just add a kill button using the execl function. Which i am going to keep.

    Quote Originally Posted by anduril462 View Post
    Thus you would only want to ignore traffic to that port for that process. But then, what if that process forks for some reason (and ends up with a new pid). I understand this is just for personal use, but I'd say it's worth thinking it through a bit and coming up with a solid plan before doing much more coding. It will get much harder to do via a simple shell script as you add functionality, think about alternatives. Consider too checking more often than 60 seconds. I can send a lot of malicious data in under 60 seconds, that you wont detect.
    You are absolutely right. I need to think it through if i am going to add anything useful!

    I actually wanted to add a great deal of functionality though. Like for starters, the ability to manually enter the ports that i want to flag or something along those lines. As of the moment, I am only monitoring 2 ports; 443 and 80. Which is minimal. So, that's just one idea for starters. I can definitely see the bash script becoming an issue in the future. Probably quite messy! There has to be a better way to do it. Python maybe??


    Quote Originally Posted by anduril462 View Post
    Here's a list of some common stuff that I could think of that you might not want to flag.
    • Torrents
    • Streaming music/media
    • SSH/SCP
    • SFTP, NFS
    • SMTP/POP
    • NetBIOS/Samba if you connect to Windows shares with your Linux machine
    • Instant message
    • VMWare
    • Video games
    Those are great things to flag. All to common exploitable services. Thanks!
    Last edited by Annonymous; 05-10-2012 at 12:45 AM.

Page 2 of 2 FirstFirst 12
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Help converting bash shell script to windows
    By chelp123 in forum Tech Board
    Replies: 2
    Last Post: 11-24-2011, 01:22 PM
  2. ssh/bash script question
    By Overworked_PhD in forum Tech Board
    Replies: 2
    Last Post: 03-30-2009, 07:48 PM
  3. Bash Script Q
    By QuestionC in forum Tech Board
    Replies: 1
    Last Post: 04-19-2007, 10:16 AM
  4. Linux: Use C to call a bash script
    By harada in forum Linux Programming
    Replies: 9
    Last Post: 10-27-2006, 01:59 PM

Tags for this Thread


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21