1

Unix pipe problem: how to process multiline input to output the line and insert some piped processes.

echo "111 22222222" | awk '{print $1 " " $2 " " piped_processes_using_2nd_argument}'

A bit of context: netstat parsed to get the list of external ip adresses connected to $current_ip, sorted decreasingly, for example (not real case):

$ netstat -plant |grep $current_ip | tr -s ' ' |cut -f5 -d' '|cut -f1 -d':' |sort -n |uniq -c |sort -nr
   7 8.8.8.8
   2 4.4.4.4
   1 186.2.168.190
   1 207.192.128.2

So far, reverse DNS using

$ host 4.4.4.4 | tr -s " " | rev  | cut -f1 -d " " |rev
alu7750testscr.xyz1.gblx.mgmt.Level3.net.

From these, I want to get something like (yes, keeping the ip in the output):

   7 8.8.8.8 google-public-dns-a.google.com.
   2 4.4.4.4 alu7750testscr.xyz1.gblx.mgmt.Level3.net.
   1 186.2.168.190 ddos-guard.net.
   1 207.192.128.2 www.NexQloud.com.

To merge these, I could only think of awk, but as I am not fluent with it, the result is very ugly:

netstat -plant |grep $current_ip | tr -s ' ' |cut -f5 -d' '|cut -f1 -d':' |sort -n |uniq -c |sort -nr  | awk '{ system("echo " $1 " " $2 " $(host "$2" | tr -s \" \" | rev  | cut -f1 -d \" \" |rev)") }' | sort -nr -k1

I want something more elegant and fast enough to get called in a watch --interval=1 :-)

  • 1
    This looks like one of those situations where I would switch from a huge pipeline to a while/read loop. That way you can assign intermediate variables and output the final fields/variables that you're interested in. – Jeff Schaller Jul 15 '15 at 12:40
1

you like | do you ?

the best I come with

current_ip=$(ifconfig eth0 | awk '$1 == "inet" { split($2,A,":") ; print A[2] ; } ')

netstat -plant | grep $current_ip | awk '{split($5,A,":") ; howmany[A[1]]++ ; }
END { for (h in howmany) printf "%d %s \n",howmany[h],h ;} '|
 sort -nr |
while read hm ho
do
  name=$(host $ho|awk '{print $NF}')
  echo $hm $ho $name
done

where

  • no need to escape new line, if last char is a pipe
  • {split($5,A,":") ; howmany[A[1]]++ ; } remember each host and count (after stripping port number)
  • END { for (h in howmany) printf "%d %s \n",howmany[h],h ;} at the end of netstat, print result
  • name=$(host $ho|awk '{print $NF}') gethostname from IP
  • use ${name%%.} to get rid of ending .
  • add current_ip

not sure my solution is better any way, I only have 4 pipe instead of 12.

  • you use bash ? that looks like host is given no argument. – Archemar Jul 16 '15 at 7:48
  • I assure you this version works ; running on bash 4.3.33 ; echo $SHELL gives /bin/bash – nicolallias Jul 16 '15 at 8:19
  • in my case, eval current_ip=$(ifconfig eth0 | grep inet | grep -v inet6 | awk '{print $2}' |cut -f2 -d":") – nicolallias Jul 16 '15 at 8:21
  • 1
    I edited my answer. less pipe, more awk. – Archemar Jul 16 '15 at 8:37
  • excellent! lacks some pipes though, but easier to maintain, thank you very much! – nicolallias Jul 16 '15 at 10:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.