I have the following code which I'm running to extract some ip addresses and count the unique of them. For example

./my_program | awk '/^10./{a[$1 FS $2]++ } END { for(i in a) print i,a[i]}' > some_file

The output that some_file contains is this i.e: 34 12 5

Now what I want is run {system("dig +short -x " i)} and append the out to the last column so the file then looks like 34  server1.rdns.domain.tld 12  server2.rdns.domain.tld 5   server3.rdns.domain.tld

Can I do this in one go and then pipe the output to a text file?



... END { for(i in a) cmd=sprintf("dig +short -x \"%s\"", $i); cmd | getline type; close(cmd);  printf("%s %s %s\n",i,a[i],type);

This is what ends in the output (stderr)

dig: '.in-addr.arpa.' is not a legal name (unexpected end of input)

This is what I see in the file 34


    END { for(i in a) split(i,ip," ")
    cmd=sprintf("dig +short -x \"%s\"", ip[1]); cmd | getline type; close(cmd);
    printf("%s %s %s %s\n",ip[1],ip[2],a[i],type)
  • 1
  • @muru thanks for the comment. got a step closer but not entirely. I get now the following. See update – holasz Aug 7 at 9:20
  • @muru sorry for confusing you. I didn't post the whole correct code but I now updated my question. I am using an array as follow a[$1 FS $2]++ and when run ./my_program the IP's are returned in this form: 34 where 34 in this case is a port number. and the 3rd field is the count. – holasz Aug 7 at 9:35
  • So you need to split i to get the IP, instead of using $i. gnu.org/software/gawk/manual/… Or save the IP in another array. – muru Aug 7 at 9:38
  • @muru thanks for the link that really helped. So we narrowed the problem now. One remaining thig is that it is now showing just 1 line instead of many which was the case earlier. See update2 – holasz Aug 7 at 9:49

You can use split() function to split the IP/port combination and get the IP for using later in the dig() command.

END { 
  for(i in a) {
    split(i, ip)
    cmd=sprintf("dig +short -x \"%s\"", ip[1])
    (cmd | getline type) > 0 ? "" : type="notResolved"
    printf("%s %s %s\n",i,a[i],type);

I've added a way to check the status of dig command and used the ? ternary operator to update the value of type. For cases of failure you can add your own string (e.g 'notResolved' in the above) and print the result.

Do note that using the getline() call in awk is not the same as using system() call. The type cmd | getline var makes use of pipes as if the cmd is run from the shell each call to getline var reads one line of output from the command. The system() command though provides no way to get the output of the command executed inside. You only have access to the exit code returned from the command ran inside.

P.S. Answer partially adopted from useful comments of muru.

  • 1
    Great. Thanks Inian. THis works a treat. Well that's what makes this community so awesome! Sharing pieces of code and making it work! Cheers. – holasz Aug 7 at 10:09
  • 1
    type = ( (cmd | getline line) > 0 ? line : "notResolved" ) would be clearer. Took me a minute to figure out what the existing code was doing and then another to figure out if/how it'd actually work. – Ed Morton Aug 7 at 15:36

You can do the following

Step 1:

./my_program | awk '/^10./{a[$1]++ } END { for(i in a) print i,a[i]}' | tee some_file | xargs -I{} dig +short -x {} > some_file2

Step 2:

paste <(awk '{print $1}' some_file ) <(awk '{print $1}' some_file2 ) > result

You can remove some_file and some_file2 if you wish. rm some_file some_file2

  • 1
    @Pushman thanks for sharing but I said I wanted to use the system() function of awk. – holasz Aug 7 at 9:50

You can also avoid using awk at all. A little slower (although using the same number of invocations of dig) but quite possibly more readable.

while read -r ip count
    printf "%s %s %s\n" "$ip" "$count" "$(dig +short -x "$ip" | xargs)"
done < ipaddresses.txt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.