I need a script for resolve hosts.

For now I run this script on local machine

for i in `tail -F access.log | awk '{print $8}' | awk '{gsub("http://|/.*","")}2' | awk '{gsub("http://|:.*","")}1' | grep -E -v "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}"`
   nslookup $i [dns-server_ip]; 
done > ips.txt

But i need tail file from some remote hosts and run nslookup on my local-machine and i don't have a clue how to do it.

  • 1
    back quote should be replace by $( ) , also it look way to many awk and grep a signle awk command should do.
    – Archemar
    Aug 13 '15 at 6:57
  • Thank you! im not so good in regexp but im looking into it
    – Alex K.
    Aug 13 '15 at 8:45

Use ssh?

Most commands can be executed remotely by just prepending the ssh command, so replace tail -F access.log with ssh REMOTEHOST tail -F access.log ==>

ssh myUSER@myREMOTEHOST tail -F access.log |
awk '{print $8}' |
awk '{gsub("http://|/.*","")}2' | awk '{gsub("http://|:.*","")}1' |
grep -E -v "([0-9]{1,3}\.){3})[0-9]{1,3}" | while read i ;
   nslookup $i $dnsServerIP; 
done > ips.txt

As @kasperd and @archemar also hinted, you could/should clean up that long pipeline. Here is my take:

ssh myUSER@myREMOTEHOST tail -F access.log |
awk '{$0=$8; gsub("https?://|[/:].*","")} !/([0-9]{1,3}.){1,3}[0-9]{1,3}/' |
while read i ; do
    nslookup $i $dnsServerIP; 
done > ips.txt


  • $0=$8. In your example output, the column with the URL you want is #8. This command overwrites $0 (the entire line) with only element $8, effectively throwing away the rest. This replaces '{print $8}'

  • gsub("https?://|[/:].*","")} replaces both of your gsub-calls with one covering all possibilities in one. In your code you also searched for "http://" twice, and did not match https.

  • !/([0-9]{1,3}.){1,3}[0-9]{1,3}/' replaces your separate grep call while using the exact same regex. It evaluates to true, when $0 is not a numeric IP, and will this implicitly {print $0}'. Slightly shorter h=$0;gsub(/[0-9.]/,"",h)} h has the same effect.

Note: The trailing 2 and 1 behind the } behind your gsub-calls evaluate to true, which in turn gets implicitly expanded to true {print $0}. This is how/why the last regex in my solution prints the non-matching line implicitly.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.