4

I want to extract part from a log file column which is like this:

xx.xxx.xx.xx#59796:

Edit:

This is the actual log line:

Jan 10 17:38:11 server named[747]: client 21x.x0.x8x.xxx#40649: view external: query (cache) 'domain.TLD/A/IN' denied

The part before the "#" in the line above is an IP address, and I want to extract only the IP. The part after the "#" is random numbers and not always the same.

I use the below command to grep a pattern, extract the IP column, and then redirect the output to text file, but then I have to use an editor to leave out the extra characters from the extracted column.

grep -E 'view external.*denied' /var/log/messages |awk '{print $7}' > view_external_denied_ip.txt

If I can extract only the IP without the extra characters in the column, I would use the sort command to sort them ( sort | uniq -c | sort -rn ).

4
  • Please show us some of the actual input, we can't help you find it if we don't know what the rest of the line looks like.
    – terdon
    Jan 10, 2014 at 17:57
  • There are a bunch of ways to do this. Most trivial would be cut -d# -f1 but you can probably get something quicker by handling the whole line at once (e.g., with perl or awk). I second the request for an example line.
    – derobert
    Jan 10, 2014 at 18:00
  • I have edited the question and added the log line.
    – archmicht
    Jan 10, 2014 at 19:04
  • Yes derobert the cut command helped too, thank you.
    – archmicht
    Jan 10, 2014 at 19:36

3 Answers 3

3

I can give a better answer if you show your actual input but if all you need is to remove the characters after # (inclusive), use any of these (assuming your lines only contain a single #):

awk '/view external.*denied/{print $7}' logfile | sed 's/#.*//'

or

awk '/view external.*denied/{print $7}' logfile | cut -d '#' -f 1
1
  • awk '/view external.*denied/{print $7}' /var/log/messages | sed 's/#.*//' |sort |uniq -c |sort -rn
    – archmicht
    Jan 10, 2014 at 19:17
1

You can do this with a single sed command. First, as a matter of general strategy, to extract a part of a line in sed, use the s command, with a regex that matches the whole line (starting with ^ and ending with $), with the part to retain in a group (\(…\)), replacing the whole line by the content of the group(s) to keep. Pass the -n option to turn off default printing and put the p modifier to print lines where there is something to extract. You can restrict the extraction further by prefixing the command with a regex that must match the line.

sed -n '/view external.*denied/ s/^.* client \([0-9.][0-9.]*\)#.*/\1/p'

If you prefer to use awk, you can use its sub function to remove a part of a string.

grep -E 'view external.*denied' /var/log/messages |
awk '{sub(/#.*/, "", $7); print $7}'

You can also use the index function to locate the # and the substr function to extract the part you want to keep.

grep -E 'view external.*denied' /var/log/messages |
awk '{print substr($7, 1, index($7, "#"))}'

You can easily combine the grep command into the awk command.

</var/log/messages awk '/view external.*denied/ {sub(/#.*/, "", $7); print $7}'
0
sed 's/\#.*$'//' /path/to/logfile # this will trim anything including and after a `#` on every line.
2
  • Maybe you could expand this a bit and explain what you're doing?
    – slm
    Jan 10, 2014 at 18:52
  • Thank you, but I had to adjust the sed command to make it work, like terdon did.
    – archmicht
    Jan 10, 2014 at 19:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .