I have a large log file that contains numerous lines of the same entry, lets call it "repeat-info". As an example here is what a portion of the log might look like:

[Timestamp] repeat-info
[Timestamp] repeat-info
[Timestamp] Log information 1
[Timestamp] Log information 2
[Timestamp] repeat-info
[Timestamp] Log information 3
[Timestamp] repeat-info

Is there a command that can output the information in the log file and exclude the repeated information? It becomes a hassle if I have to use more file.log and sift through all the repeating information to find what it is I want to look at.

I am reading through the man pages for sed and awk as I saw those appear in searches for my question, however I haven't found anything conclusive that would do what I need.

I was searching through the older questions and found this question which is related to mine.

I was looking for a way to do this with a single command, or two piped together, without having to create a script.

Any help is appreciated!

  • 2
    I'm not entirely sure what you are asking. Is it just a simple grep -v or is it something more? – jw013 May 10 '12 at 15:43

There are a few ways to do this. Best would be grep:

grep -v 'repeat-info' file.log

Other ways:

sed '/repeat-info/d' file.log
sed -n '/repeat-info/!p' file.log

awk '!/repeat-info/' file.log
  • I would add perl -ni -e 'print unless /repeat-info/' file.log – tmow May 10 '12 at 16:28
  • @Kevin Thank you for the help! I was not aware one of the options for grep was to inverse the command. – SirCobalt May 10 '12 at 17:00
  • @Kevin Is there a way to use this same thing and exclude two different lines of information? (Eg. exclude two sets of repeated information from the same log file. Such as grep -v 'repeat-info' && 'more-repeated-info' file.log). – SirCobalt May 10 '12 at 17:08
  • @SirCobalt yes, you can use -e to specify more than one pattern, e.g. grep -v -e 'repeat-info' -e 'more-repeat' file.log – Kevin May 10 '12 at 17:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.