Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a large log file that contains numerous lines of the same entry, lets call it "repeat-info". As an example here is what a portion of the log might look like:

[Timestamp] repeat-info
[Timestamp] repeat-info
[Timestamp] Log information 1
[Timestamp] Log information 2
[Timestamp] repeat-info
[Timestamp] Log information 3
[Timestamp] repeat-info

Is there a command that can output the information in the log file and exclude the repeated information? It becomes a hassle if I have to use more file.log and sift through all the repeating information to find what it is I want to look at.

I am reading through the man pages for sed and awk as I saw those appear in searches for my question, however I haven't found anything conclusive that would do what I need.

I was searching through the older questions and found this question which is related to mine.

I was looking for a way to do this with a single command, or two piped together, without having to create a script.

Any help is appreciated!

share|improve this question
I'm not entirely sure what you are asking. Is it just a simple grep -v or is it something more? – jw013 May 10 '12 at 15:43
up vote 7 down vote accepted

There are a few ways to do this. Best would be grep:

grep -v 'repeat-info' file.log

Other ways:

sed '/repeat-info/d' file.log
sed -n '/repeat-info/!p' file.log

awk '!/repeat-info/' file.log
share|improve this answer
I would add perl -ni -e 'print unless /repeat-info/' file.log – tmow May 10 '12 at 16:28
@Kevin Thank you for the help! I was not aware one of the options for grep was to inverse the command. – SirCobalt May 10 '12 at 17:00
@Kevin Is there a way to use this same thing and exclude two different lines of information? (Eg. exclude two sets of repeated information from the same log file. Such as grep -v 'repeat-info' && 'more-repeated-info' file.log). – SirCobalt May 10 '12 at 17:08
@SirCobalt yes, you can use -e to specify more than one pattern, e.g. grep -v -e 'repeat-info' -e 'more-repeat' file.log – Kevin May 10 '12 at 17:11

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.