Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a file containing two paths on each line. I want to remove the lines containing the same path twice.

I work on Linux and Solaris. I would like a one-liner in sed or awk or perl.

Example input file:

     /usr/lib/libgmp.so.3.3.3 /usr/lib/libgmp.so.3.3.3
     /usr/lib/libxslt.so.1.1.17 /usr/lib/libxslt.so.1.1.17
     /usr/lib/sse2/libgmp.so.3.3.3 /usr/lib/sse2/libgmp.so.3.3.3
     /usr/local/swp-tomcat-6.0/lib/commons-logging-1.1.1.jar /usr/local/swp-tomcat-6.0/lib/commons-logging-1.1.1.jar
     /usr/share/doc/libXrandr-1.1.1 /usr/share/doc/libXrandr-1.1.1
     /usr/share/doc/libxslt-1.1.17 /usr/share/doc/libxslt-1.1.17
     /etc/3.3.3.255 /etc/172.17.211.255
     /etc/1.1.1.255 /etc/172.17.213.255

Expected output:

     /etc/3.3.3.255 /etc/172.17.211.255
     /etc/1.1.1.255 /etc/172.17.213.255
share|improve this question
    
grep isn't good? That is the dedicated tool to output lines from a file matching certain condition: grep -vx '\s*\(\S\+\)\s\+\1\s*' file. –  manatwork Jun 4 '13 at 10:47

3 Answers 3

up vote 3 down vote accepted
awk '{ if ($1 != $2 ) print $1" "$2; }' file

Just replace file for the appropriate file.

Or as @manatwork mentioned in the comments and simpler

awk '$1!=$2' file
share|improve this answer
4  
Why not just awk '$1!=$2' file? –  manatwork Jun 4 '13 at 10:40
    
I was not aware it could be that simple. :/ –  BitsOfNix Jun 4 '13 at 10:43

You can express repeated text in grep's regexps (this is an extension to the mathematical notion of regular expression).

grep -v '^ *\([^ ][^ ]*\)  *\1 *$'

[^ ][^ ]* matches one or more non-space character. The backslash-parentheses make this a group, and \1 means “the same text as the first group”.

share|improve this answer
    
Except from being an alternative, can you give an argument why one should use grep over awk here? –  Bernhard Jun 5 '13 at 8:35
1  
@Bernhard Both are good methods. More people know grep basics than awk basics, but on the other hand the awk code is clearer, so I think there's no clear winner. –  Gilles Jun 5 '13 at 8:55

This might work for you (GNU sed):

sed -r '/(\S+)\s\1/d' file
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.