0

I want to delete duplicate entry e from file1 if e is also present in file2.

Input file1:

x1  y1
x2  y2
x3  y3
x4  y4
y1  x1
x5  y5
y3  x3
x6  y6
x5  y5

Input file2:

y1  x1
y2  x2
y3  x3
y4  x4
x1  y1
y5  x5
x3  y3
y6  x6
x5  y5

Desired output:

x1  y1
x2  y2
x3  y3
x4  y4
x5  y5
x6  y6

I have used following shell script:

awk 'FNR==NR {

   lines[NR,"col1"] = $1
   lines[NR,"col2"] = $2
   lines[NR,"line"] = $0
   next
    }

  (lines[FNR,"col1"] != $1) {($1 in lines)
    print lines[FNR,"line"]
    next
}' file1.txt file2.txt

But it is giving following output:

x1  y1
x2  y2
x3  y3
x4  y4
y1  x1
x5  y5
y3  x3
x6  y6
1
  • x1 y1 appears in both files?
    – cuonglm
    Jul 31 '14 at 11:10
2

First : your desired output should be :

y2  x2
y4  x4
y5  x5
y6  x6

because "x3 y3" and "x1 y1" are present in the both files

to get lines that are present in file1 only you can simply do

grep -v -f file1 file2

From man page

-v
--invert-match
 Invert the sense of matching, to select non-matching lines. (-v is specified by POSIX.)

-f file
   --file=file
Obtain patterns from file, one per line. The empty file contains zero patterns, and therefore matches nothing. (-f is specified by POSIX.)  
0

Try this:

awk '{if($1>$2) print $2 "  " $1; else print $0;}' file1.txt file2.txt | sort -u > out.txt

this will output:

x1  y1
x2  y2
x3  y3
x4  y4
x5  y5
x6  y6

awk simply reorders columns by alphabetical order, sort -u (unique) removes duplicated lines.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.