2

I have two text files with the same number of newline terminated strings. The lines in the two files correspond to each other. The lines in each file individually might be repeated.

I would like to quickly identify which two lines differ and output either from the first file or from the second one.

File A:

this is a string
this is another string
empty string

File B:

this is A string
this is another string
Empty string

From perspective of file A, I would like to output first and third lines since they are different than in file B. Similarly for file B, I would output first and third lines of this files.

My standard approach to comparing files was to sort the two files and then use the comm binary but sorting would destroy the correspondence between the two files. I also tried to get this done with diff but it looks to be designed for a different task.

It would also be ok to output the two differing lines from both files with a tab separation.

  • Is every line in File A uniqe and is every line in File B uniqe? – Cyrus Apr 6 '15 at 21:51
  • @Cyrus No, the lines are not unique. The only information is that the two same line number strings are corresponding. For example the line 10 from file A and the line 10 from file B are comparable. – Vladislavs Dovgalecs Apr 6 '15 at 21:52
  • File A contains multiple times the same lines? – Cyrus Apr 6 '15 at 21:57
  • @Cyrus Yes, it can happen that exactly the same line appears in file A (as in file B). – Vladislavs Dovgalecs Apr 6 '15 at 21:58
  • 1
    @Cyrus Yes, the lines in file A can appear multiple times. – Vladislavs Dovgalecs Apr 6 '15 at 22:01
2

This does the comparison from the perspective of fileA:

$ awk 'FNR==NR{a[NR]=$0;next;} $0!=a[FNR]' fileB fileA
this is a string
empty string

This approach reads the whole of fileB into memory. Thus, if your files are huge (too big for memory), you should choose another approach.

Similarly, to obtain the output from the perspective of fileB:

$ awk 'FNR==NR{a[NR]=$0;next;} $0!=a[FNR]' fileA fileB
this is A string
Empty string

More Memory Efficient Approach

This approach only reads in two lines at a time and thus is more memory efficient. From the perspective of FileA:

$ awk '{a=$0;getline <"fileA";} $0!=a' fileB
this is a string
empty string

From the perspective of fileB:

$ awk '{a=$0;getline <"fileB";} $0!=a' fileA
this is A string
Empty string
  • Can this be improved by actually storing ones and zeros in the "a" array instead of strings? It could be more memory efficient. – Vladislavs Dovgalecs Apr 6 '15 at 21:55
  • 1
    @xeon I just added a memory-efficient approach. – John1024 Apr 6 '15 at 22:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.