I have 2 files containing a list of songs. hdsongs.txt and sdsongs.txt

I wrote a simple script to list all songs and output to text files, to then run a diff against. It works fine for the most part, but the actual diff command in the script is showing the same line as being different. This is actually happening for multiple lines, but not all.

Here is an example of a song in both files:

$ grep Apologize \*songs\*

There is no trailing special character that I can see:

$ cat -A hdsongs.txt sdsongs.txt | grep Apologize

When I run diff, it shows the same line being in each file; but aren't the lines the same?

$ diff hdsongs.txt sdsongs.txt | grep Apologize
> Timbaland/Apologize.mp3
< Timbaland/Apologize.mp3

This is similar to the thread here: diff reports two files differ, although they are the same!

but this is for lines within the file, not the whole file, and the resolution there doesn't seem to fit in this case.

$ diff <(cat -A phonesongsonly.txt) <(cat -A passportsongsonly.txt) | grep Apologize
< Timbaland/Apologize.mp3$
> Timbaland/Apologize.mp3$

$ wdiff -w "$(tput bold;tput setaf 1)" -x "$(tput sgr0)" -y "$(tput bold;tput setaf 2)" -z "$(tput sgr0)" hdsongs.txt sdsongs.txt | grep Apologize

Does anyone know why diff would report the same line twice like this?

  • 1
    Maybe try to hexdump those two lines and see what differs?
    – user43791
    Commented Dec 14, 2015 at 23:32

3 Answers 3


My guess is you simply haven't sorted the files. That's one of the behaviors you can get on unsorted input:

$ cat file1 
$ cat file2
$ $ diff file1 file2
< foo
> foo

But, if you sort:

$ diff <(sort file1) <(sort file2)

The diff program's job is to tell you whether two files are identical and, if not, where they differ. It is not designed to find similarities between different lines. If line X of the one file is not the same as line X of the other, then the files are not the same. It doesn't matter if they contain exactly the same information, if that information is organized in a different way, the files are reported as different.


Since you have not stated that the files are sorted, I'll assume that they aren't.  This is the expected output from diff when a line appears in both files, but in different locations.  This would be clear if you looked at the entire diff output, rather than piping it through grep.


I would suggest trying to use something like the hexdiff program to get a binary/hexadecimal output, as the human eye can't always tell the difference between the characters a computer displays, and some characters may not be displayed.

  • 2
    cat -A would have shown most (all?) cases of non-printing characters. I'm not sure how it would have coped with unicode strangeness but other non-printing chars should have been shown.
    – terdon
    Commented Dec 14, 2015 at 23:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .