2

I have a directory sub1 with the following files:

$ wc -l *

5 file1.csv
5 file3.csv
1 file4.csv

In sub2, I have the following:

$ wc -l *
5 file1.csv
5 file2.csv
1 file3.csv
5 file4.csv
1 file5.csv

In the first directory, I might have files with added lines, which then go to the second dir. In this example, I might need to update file3 in sub2.

How do I get a list of the files with differences?


I did some tests with diff and grep, but it doesn't work because the directories have different files (and hence the lines are different):

~/dir1/$ wc -l >> wc.luis

~/dir1/$ wc -l * | awk '{ gsub(/\/home.*dir1\//,""); print $0 }' 
                 | diff --side-by-side wc.luis -
                 | grep \|

Ideally, I would get a list like this:

5 file3.csv | 1 file3.csv
1 file4.csv | 5 file4.csv

Any help is appreciated!


Notes:

  • I cannot check on the date, because all files were updated, with or without changes.

  • Sometimes the newest files lack some lines, for which reason I cannot just take the bigger one.

  • Isn't this just diff -r sub1 sub2 or am I missing something? – Stephen Harris Sep 5 '16 at 16:35
  • That would show all different lines in all different files, and I need only check which files are present in both, but different; and print their line count if possible. – Luis Sep 5 '16 at 16:46
  • 1
    diff -r sub1 sub2 | grep ^diff will just report on the filenames that are present in both but different. – Stephen Harris Sep 5 '16 at 16:47
1

Here is a quick and dirty shell "one-liner" with example output:

$ join -j2 <(cd sub1; wc -l *) <(cd sub2; wc -l *) | awk '$2!=$3'
file3.csv 5 1
file4.csv 1 5
total 11 17

The total line is an artifact from the output of wc. It can be removed with another filter:

$ join -j2 <(cd sub1; wc -l *) <(cd sub2; wc -l *) | awk '$2!=$3' | head -n-1
file3.csv 5 1
file4.csv 1 5

Explanation:

join will join two files based on a common column. In this case we join based on the second column (-j2). In the output of wc the second column is the filename. This will only print files which are common in both directories.

The wc invocations are done in process substitutions with working directory changed to sub1 or sub2 so the filenames are printed without directory name. This is so that join can find the common files.

The awk command compares the value in the second and third column and only prints the line if the values differ. This will filter out files with the same line count.

head -n-1 will print all lines but not the last line. This will filter out the last total line from wc.

  • Great, thank you! Your approach has also the plus that the line numbers are side-to-side, cool! – Luis Sep 6 '16 at 9:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.