The object of the script I'm making is to compare two series of files. The file names are themselves stored into two separate files, one path per line. My idea is to have two while read loops, one for each list of file names, but how can I mix the two loops together?

while read compareFile <&3; do     
 if [[ ! $server =~ [^[:space:]] ]] ; then  #empty line exception
    echo "Comparing file - $compareFile"
 if diff "$compareFile" _(other file from loop?_) >/dev/null ; then
     echo Same
      echo Different
done 3</infanass/dev/admin/filestoCompare.txt

I need to be able to compare files from two different lists at the same time through two while read loops... Is this even possible?

closed as off-topic by slm Sep 14 '14 at 3:58

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question has been posted on multiple sites. Cross-posting is strongly discouraged; see the help center and community FAQ for more information." – slm

  • Are you intenting to make this script as a king of challenge? If not so, powerful tools already exist to compare files, like diff. – lgeorget Jul 10 '13 at 20:27
  • "kind of challenge", sorry – lgeorget Jul 10 '13 at 20:41
  • @lgeorget the OP is using diff. – terdon Jul 10 '13 at 20:43
  • ah, files from two lists. Sorry for the useless comments... – lgeorget Jul 10 '13 at 20:55
  • Please avoid cross-posting – iruvar Jul 11 '13 at 14:05

You don't need two loops; you just need to read from two files in the one loop.

while read compareFile1 <&3 && read compareFile2 <&4; do     
 if [[ ! $server =~ [^[:space:]] ]] ; then  #empty line exception
    echo "Comparing file - $compareFile"
 if diff "$compareFile1" "$compareFile2" >/dev/null ; then
     echo Same
      echo Different
done 3</infanass/dev/admin/filestoCompare.txt 4<other_file
  • thats alot more less code thanks! how to i handle an empty line exception at the same time then for the two loops? – mkrouse Jul 11 '13 at 13:49
  • @mkrouse, I don't know what you were doing with that $server variable there before, but however you test for an empty line on the one variable, you just do the same on the other... – psusi Jul 11 '13 at 14:13

Method 1: use what you know

Since you already know how to loop over one file, you could combine the files and then process the combined files. The command paste joins two files line by line. It puts a tab between the lines coming from the two files, so this solution assumes that there are no tabs in your file names. (You can change the separator but you have to find a character that isn't present in a file name.)

paste -- "$list1.txt" "list2.txt" |
while IFS=$'\t' read -r file1 file2 rest; do
  diff -q -- "$file1" "$file2"
  case $? in
    0) status='same';;
    1) status='different';;
    *) status='ERROR';;
  echo "$status $file1 $file2"

If you want to skip blank lines, you need to do it in each file separately, since paste might match a blank line from one file with a non-blank line from another file. You can use grep to filter the non-blank lines.

paste -- <(grep '[^[:space:]]' "$list1.txt") <(grep '[^[:space:]]' "list2.txt") |
while IFS=$'\t' read -r file1 file2 rest; do

Note that if the two files have different lengths, you'll get an empty $file2 (regardless of which list ended first).

Method 2: loop over two files

You can put as complex a command as you like in the condition of the while loop. If you put read file1 <&3 && read file2 <&4 then the loop will run as long as both files have a line to read, i.e. until one file runs out.

while read -u 3 -r file1 && read -u 4 -r file2; do
done 3<list1..txt 4<list2.txt

If you want to skip over blank lines, it's a little more complicated, because you have to do the skipping in the two files independently. The easy way is to break the problem into two parts: skip the blank lines from one file, and process the non-blank lines. One method to skip the blank lines is to process through grep as above. Watch out for the necessary space between the < redirection operator and the <( that starts a command susbtitution.

while read -u 3 -r file1 && read -u 4 -r file2; do
done 3< <(grep '[^[:space:]]' "$list1.txt") 4< <(grep '[^[:space:]]' "list2.txt")

Another method is to write a function that behaves like read but skips blank lines. This function can work by calling read in a loop. It doesn't have to be a function, but a function is the best approach, both to organize your code and because that piece of code needs to be called twice. In the function, ${!#} is an instance of the bash construct ${!VARIABLE} which evaluates to the value of the variable whose name is the value of VARIABLE; here the variable is the special variable # which contains the number of positional parameter, so ${!#} is the last positional parameter.

function read_nonblank {
  while read "$@" &&
        [[ ${!#} !~ [^[:space:]] ]]
  do :; done
while read_nonblank -u 3 -r file1 && read_nonblank -u 4 -r file2; do
done 3<list1..txt 4<list2.txt
  • I like the use of read's -u option – Felipe Alvarez Nov 26 '13 at 22:31

One approach would be to use read -ra instead of just read. Assuming the filestoCompare.txt contained 2 columns with the file names in each, the read -ra would read both columns in at the same time and assign them into an array, compareFile. This array could then be accessed so that index 0 would be the first file and index 1 would be the 2nd file each time through the while loop.


Say I have this file: filestoCompare.txt, and it contains the following:

file1 file2
file3 file4
file5 file6

The command to go through this file would be as follows:

$ while read -ra a ; do printf "%s\t%s\n" ${a[0]} ${a[1]}; done < filestoCompare.txt
file1   file2
file3   file4
file5   file6

If the 2 files are indeed separate files such as:



They can be joined together with the paste command like so:

$ paste list1 list2 > list1and2

Here's the contents of list1and2:

$ cat list1and2
file1   file4
file2   file5
file3   file6
  • But that's not the input format: the lists are in two different files. You could join them first. – Gilles Jul 11 '13 at 13:22
  • @Gilles - I know that's not the input format, I believe I even said that "...Assuming the filestoCompare.txt contained 2 columns with the file names in each...". I understand your contention and don't disagree. The OP hasn't provided any further guidance on this question since posting it. – slm Jul 11 '13 at 13:26
  • @Gilles - what if I add an example showing how to use the command paste to join the 2 files? Would that get you to undownvote? – slm Jul 11 '13 at 13:35

Not the answer you're looking for? Browse other questions tagged or ask your own question.