Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a directory containing about 7,000 music files. I used lame to recursively re-encode all files in it to a separate directory, outputting all files with the same relative path and file name. The output files have a .mp3 extension, but some of the input files had different extensions (.wma, .aac, etc).

I can see that there is a file count difference of ~100 files missing in the output directory. What I want to do is run a compare of the two directories and obtain a list of the files that exist in the source, but not in the destination. This would be simple enough except I need to ignore differences in file extension.

I've tried using rsync with dry-run turned on but I couldn't figure out a way to ignore file extensions. I've also tried diff but was unable to find an option to only check by name but ignore file extensions. I started thinking I could just do a recursive ls on both directories, remove the file extensions, and then compare the outputs, but I really have no idea on where to start with modifying the ls output using sed or awk since I'm a compete noob with those.

share|improve this question

1 Answer 1

To see a listing, here are two variants, one that recurses into subdirectories and one that doesn't. All use syntax specific to bash, ksh and zsh.

comm -3 <(cd source && find -type f | sed 's/\.[^.]*$//' | sort) \
        <(cd dest && find -type f | sed 's/\.[^.]*$//' | sort)
comm -3 <(cd source && for x in *; do printf '%s\n' "${x%.*}"; done | sort) \
        <(cd dest && for x in *; do printf '%s\n' "${x%.*}"; done | sort)

Shorter, in zsh:

comm -3 <(cd source && print -lr **/*(:r)) <(cd dest && print -lr **/*(:r))
comm -3 <(print -lr source/*(:t:r)) <(print -lr dest/*(:t:r))

The comm command lists the lines that are common to two files (comm -12), that are only in the first file (comm -23) or that are only in the second file (comm -13). The numbers indicate what is subtracted from the output¹. The two input files must be sorted.

Here, the files are in fact the output of a command. The shell evaluates the <(…) construct by providing a “fake” file (a FIFO or a /dev/fd/ named file descriptor) as the argument to the command.

¹ So here the minus sayers are fully justified.


If you want to perform actions on the files, you'll probably want to iterate over the source files.

cd source
for x in *; do
  set -- "…/dest/${x%.*}".*
  if [ $# -eq 1 ] && ! [ -e "$1" ]; then
    echo "$x has not been converted"
  elif [ $# -gt 1 ]; then
    echo "$x has been converted to more than one output file: " "$@"
  else
    echo "$x has been converted to $1"
  fi
done
share|improve this answer
    
+1 for the footnote especially (Are you one? I am.), but also for the, as usual, excellent answer. –  Dennis Williamson Nov 26 '10 at 23:08

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.