I have a directory containing about 7,000 music files. I used lame to recursively re-encode all files in it to a separate directory, outputting all files with the same relative path and file name. The output files have a .mp3 extension, but some of the input files had different extensions (.wma, .aac, etc).

I can see that there is a file count difference of ~100 files missing in the output directory. What I want to do is run a compare of the two directories and obtain a list of the files that exist in the source, but not in the destination. This would be simple enough except I need to ignore differences in file extension.

I've tried using rsync with dry-run turned on but I couldn't figure out a way to ignore file extensions. I've also tried diff but was unable to find an option to only check by name but ignore file extensions. I started thinking I could just do a recursive ls on both directories, remove the file extensions, and then compare the outputs, but I really have no idea on where to start with modifying the ls output using sed or awk.

1 Answer 1


To see a listing, here are two variants, one that recurses into subdirectories and one that doesn't. All use syntax specific to bash, ksh and zsh.

comm -3 <(cd source && find -type f | sed 's/\.[^.]*$//' | sort) \
        <(cd dest && find -type f | sed 's/\.[^.]*$//' | sort)
comm -3 <(cd source && for x in *; do printf '%s\n' "${x%.*}"; done | sort) \
        <(cd dest && for x in *; do printf '%s\n' "${x%.*}"; done | sort)

Shorter, in zsh:

comm -3 <(cd source && print -lr **/*(:r)) <(cd dest && print -lr **/*(:r))
comm -3 <(print -lr source/*(:t:r)) <(print -lr dest/*(:t:r))

The comm command lists the lines that are common to two files (comm -12), that are only in the first file (comm -23) or that are only in the second file (comm -13). The numbers indicate what is subtracted from the output¹. The two input files must be sorted.

Here, the files are in fact the output of a command. The shell evaluates the <(…) construct by providing a “fake” file (a FIFO or a /dev/fd/ named file descriptor) as the argument to the command.

¹ So here the minus sayers are fully justified.

If you want to perform actions on the files, you'll probably want to iterate over the source files.

cd source
for x in *; do
  set -- "…/dest/${x%.*}".*
  if [ $# -eq 1 ] && ! [ -e "$1" ]; then
    echo "$x has not been converted"
  elif [ $# -gt 1 ]; then
    echo "$x has been converted to more than one output file: " "$@"
    echo "$x has been converted to $1"
  • 1
    +1 for the footnote especially (Are you one? I am.), but also for the, as usual, excellent answer. Nov 26, 2010 at 23:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.