2

I have looked through MANY questions asked and none of the solutions are working for me. I want to run an md5sum check on all files in all subdirectories from that specified in cd [directory]. Example:

cd [directory]

for f in *; do 
    if [[ -f "$f" ]]; then
        (md5sum -- "$f" > "md5_${f}") 
    fi
done

Thanks to a commentor, I understood my initial error which took my command as a directory instead of running it.

The issue now is that it runs successfully in the first directory it is looking at, but does not run correctly in sub-directories.

Per comment

find . - type f ! - name /'md5_*' -execdir sh -c /
    for f; do
         b="$(basename "$f")";
         [ ! -e "md5_$b" ] && md5sum "$b" > "md5$b"
    done


output find: paths must precede expression: - Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]

  • Maybe replace ( MY COMMAND ) by ( ./PATH/TO/MY/COMMAND )? – 41754 Aug 29 '19 at 12:26
  • So, I cd to the directory I want. the (My Command) Is a check that I want run on all files in all sub-directories. – Pandasncode Aug 29 '19 at 12:27
  • Is your command a program or script you have written? Or is it a program in one of the directories already added to your path environment variable? – 41754 Aug 29 '19 at 12:29
  • It is a one line command that can be executed on a single file. (md5sum -- "$file" > "md5_${file}") – Pandasncode Aug 29 '19 at 12:30
  • 1
    Please use the body of your question to describe what you want to do and exactly what's happening (instead). The example you found executes one command per directory (without passing in that directory name). If you "want to run on all files in all sub-directories", then people might have an answer that looks very different from what you have. – Jeff Schaller Aug 29 '19 at 12:36
3

This can be done with one find command:

find . -type f ! -name 'md5_*' -execdir sh -c \
  'for f; do
     b="$(basename "$f")"
     [ ! -e "md5_$b" ] || [ "$b" -nt "md5_$b" ] && md5sum "$b" >  "md5_$b"
   done' sh {} +

The ! -name 'md5_* option excludes md5_* files, and the -execdir option causes find to cd to each directory containing files, and then running the sh -c '....' command in it. It runs as few sh -c ... commands per directory as possible (usually only one, depending on the maximum command line limit on your system - typically 2MB or more these days).

The [ ! -e "md5_$b" ] || [ "$b" -nt "md5_$b" ] test in the shell script ensures that md5sum is only run for files where either the md5_ file does not exist or if the file is newer than its md5_ file.


If you wanted to have only one md5sums.txt file in each directory, it would be simpler:

find . -type f ! -name 'md5sums.txt' -execdir sh -c \
  'md5sum "$@" >md5sums.txt' sh {} +

and even simpler if you wanted just one md5sums.txt file listing all files in all subdirectories:

rm -f md5sums.txt
find . -type f -exec md5sum {} + >> md5sums.txt

This version should work with busybox's version of find, and maybe other minimalist versions:

find . -type f ! -name 'md5_*' -exec sh -c \
  'for f; do
     cd "$(dirname "$f")"
     b="$(basename "$f")"
     [ ! -e "md5_$b" ] || [ "$b" -nt "md5_$b" ] && md5sum "$b" >  "md5_$b"
     cd - >/dev/null
   done' sh {} +

It'll run a bit slower because it has to do the directory changing in the shell script...and it has to do cd twice for every file, rather than just once per directory.

I would have used pushd and popd to change directories, but if you're running busybox find or similar, you're probably also running a minimalist sh like dash.

| improve this answer | |
  • 1
    btw, do you need one md5_ file per filename? why not just have a single md5sums.txt file in each directory? – cas Aug 29 '19 at 13:31
  • 1
    that's strange. I did test that before posting it. are you still having problems with this? did you copy-paste this or type it? – cas Aug 30 '19 at 0:48
  • 1
    1. are you running busybox or similar? on some tiny distro? 2. you typed -name, not just name right? 3. the quotes around 'md5_*' are necessary, to make sure the shell doesn't expand it if there are any md5_* files in the current directory. – cas Aug 30 '19 at 14:42
  • 1
    I get a similar error message when I run it with busybox find .... (but complaining about the -execdir option not being recognized). Looks like that's what your version of find is. Which linux distribution are you running? can you install the GNU findutils package? – cas Aug 30 '19 at 14:50
  • 1
    what's an "active staging area"? what version of find is available in that environment? what do you get if you run find --version? do you get about 7 lines of version and licensing message starting with find (GNU findutils) or an error message (perhaps mentioning busybox or tinybox) and brief usage info? or something else? – cas Aug 30 '19 at 15:03
0

This will find all files in a /dir and run md5sum /dir/file

find /path/to/directory -type f -print0 | xargs -0I{} md5sum {}
| improve this answer | |
0

This looks like a job for find:

find /directory -type f -exec sh -c 'a={}; md5sum -- $a > $a.md5' \;
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.