4

I have a directory full of files. Each file will be copied to a specific type of destination host.

I want to calculate an MD5 sum for each file in the directory, and store that md5 sum in a file that matches the name of the file that generated the sum, but with .md5 appended.

So, for instance, if I have a directory with:

a.bin
b.bin
c.bin

The final result should be:

a.bin
a.bin.md5     # a.bin's calculated checksum
b.bin
b.bin.md5     # b.bin's calculated checksum
c.bin
c.bin.md5     # c.bin's calculated checksum

I have attempted this with find exec, and with xargs.

With find, I tried this command:

find . -type f -exec md5sum {} + > {}.md5

Using xargs, I tried this command:

find . -type f | xargs -I {} md5sum {} > {}.md5

In either case, I end up with a file called {}.txt, which isn't really what I am looking for.

Could anyone point out how to tweak these to generate the md5 files I am looking to generate?

3
  • 2
    Dumping all the output to a single file will let you use md5sum -c later to verify all files at once. Commented Nov 6, 2017 at 20:21
  • The problem is we won't be copying all files to the destinations. Only one file will be copied. So, we'd end up with a monolithic file listing files that mostly aren't present on the destination hosts. Using the example above, some hosts get a.bin, some get b.bin. The a.bin hosts don't need md5sum info for b.bin and c.bin. I hope that makes sense. :)
    – Matthew
    Commented Nov 6, 2017 at 20:38
  • Properly escaped xargs call find . -type f | xargs -I {} sh -c 'md5sum '{}' > '{}'.md5'
    – dtmland
    Commented Dec 20, 2019 at 4:11

5 Answers 5

3
cd /path/to/files &&
for file in *; do
    if [[ -f "$file" ]]; then
        md5sum -- "$file" > "${file}.md5"
    fi
done
0
3

Redirection is an operator of the shell.

In:

find some args  > {}.md5

find is run with some args and its stdout redirected to {}.md5.

Here, you'd need find to start a shell to perform one redirection for each file:

find . -type f -exec sh -c '
  for file do
    md5sum "$file" > "$file.md5"
  done' sh {} +

If you want to avoid the directory component in the .md5 part, you can either use the -execdir alternatives supported by many find implementations (though with some you'll still get a ./ directory component, and with some, that will still run one shell per file even with {} +:

find . -type f -execdir sh -c '
  for file do
    md5sum "$file" > "$file.md5"
  done' sh {} +

Or you can do it as:

find . -type f -exec sh -c '
  for file do
    (cd "${file%/*" && exec md5sum -- "${file##*/}") > "$file.md5"
  done' sh {} +
0

I know there are side effects to piping into a while loop, but I like the simplicilty, readability and extensibilty of this solution

find . -type f ! -name \*.md5 |\
  while read a; do md5sum $a > $a.md5; done

To use this scipplet to check sums simply:

find . -type f ! -name \*.md5 |\
  while read a; do echo $a; md5sum -c $a.md5; done
-1
find . -name '*.iso' -type f -exec sh -c "ls -lh {}; pv {} | md5sum > {}.md5" \;

optional

ls -lh {};

to show processing file, and

pv {}

to show progressbar on big (slow) iso file in my case

1
  • Looks OK, but you're restricting it to .iso files.
    – telcoM
    Commented Jan 20, 2018 at 17:50
-1
cd /path/to/files && ls -1| xargs md5sum |  xargs -l bash -c 'echo $0"  "$1 > $1.md5'

first xargs generate , second xargs picks up and save to .md5

If you need filter file patter, you can replace ls with find or add grep afterwards.

6
  • Parses output of ls and uses unquoted variable expansions.
    – Kusalananda
    Commented Feb 1, 2018 at 17:43
  • Can you explain your comments? you can replace ls with find or any other command.
    – Ming Jia
    Commented Feb 1, 2018 at 19:12
  • Assume spaces in any of the names in the files directory, for example. Also consider what would happen if one of the names is a directory.
    – Kusalananda
    Commented Feb 2, 2018 at 6:33
  • @MingJia Parsing output of ls is considered generally bad practice (see this ). Even if there aren't any difficult filenames, that doesn't mean one shouldn't anticipate them. Second big problem is the $0 and $1 should be double-quoted. Because of "word splitting", if you have file with spaces or tabs in filename, ` > $1` will have ambiguous redirect error. Try var="with space.txt"; echo "hi" > $var and then var="with space.txt"; echo "hi" > "$var" Commented Feb 2, 2018 at 6:55
  • @MingJia Don't get discouraged, you'll learn to pick up on these things if you stick around on the site, with time of course. Welcome, to the site and consider re-working your answer. Commented Feb 2, 2018 at 6:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .