0

in folder , we have the following HADOOP binary files and their size (BYTES)

 du -sb * | grep HADOOP[a-z]
334542327       HADOOPaa
334542327       HADOOPab
334542327       HADOOPac
334542327       HADOOPad
334542327       HADOOPae
334542327       HADOOPaf
334542327       HADOOPag
334542327       HADOOPah
334542327       HADOOPai
334542327       HADOOPaj
334542327       HADOOPak
334542327       HADOOPal
334542327       HADOOPam
334542327       HADOOPan
334542327       HADOOPao
334542327       HADOOPap
334542327       HADOOPaq
334542327       HADOOPar
334542327       HADOOPas
334542327       HADOOPat
334542327       HADOOPau
334542327       HADOOPav
334542327       HADOOPaw
334542327       HADOOPax
334542327       HADOOPay
334542327       HADOOPaz
334542327       HADOOPba
334542327       HADOOPbb
932542327       HADOOPbc
334542327       HADOOPbd
334542327       HADOOPbe
434542327       HADOOPbf
934542327       HADOOPbg
108883803       HADOOPbh

by awk we success to sum all the numbers to total size in bytes

example

 du -sb * | grep HADOOP[a-z] | awk '{ sum+=$1} END {print sum}'

now we want to do the same with md5

we try

md5sum * | grep HADOOP[a-z] | md5sum | awk '{print $1}' 
2a85626137ae7d689b85e8e04e8a2523  -

but not so good and not so elegant , because we want only the sum of all md5 files ( left side is the md5 for each file ) that match HADOOP[a-z]

any suggestions?

  • 2
    What's wrong with the result of md5sum HADOOP* >files.md5? It will give you the MD5 hashes of each file, which means that you know which one is corrupt if an individual hash fails to verify with md5sum -c files.md5. Keeping track of the sizes would also not be neccesary, at least not for verification purposes, as the hashes would be different if the size of a file changes. Note that a hash of a set of hashes is dependent on the order of those hashes. – Kusalananda Aug 18 '19 at 15:22
  • yes I took his advice , and no problem its good offer – yael Aug 18 '19 at 21:32
1

Not sure what you're going for here... but it sounds like you want awk (or cut) after the grep to only print the sums. But then a checksum of checksums to ensure you have all the files? Is that the end result you wanted?

BTW, I'm almost positive the glob md5sum * returns a random order, so you probably want a sort in there somewhere to ensure it's the same each time and repeatable across machines.

  • what I want is the total md5 for all files that match HADOOP[a-z] , can you give your example for that? – yael Aug 18 '19 at 12:52
  • Pretty sure that is just md5sum HADOOP[a-z] - see man bash for globbing rules. – Aaron D. Marasco Aug 19 '19 at 13:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.