in folder , we have the following HADOOP binary files and their size (BYTES)
du -sb * | grep HADOOP[a-z]
334542327 HADOOPaa
334542327 HADOOPab
334542327 HADOOPac
334542327 HADOOPad
334542327 HADOOPae
334542327 HADOOPaf
334542327 HADOOPag
334542327 HADOOPah
334542327 HADOOPai
334542327 HADOOPaj
334542327 HADOOPak
334542327 HADOOPal
334542327 HADOOPam
334542327 HADOOPan
334542327 HADOOPao
334542327 HADOOPap
334542327 HADOOPaq
334542327 HADOOPar
334542327 HADOOPas
334542327 HADOOPat
334542327 HADOOPau
334542327 HADOOPav
334542327 HADOOPaw
334542327 HADOOPax
334542327 HADOOPay
334542327 HADOOPaz
334542327 HADOOPba
334542327 HADOOPbb
932542327 HADOOPbc
334542327 HADOOPbd
334542327 HADOOPbe
434542327 HADOOPbf
934542327 HADOOPbg
108883803 HADOOPbh
by awk we success to sum all the numbers to total size in bytes
example
du -sb * | grep HADOOP[a-z] | awk '{ sum+=$1} END {print sum}'
now we want to do the same with md5
we try
md5sum * | grep HADOOP[a-z] | md5sum | awk '{print $1}'
2a85626137ae7d689b85e8e04e8a2523 -
but not so good and not so elegant , because we want only the sum of all md5 files ( left side is the md5 for each file ) that match HADOOP[a-z]
any suggestions?
md5sum HADOOP* >files.md5
? It will give you the MD5 hashes of each file, which means that you know which one is corrupt if an individual hash fails to verify withmd5sum -c files.md5
. Keeping track of the sizes would also not be neccesary, at least not for verification purposes, as the hashes would be different if the size of a file changes. Note that a hash of a set of hashes is dependent on the order of those hashes.