Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am in need of tools to verify data integrity of local harddrives, usb-drives, etc.

I am looking for a solution like the famous www.heise.de/download/h2testw or something that is at least common within repositories. (h2testw writes a specific datastring over and over onto the medium, then reads it again to verify if it was written correctly and displays write/read time/speed.)

I am not looking for

dd if=/dev/random of=/dev/sdx bs=1k && dd if=/dev/sdx of=/dev/null bs=1k

since it won't verify if everything was written correctly. It is only a test if read/write is successful to the device.

So far, I'm not too happy with

badblocks -w -v /dev/sdx1

either, since it seems very slow, and I don't know what it exactly writes, and if it considers wear-leveling on flash media.

There is a program named F3 http://oss.digirati.com.br/f3/ that needs to be compiled. Designed after h2testw, the concept sounds interesting, I'd just rather have it a pre-packaged binary.

share|improve this question
1  
Can you explain the requirement for this to be for bash in specific? Do you actually want it to be written in pure shell script? –  mattdm May 3 '12 at 13:11
1  
Please don't cross-post. Cross-posting is strongly discouraged -- if you post on one site and then change your mind it can always be migrated to another. unix.stackexchange.com/faq#cross-posting –  uther May 3 '12 at 13:42
    
@mattdm thanks for improving. Bash would be nice because it would work out of the box. It could be a common GUI tool too, if it works like mentioned above. badblocks seems unbelievably slow. Maybe something that writes a random block of xMB over and over, then verifies each copy via md5? no idea if that would be practical... –  Mr. Bash May 3 '12 at 19:14
    
@Mr.Bash What are you testing? Just that writing once and reading back gives the same result? Or a more involved reliability test with different patterns? –  Gilles May 3 '12 at 23:36
    
@Gilles yes. i could repeat that script several times, but once would give me enough security. Maybe testing, then again with a different pattern would be more involved? It shouldn't be just zero's ofc. And I'm not too fond of s.m.a.r.t. and the like. It's informative ofc, but real stress tests on a new medium seem more reliable to me. –  Mr. Bash May 4 '12 at 17:26

1 Answer 1

The following script creates a random file (around 100M) in shared memory, calculates its checksum, then copy it to the given block device several times so that the whole device is filled with copies of the random data, while reading it again and calculating its checksum to check if it does match the original one.

It produces output like:

We will create a test data file /dev/shm/testsource with size 100000000.
We will fill the device /dev/sdb (of size 16001036288) with this data (in 160 blocks) and then will check if the data is not corrupted.
This will erase all data in /dev/sdb.
Do you want to continue?
1) yes
2) no
#? 1
Creating test source file... (100000000)
Calculating source checksum...
9f4c31858b3bb1122974a5c9d8ec28c6f71b3367
Writing block 0 ...
Checking block 0 ... ok
Writing block 1 ...
Checking block 1 ... ok
Writing block 2 ...
Checking block 2 ... ok

It requires dd, blockdev, cut, sha1sum, /dev/urandom and /dev/shm. It requires access to the device (usually root). This has been tested on Linux 2.6.3x. It is slow. You can modify it to do several passes or to do all writing then all checking or wait sometime before checking (a "fade" test).

#!/bin/bash
TARGETDEVICE=$1
TESTFILE="/dev/shm/testsource"
BLOCKSIZE="100000000"
if [ -b "$1" ]; then
    TARGETSIZE=$(blockdev --getsize64 $TARGETDEVICE)
    if [ "$TARGETSIZE" -gt 0 ] ; then
      let "BLOCKS=$TARGETSIZE / $BLOCKSIZE"
      if [ "$BLOCKS" -lt 2 ] ; then
        BLOCKSIZE=$TARGETSIZE
        BLOCKS=1
      fi
    fi
    echo "We will create a test data file $TESTFILE with size $BLOCKSIZE."
    echo "We will fill the device $TARGETDEVICE (of size $TARGETSIZE) with this data (in $BLOCKS blocks) and then will check if the data is not corrupted."
    echo "This will erase all data in $TARGETDEVICE."
    echo "Do you want to continue?"
    select choice in yes no ; do
      if [ "$choice" == "yes" ] ; then

        echo "Creating test source file... ($BLOCKSIZE)"
        dd if=/dev/urandom of=$TESTFILE bs=$BLOCKSIZE count=1 status=noxfer 2> /dev/null

        echo "Calculating source checksum..."
        CHECKSUM=$(sha1sum $TESTFILE | cut -d " " -f 1)
        echo $CHECKSUM

        for ((y=0 ; y<$BLOCKS ; y++)) ; do
          echo "Writing block $y ..."
          dd if=$TESTFILE of=$TARGETDEVICE bs=$BLOCKSIZE count=1 seek=$y status=noxfer 2> /dev/null
          echo -n "Checking block $y ... "
          TESTCHECKSUM=$(dd if=$TARGETDEVICE bs=$BLOCKSIZE count=1 skip=$y status=noxfer 2> /dev/null | sha1sum | cut -d " " -f 1)
          if [ "$CHECKSUM" == "$TESTCHECKSUM" ] ; then
            echo "ok"
          else
            echo "MISMATCH"
            echo "(found $TESTCHECKSUM)"
          fi
        done
        rm -f $TESTFILE
        break

      fi
      if [ "$choice" == "no" ] ; then
        echo
        echo "Operation cancelled"
        echo
        break
      fi

    done

else 
    echo
    echo "Missing or wrong target device"
    echo "Usage: $0 /dev/device"
    echo
fi
share|improve this answer
    
How is this much different than the read-write test in badblocks? –  Michael Kjörling Jul 24 '12 at 8:54
    
Not too much, except you can modify it easily (bash vs. C). You could define the pattern, add a delay for a fade test, do several passes, etc. This script would be like badblocks -s -w -t random /dev/xxx. However, the main reason to post it was the OP asked for a Bash script. –  MV. Jul 24 '12 at 18:39

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.