Where can I find storage integrity (write/read) test tools?

Question

I am in need of tools to verify data integrity of local harddrives, usb-drives, etc.

I am looking for a solution like the famous www.heise.de/download/h2testw or something that is at least common within repositories. (h2testw writes a specific datastring over and over onto the medium, then reads it again to verify if it was written correctly and displays write/read time/speed.)

I am not looking for

dd if=/dev/random of=/dev/sdx bs=1k && dd if=/dev/sdx of=/dev/null bs=1k

since it won't verify if everything was written correctly. It is only a test if read/write is successful to the device.

So far, I'm not too happy with

badblocks -w -v /dev/sdx1

either, since it seems very slow, and I don't know what it exactly writes, and if it considers wear-leveling on flash media.

There is a program named F3 http://oss.digirati.com.br/f3/ that needs to be compiled. Designed after h2testw, the concept sounds interesting, I'd just rather have it a pre-packaged binary.

Answer 1

The following script creates a random file (around 100M) in shared memory, calculates its checksum, then copy it to the given block device several times so that the whole device is filled with copies of the random data, while reading it again and calculating its checksum to check if it does match the original one.

It produces output like:

We will create a test data file /dev/shm/testsource with size 100000000.
We will fill the device /dev/sdb (of size 16001036288) with this data (in 160 blocks) and then will check if the data is not corrupted.
This will erase all data in /dev/sdb.
Do you want to continue?
1) yes
2) no
#? 1
Creating test source file... (100000000)
Calculating source checksum...
9f4c31858b3bb1122974a5c9d8ec28c6f71b3367
Writing block 0 ...
Checking block 0 ... ok
Writing block 1 ...
Checking block 1 ... ok
Writing block 2 ...
Checking block 2 ... ok

It requires dd, blockdev, cut, sha1sum, /dev/urandom and /dev/shm. It requires access to the device (usually root). This has been tested on Linux 2.6.3x. It is slow. You can modify it to do several passes or to do all writing then all checking or wait sometime before checking (a "fade" test).

#!/bin/bash
TARGETDEVICE=$1
TESTFILE="/dev/shm/testsource"
BLOCKSIZE="100000000"
if [ -b "$1" ]; then
    TARGETSIZE=$(blockdev --getsize64 $TARGETDEVICE)
    if [ "$TARGETSIZE" -gt 0 ] ; then
      let "BLOCKS=$TARGETSIZE / $BLOCKSIZE"
      if [ "$BLOCKS" -lt 2 ] ; then
        BLOCKSIZE=$TARGETSIZE
        BLOCKS=1
      fi
    fi
    echo "We will create a test data file $TESTFILE with size $BLOCKSIZE."
    echo "We will fill the device $TARGETDEVICE (of size $TARGETSIZE) with this data (in $BLOCKS blocks) and then will check if the data is not corrupted."
    echo "This will erase all data in $TARGETDEVICE."
    echo "Do you want to continue?"
    select choice in yes no ; do
      if [ "$choice" == "yes" ] ; then

        echo "Creating test source file... ($BLOCKSIZE)"
        dd if=/dev/urandom of=$TESTFILE bs=$BLOCKSIZE count=1 status=noxfer 2> /dev/null

        echo "Calculating source checksum..."
        CHECKSUM=$(sha1sum $TESTFILE | cut -d " " -f 1)
        echo $CHECKSUM

        for ((y=0 ; y<$BLOCKS ; y++)) ; do
          echo "Writing block $y ..."
          dd if=$TESTFILE of=$TARGETDEVICE bs=$BLOCKSIZE count=1 seek=$y status=noxfer 2> /dev/null
          echo -n "Checking block $y ... "
          TESTCHECKSUM=$(dd if=$TARGETDEVICE bs=$BLOCKSIZE count=1 skip=$y status=noxfer 2> /dev/null | sha1sum | cut -d " " -f 1)
          if [ "$CHECKSUM" == "$TESTCHECKSUM" ] ; then
            echo "ok"
          else
            echo "MISMATCH"
            echo "(found $TESTCHECKSUM)"
          fi
        done
        rm -f $TESTFILE
        break

      fi
      if [ "$choice" == "no" ] ; then
        echo
        echo "Operation cancelled"
        echo
        break
      fi

    done

else 
    echo
    echo "Missing or wrong target device"
    echo "Usage: $0 /dev/device"
    echo
fi