Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have been having odd and rare filesystem corruption lately that I suspect is the fault of my SSD. I am looking for a good drive torture test tool. Something that can write to the whole disk, then go back and read it looking for flying writes, corrupted blocks, blocks reverted to older revisions, and other errors. This would be much more than what badblocks does. Is there such a tool?

Note I am not looking for a performance benchmark and already checked the SMART status; says healthy and no bad blocks reported.

share|improve this question
    
Did you try? iozone.org –  positron Apr 16 '13 at 4:11
1  
Btw, you didn't indicate which OS/system hardware you are using. SSD(s) have been reported to experience the corruption you are talking about on some Mac OS X boxen when configured to power down the hard drive often to save power. This will cause corruption. I would imagine the same is likely in other OS/Hardware combos, if the drive is forced to sleep via a hard drive power down issued to an SSD. I would check the configuration of your system before burning up your SSD with drive test. –  Wing Tang Wong Apr 16 '13 at 15:49
1  
@WingTangWong, wow. I've ready that many SSDs screw up when they lose power, but when asked to go to sleep? That's one buggy drive. I'll keep an eye out for this. I'm using Linux and don't have it sleeping except maybe when I suspend the system... –  psusi Apr 16 '13 at 16:04

3 Answers 3

Might be overkill but there's Phoronix Test Suite. There's also bonnie++, as well as hdparam.

I usually use hdparam, for example:

% hdparm -Tt /dev/hdb
/dev/hdb:
 Timing buffer-cache reads:   128 MB in  1.25 seconds =102.40 MB/sec
 Timing buffered disk reads:  64 MB in 16.70 seconds =  3.83 MB/sec

I wouldn't call hdparam a torture test but it does give you a rough idea of a drives overall performance.

Determining a drives health

After you've tortured the drive you can use this command to check out the general health of the drive:

% sudo udisks --dump | grep -A 24 Updates
 Attribute       Current|Worst|Threshold  Status   Value       Type     Updates
===============================================================================
 raw-read-error-rate         103| 99| 34   good    5854752     Pre-fail Online 
 spin-up-time                100| 99|  0    n/a    0           Pre-fail Online 
 start-stop-count             98| 98| 20   good    2785        Old-age  Online 
 reallocated-sector-count    100|100| 36   good    0 sectors   Pre-fail Online 
 seek-error-rate              72| 60| 30   good    25872884688 Pre-fail Online 
 power-on-hours               89| 89|  0    n/a    424.4 days  Old-age  Online 
 spin-retry-count            100|100| 97   good    0           Pre-fail Online 
 power-cycle-count            98| 98| 20   good    2753        Old-age  Online 
 attribute-184               100|100| 99   good    0           Old-age  Online 
 reported-uncorrect          100|100|  0    n/a    0 sectors   Old-age  Online 
 attribute-188               100| 96|  0    n/a    0           Old-age  Online 
 high-fly-writes             100|100|  0    n/a    0           Old-age  Online 
 airflow-temperature-celsius  58| 42| 45 FAIL_PAST 42C / 108F  Old-age  Online 
 g-sense-error-rate          100|100|  0    n/a    124         Old-age  Online 
 power-off-retract-count     100|100|  0    n/a    15          Old-age  Online 
 load-cycle-count              1|  1|  0    n/a    248327      Old-age  Online 
 temperature-celsius-2        42| 58|  0    n/a    42C / 108F  Old-age  Online 
 hardware-ecc-recovered       45| 38|  0    n/a    5854752     Old-age  Online 
 reallocated-event-count      89| 89| 30   good    14877766723263 Pre-fail Online 
 current-pending-sector      100|100|  0    n/a    0 sectors   Old-age  Online 
 offline-uncorrectable       100|100|  0    n/a    0 sectors   Old-age  Offline
 udma-crc-error-count        200|200|  0    n/a    0           Old-age  Online 
 attribute-254               100|100|  0    n/a    0           Old-age  Online 

Disk health/maintenance tools

We've had good success using the following 2 tools where I work. HDAT2 & Spinrite. The latter is a commercial tool but the former, HDAT2, is an opensource project.

Here are a couple of screenshots of HDAT2:

ss #1 HDAT2

ss #2 HDAT2

You have to reboot your system into both of these so it's offline while you're performing these operations but they've both recovered drives that had failed or were starting to exhibit failures. The UI in HDAT2 is a little rough to navigate, we generally used the default choices for the most part and tried not to wander too far from there.

share|improve this answer
    
The first three you mention are performance benchmarks. Already checked SMART status and it is good with no bad sectors, so it looks like HDAT2 isn't what I'm looking for either. –  psusi Apr 16 '13 at 13:38
    
I wouldn't dismiss HDAT2, we were running chkdsks and they were clean as well, yet disk was still not bootable, running HDAT2 was able to find surface issues with the disk that it was able to repair enough to make the drive bootable. –  slm Apr 16 '13 at 14:08

bonnie++ comes to mind:

So, depending on your box's hardware configuration:

bonnie++ -d /path/to/mounted/ssd -r your-system-ram-size-in-MB

Example:

# For a 32GB system with the SSD formatted and mounted at /mnt/mounted-ssd-001
bonnie++ -d /mnt/mounted-ssd-001 -r 32000

It should give your device a good stress test. You can customize it as well.

Note, with an SSD, when a bad block happens, it may get remap'd automatically by the drive hardware, depending on the drive you are working with. Also, a torture test eats away at your SSD's write lifespan. So, use at your own discretion.

EDIT:

Adding a note about SSD failures, since it's been pointed out that Bonnie++ stress tests, but doesn't track errors. The way SSD(s) "remap bad blocks" is different from the way hard drives do remapping. How it goes about it depends entirely on which brand/make/model of SSD you have:

  • Cheap SSD(s) just fail, because they have no spare capacity to remap, or because they have no means of segregating the failed flash blocks. They will just hang or go offline and won't come back online.
  • Midrange SSD(s) with no spare capacity may generate Smartd alerts or perhaps even generate OS level block device errors when a failed block is detected. However, when the fail happens, the SSD's registered size will change. This can result in an error and the device being taken offline by the OS or it can result in the device itself hanging and needing to be pulled out and re-inserted for it to be recognized again. On re-registering, the device's available block size will be diminished.
  • High end SSD(s) with spare capacity will remap the bad blocks behind the scenes and may generate OS level alerts/warnings. When the spare capacity runs out, the device will probably fail along the lines of the Midrange SSD(s).

When the SSD resizes itself due to bad blocks being isolated, you might need to do the following to revive the drive, if the drive's firmware doesn't automatically do the proper updates automatically:

http://communities.intel.com/message/145676

Unless the stress test and error logging tool is specifically designed with SSD(s), in mind, you are just using up the lifespan of the device.

EDIT:

Based on info from answers above, suggest either replacing the cable with a better one or replacing the drive(RMA/Warrantee replacement), as that kind of OS filesystem level error is not normal.

Also, if your drive supports it, you can increase the amount of space reserved for handling errors:

http://www.thomas-krenn.com/en/wiki/SSD_Over-provisioning_using_hdparm

share|improve this answer
    
That's a performance benchmark. It may give the drive a workout, but I don't think it detects errors. –  psusi Apr 16 '13 at 13:41
    
The way that SSD(s) work, error detection comes in one of several forms, depending on the make/model of the SSD: Smartd error if a block is remapped and spare capacity is used(no faults), capacity of the device is reduced as a section of the flash storage is faulted(can cause smartd errors, can cause filesystem errors, can cause device to hang the bus by going offline. On pull/re-insert, drive is avaialble again, but may need to be reformatted), and the SSD can just outright appear to hang with no remapping(device becomes unresponsive even after re-insert). The failure path is not equiv to HD –  Wing Tang Wong Apr 16 '13 at 15:35
1  
the errors aren't detected by the drive, hence the need for a test tool. It manifests itself by the filesystem being remounted ro, and e2fsck finding and fixing lots of errors in the metadata. Also I've had some of my git repository pack files corrupted. It's a silent corruption that happens maybe once every month or two. At first I thought it may be a bug involving TRIM since I don't recall this happening before I enabled it, so I turned it back off and it's still happening. –  psusi Apr 16 '13 at 16:10
    
A couple of potential issues: bad drive cable or bad drive. You can test the bad drive cable by replacing the cable with another one. I've had this in the past and replacing with a better spec'd cable worked. In the case of a bad drive, RMA or send in for warrantee repair. –  Wing Tang Wong Apr 16 '13 at 16:30
1  
the problem is proving that it is a bad drive ( or really, a bug in the firmware ) and not say, a bug in the kernel. If it were a bad cable, it would manifest as sata ecc errors rather than random silent corruption. –  psusi Apr 16 '13 at 17:18

Use the maker test tool, its the best way to test a HD, as it can access to the low level tests, remap bad sectors, test all the smart health status (specially for a SSD, there are many registers unknown for most of us but can help the maker to see the hard disk status)

hirensCD have many testing tools, but i think that it was not updated to SSD enabled ones, so check directly the maker website. Some do support linux, others may require a windows liveCD (check the hirenCd again) or booting from a pendrive (freedos, special OS, etc)

Most older HD tools are not good for testing SSD, as a sector is never on the same places, is dynamically mapped by the firmware to spread the writes all over the drive. So if they do write tests, you are just burning write cycles instead of truly testing the disk.

Read tests are don't burn the SSD but also might not really test all SSD sectors, again due the firmware hiding the real layout.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.