2

A few years ago, I bought a 500GB Essentiel B external USB hard drive from Boulanger (a multimedia store in France). Since then I have had 2 failures of the order of 1x / 2 years which seem to lose the file system (currently NTFS) and which requires reformatting, after that everything is fine until the next failure ...

General symptom : copy problem between HDD and other media => I didn't notice the error message since it's been a long time since I touched the disk, but it must have been something like "Input / output error"

Here is what the old FS gave me before reformatting :

user@host:~$ ls -al /media/user/USER-EHD
ls: impossible d'accéder à '/media/user/USER-EHD/anniv_fany.avi': Erreur d'entrée/sortie
total 15841832
drwxrwxrwx  1 user  user        4096 oct.   3 21:14 .
drwx---rwx+ 6 root root       4096 mars  12 19:11 ..
-?????????? ? ?    ?             ?              ? anniv_fany.avi
drwxrwxrwx  1 user  user        4096 août  23  2020 xen_build
-rwxrwxrwx  1 user  user  1569481183 août   7  2016 踔ororite.avi
drwxrwxrwx  1 user  user        4096 août  23  2020 $RECYCLE.BIN
drwxrwxrwx  1 user  user        4096 août  21  2020 System Volume Information

2 weird things:

  • anniv_fany.avi : no metadata ????? + I/O error
  • 踔ororite.avi : was actually named sororite.avi

And

user@host:~$ lsblk -f /dev/sdg
NAME   FSTYPE LABEL    UUID                                 MOUNTPOINT
sdg                                                        
└─sdg1 ntfs   USER-EHD 67AC02BC429C25D2             /media/user/USER-EHD

SMART verification => seems OK (see below)

Note : I am aware of the fact that the SMART is not 100% reliable (see that it is not useful to know the real health of the HDD)

user@host:~$ sudo smartctl -a -d sat -t long /dev/sdg
# so ... 2 hour later
user@host:~$ sudo smartctl -l selftest -d sat /dev/sdg
smartctl 6.5 2016-01-24 r4214 [x86_64-linux-4.4.0-148-generic] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%       586         -
# 2  Extended offline    Completed without error       00%       500         -
# 3  Extended offline    Aborted by host               90%       499         -
# 4  Short offline       Completed without error       00%       499         -
# 5  Extended offline    Completed: read failure       00%       489         769438584
# 6  Extended offline    Completed: read failure       00%       488         769438584
# 7  Extended offline    Aborted by host               90%       486         -
# 8  Short offline       Completed without error       00%       486         -
# 9  Short offline       Completed without error       00%       112         -
2 of 2 failed self-tests are outdated by newer successful extended offline self-test # 1

=> would be OK : the 2 old reading errors are obsolete and smartctl considers that the long test went smoothly !

Here the details :

user@host:~$ sudo smartctl -a -d sat /dev/sdg
[sudo] Mot de passe de user : 
smartctl 6.5 2016-01-24 r4214 [x86_64-linux-4.4.0-148-generic] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Toshiba 2.5" HDD MQ01ABF...
Device Model:     TOSHIBA MQ01ABF050
Serial Number:    67ODTDZST
LU WWN Device Id: 5 000039 7d160c9a5
Firmware Version: AM001U
User Capacity:    500 107 862 016 bytes [500 GB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Form Factor:      2.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS (minor revision not indicated)
SATA Version is:  SATA 2.6, 3.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Sat Mar 13 16:30:13 2021 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                    was completed without error.
                    Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                    without error or no self-test has ever 
                    been run.
Total time to complete Offline 
data collection:        (  120) seconds.
Offline data collection
capabilities:            (0x5b) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    Offline surface scan supported.
                    Self-test supported.
                    No Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                    General Purpose Logging supported.
Short self-test routine 
recommended polling time:    (   2) minutes.
Extended self-test routine
recommended polling time:    ( 115) minutes.
SCT capabilities:          (0x003d) SCT Status supported.
                    SCT Error Recovery Control supported.
                    SCT Feature Control supported.
                    SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000b   100   100   050    Pre-fail  Always       -       0
  2 Throughput_Performance  0x0005   100   100   050    Pre-fail  Offline      -       0
  3 Spin_Up_Time            0x0027   100   100   001    Pre-fail  Always       -       2054
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       491
  5 Reallocated_Sector_Ct   0x0033   100   100   050    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000b   100   100   050    Pre-fail  Always       -       0
  8 Seek_Time_Performance   0x0005   100   100   050    Pre-fail  Offline      -       0
  9 Power_On_Hours          0x0032   099   099   000    Old_age   Always       -       588
 10 Spin_Retry_Count        0x0033   109   100   030    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       355
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       1
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       123
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       6041
194 Temperature_Celsius     0x0022   100   100   000    Old_age   Always       -       30 (Min/Max 14/49)
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
220 Disk_Shift              0x0002   100   100   000    Old_age   Always       -       0
222 Loaded_Hours            0x0032   100   100   000    Old_age   Always       -       132
223 Load_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
224 Load_Friction           0x0022   100   100   000    Old_age   Always       -       0
226 Load-in_Time            0x0026   100   100   000    Old_age   Always       -       262
240 Head_Flying_Hours       0x0001   100   100   001    Pre-fail  Offline      -       0

SMART Error Log Version: 1
ATA Error Count: 12 (device log contains only the most recent five errors)
    CR = Command Register [HEX]
    FR = Features Register [HEX]
    SC = Sector Count Register [HEX]
    SN = Sector Number Register [HEX]
    CL = Cylinder Low Register [HEX]
    CH = Cylinder High Register [HEX]
    DH = Device/Head Register [HEX]
    DC = Device Command Register [HEX]
    ER = Error register [HEX]
    ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 12 occurred at disk power-on lifetime: 492 hours (20 days + 12 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 08 78 b3 dc 4d  Error: UNC 8 sectors at LBA = 0x0ddcb378 = 232567672

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  25 03 08 78 b3 dc 40 00      01:08:22.969  READ DMA EXT
  ef 03 45 78 b3 dc 00 00      01:08:22.969  SET FEATURES [Set transfer mode]
  ef 03 0c 78 b3 dc 00 00      01:08:22.969  SET FEATURES [Set transfer mode]
  ec 03 08 78 b3 dc 00 00      01:08:22.968  IDENTIFY DEVICE
  ff ff ff ff ff ff ff 0c      01:08:22.967  [VENDOR SPECIFIC]

Error 11 occurred at disk power-on lifetime: 492 hours (20 days + 12 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 41 00 78 b3 dc 40  Error: UNC at LBA = 0x00dcb378 = 14463864

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 08 00 78 b3 dc 40 00      01:08:19.211  READ FPDMA QUEUED
  25 03 08 78 b3 dc 40 00      01:08:15.457  READ DMA EXT
  ef 03 45 68 b3 dc 00 00      01:08:15.457  SET FEATURES [Set transfer mode]
  ef 03 0c 68 b3 dc 00 00      01:08:15.457  SET FEATURES [Set transfer mode]
  ec 03 08 68 b3 dc 00 00      01:08:15.456  IDENTIFY DEVICE

Error 10 occurred at disk power-on lifetime: 492 hours (20 days + 12 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 08 78 b3 dc 4d  Error: UNC 8 sectors at LBA = 0x0ddcb378 = 232567672

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  25 03 08 78 b3 dc 40 00      01:08:15.457  READ DMA EXT
  ef 03 45 68 b3 dc 00 00      01:08:15.457  SET FEATURES [Set transfer mode]
  ef 03 0c 68 b3 dc 00 00      01:08:15.457  SET FEATURES [Set transfer mode]
  ec 03 08 68 b3 dc 00 00      01:08:15.456  IDENTIFY DEVICE
  ff ff ff ff ff ff ff 0c      01:08:15.455  [VENDOR SPECIFIC]

Error 9 occurred at disk power-on lifetime: 492 hours (20 days + 12 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 41 00 78 b3 dc 40  Error: UNC at LBA = 0x00dcb378 = 14463864

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 08 00 78 b3 dc 40 00      01:08:11.708  READ FPDMA QUEUED
  60 08 00 70 b3 dc 40 00      01:08:11.686  READ FPDMA QUEUED
  25 03 08 68 b3 dc 40 00      01:08:11.668  READ DMA EXT
  ef 03 45 00 b3 dc 00 00      01:08:11.668  SET FEATURES [Set transfer mode]
  ef 03 0c 00 b3 dc 00 00      01:08:11.667  SET FEATURES [Set transfer mode]

Error 8 occurred at disk power-on lifetime: 492 hours (20 days + 12 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 41 08 78 b3 dc 40  Error: UNC at LBA = 0x00dcb378 = 14463864

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 00 e8 b3 dc 40 00      01:08:07.918  READ FPDMA QUEUED
  60 80 08 68 b3 dc 40 00      01:08:07.918  READ FPDMA QUEUED
  60 40 00 28 b3 dc 40 00      01:08:07.918  READ FPDMA QUEUED
  60 20 00 08 b3 dc 40 00      01:08:07.907  READ FPDMA QUEUED
  25 03 08 00 b3 dc 40 00      01:08:07.890  READ DMA EXT

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%       586         -
# 2  Extended offline    Completed without error       00%       500         -
# 3  Extended offline    Aborted by host               90%       499         -
# 4  Short offline       Completed without error       00%       499         -
# 5  Extended offline    Completed: read failure       00%       489         769438584
# 6  Extended offline    Completed: read failure       00%       488         769438584
# 7  Extended offline    Aborted by host               90%       486         -
# 8  Short offline       Completed without error       00%       486         -
# 9  Short offline       Completed without error       00%       112         -
2 of 2 failed self-tests are outdated by newer successful extended offline self-test # 1

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

Nota : the Power On Hours is 588

According to this link, the attribute Current Pending Sector count indicates the current number of unstable sectors :

user@host:~$ sudo smartctl -a -d sat /dev/sdg | grep "Current_Pending_Sector"
197 Current_Pending_Sector  0x0032   100   100   000    Old_age   Always       -       0

=> there would be none !

Finally ...

user@host:~$ date ; badblocks -svn /dev/sdg ; date
samedi 13 mars 2021, 16:31:58 (UTC+0100)
Vérification des blocs défectueux dans un mode non destructif de lecture-
écriture
Du bloc 0 au bloc 488386583
Vérification des blocs défectueux (test non destructif de lecture-écriture)
Test en cours avec un motif aléatoire : complété                                             
Passe complétée, 0 blocs défectueux repérés. (0/0/0 erreurs)
dimanche 14 mars 2021, 09:48:53 (UTC+0100)

=> about 17 hours later, badblocks announces that there is no error !

So I ask myself, how do I know if my drive is really HS and good for recycling ?

Does anyone have any expertise on the subject ?


I going,

I tried to long format (fulling with 0) with disk (Gnome disk utility) but at starting I saw the error "Error erasing device: Error writing 1048576 bytes to /dev/sdg: Input/output error (udisks-error-quark, 0)". So I tried to format with gparted and directly got "I/O error reading from /dev/sdg" (FR to EN translation) and when I tried to add a msdos partition table I had in add "Synchronization error / closing /dev/sdg : I/O error on target host" (FR to EN translation) then the disk disappeared completely from gparted ...

As smartctl reports everything is fine (as you assured me) as well as badblocks, I assumed the problem was not directly with the drive between it and the connector attached to the PC. So I disassembled the external drive case (the procedure was identical to this one) and tested the SATA drive with a SATA-USB docking station. I made the same tests and saw the same results.

I deduced that the problem came from the device itself and I asked to myself if there is not a low level utility to format the disk and clean to get a factory version. At one point, I gave up and could already see myself putting the record in recycling, suddenly as I have nothing to lose, broken for broken, I wanted to try everything possible and I found this topic which is related of the hdparm command ...


I studied on the surface topics 1 and 2. This site allowed me to find my hard drive password (subsequently stored in the PASS variable and which for a TOSHIBA disk consists of 32 spaces - yes, I too was surprised) which is different according to the manufacturer (adapt according to his). From these explanations, I stupidly applied without really understanding, the lines that seemed to work :

user@host:~$ PASS=$( printf %32s )
user@host:~$ sudo hdparm -I /dev/sdg
...
Security: 
    Master password revision code = 65534
        supported
        enabled
        locked
    not frozen
    not expired: security count
        supported: enhanced erase
    Security level high
    108min for SECURITY ERASE UNIT. 108min for ENHANCED SECURITY ERASE UNIT. 
...

user@host:~$ sudo hdparm --user-master u --security-set-pass "$PASS" /dev/sdg
security_password: "                                "

/dev/sdg:
 Issuing SECURITY_SET_PASS command, password="                                ", user=user, mode=high

user@host:~$ sudo hdparm -I /dev/sdg
...
Security: 
    Master password revision code = 65534
        supported
        enabled
        locked
    not frozen
    not expired: security count
        supported: enhanced erase
    Security level high
    108min for SECURITY ERASE UNIT. 108min for ENHANCED SECURITY ERASE UNIT. 
...

user@host:~$ date ; sudo hdparm --user-master u --security-erase "$PASS" /dev/sdg ; date
mardi 16 mars 2021, 19:22:49 (UTC+0100)
security_password: "                                "

/dev/sdg:
 Issuing SECURITY_ERASE command, password="                                ", user=user
mardi 16 mars 2021, 20:50:47 (UTC+0100)

user@host:~$ sudo hdparm -I /dev/sdg
...
Security: 
    Master password revision code = 65534
        supported
    not enabled
    not locked
    not frozen
    not expired: security count
        supported: enhanced erase
    108min for SECURITY ERASE UNIT. 108min for ENHANCED SECURITY ERASE UNIT. 
...

# enable -> not enabled AND locked -> not locked

user@host:~$ sudo hdparm --user-master m --security-unlock "$PASS" /dev/sdg
security_password: "                                "

/dev/sdg:
 Issuing SECURITY_UNLOCK command, password="                                ", user=master


user@host:~$ sudo hdparm --user-master m --security-disable "$PASS" /dev/sdg
security_password: "                                "

/dev/sdg:
 Issuing SECURITY_DISABLE command, password="                                ", user=master

I was pleasantly surprised to find that these commands had unlocked something (but I don't know what):

  • test gparted add MSDos partition table + NTFS formatting => OK
  • test Gnome disk (long formatting) => OK
  • test adding / deleting files / folders on Windows and Linux => OK

Warning: I am not too sure of the "non-destructive" character of hdparm (if the command can make a hard disk physically inaccessible, a bit like bricked smartphones). In any case, using the ABOVE commands, even if at first I used the wrong password (I took the dummy one that was given : "llformat"), something has changed or is unlocked (I can not say what). So to dig ...

So everything seems to be back to normal !

Does anyone knows what hdparm changed to make it work again ?

2
  • Please edit your question and include all the other SMART values (don't grep them away), because with those, we can assess how defective the disk is. All modern harddisks (and this is one) have spare sectors and automatically reallocate them, so badblocks is not a good tool to assess the health. – dirkt Mar 14 at 14:04
  • I did a grep so as not to overload my question too much. OK I add the rest ... – Le Nain Jaune Mar 14 at 14:21
0

From the SMART values, the drive looks very good (values are normed to 100, lower is worse). Zero raw read errors, no reallocated sectors.

So whatever happened during the two times you had problems could be as well something else. From your directory listing it looked like data was written, but not the correct data. Could be bad RAM where this sector was located, or an interrupted USB transfer, or some funny race condition in the firmware, or something completely different.

Note that this doesn't explain the bad LBA that happened at some time - I would have expected that sector to get reallocated, but maybe simply overwriting it when the harddisk was reformatted fixed it.

If you were using NTFS from Linux, my money is on a bug in the NTFS implementation - it used to be read-only for quite a long time, so apparently implementing writing NTFS correctly is not trivial.

So in that case, unless you need it to be NTFS for some reason, I'd change the format to ext4 etc. and see if the error still happens.


LBA means "logical block address". There are two offline tests which stopped at a bad sector with the same LBA:

Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
...
# 5  Extended offline    Completed: read failure       00%       489         769438584
# 6  Extended offline    Completed: read failure       00%       488         769438584

So this particular sector couldn't be read. However, it did not get reallocated (reallocation count is zero), hence my assumption that it got written during the reformat, which cured the read failure.


Disassembling the drive and testing with a different SATA-USB station is a good test; this means the problem is not in the internal SATA-USB adapter.

The ATA SECURE ERASE actually doesn't do a low level format, it will just write to every block. To my knowledge, there is no way to low level format modern drives.

It would have been interesting to look at the actual error messages in dmesg or /var/log/syslog when you tried to read and write the disk and it failed. If this was my drive, my next step would have been to use dd or the tools from sg3_utils to directly read and write the blocks. In particular, I'd have tried to write a block that cannot be read, to see if it gets reallocated.

But if you already started the SECURE ERASE, I guess it's too late for that. Please make sure to check the SMART values again after the SECURE ERASE has finished.

5
  • Thank you for your expertise. I have not detailed but it was several months that the same files were on this disk and that from time to time I added or deleted data. But like that from one day to the next, impossible to play certain files or to copy them. No explanations. In addition, I use an NTFS disk because I am 90% on Linux and 10% on Windows (if only for my partner but not only for her). So what you say to me is a very good news ... but i feel like I can't trust this device. It is very uncomfortable. – Le Nain Jaune Mar 14 at 19:23
  • Could you explain more what you say about "the bad LBA" ? – Le Nain Jaune Mar 14 at 19:25
  • Thank you in my despair I did not notice that the 2 read failures reported concerned the same sector – Le Nain Jaune Mar 17 at 7:42
  • I edited my first post to add my new investigations and their result. Everything seems to be back to normal, but I wonder if you can explain me the reason why using hdparm unlocked something to make my hard disk work again ? – Le Nain Jaune Mar 17 at 18:37
  • As I wrote, make sure to check the SMART values again. If the reallocated sector count has gone up, you had at least one sector that could not be read, but writing to it caused it to be reallocated and fixed it. If the reallocated sector count has not gone up, then I don't know what happened. One other option is that the read/write problems are actually a transport problem on USB/SATA... and unrelated to whatever happens on the harddisk itself. So you'll get randomly read/write errors. – dirkt Mar 17 at 18:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.