29

Following an unclean shutdown on an SD card based device, I took the SD card out to fsck the root filesystem. This led to variations on the following:

e2fsck 1.43.1 (08-Jun-2016)
/dev/sdc2: recovering journal
Superblock needs_recovery flag is clear, but journal has data.
Run journal anyway<y>? no
Clear journal<y>? no
e2fsck: unable to set superblock flags on /dev/sdc2

Here I've answered "no" both times but there is no sequence of yes/no that does not immediately lead to the same outcome.

The filesystem can be mounted and on casual inspection appears okay; it also works fine in the device, and that's the root filesystem (actually it turned out to be not quite fine, see comments; tldr some irretrievably corrupted directories).

I dd'd the partition (8 GB) to a file, and tried fsck on that. Interestingly:

e2fsck 1.43.1 (08-Jun-2016)
plush.rootfs: recovering journal
Clearing orphaned inode 18290 (uid=0, gid=0, mode=0100644, size=34096)
Clearing orphaned inode 18270 (uid=0, gid=0, mode=0100644, size=38916)
Clearing orphaned inode 18250 (uid=0, gid=0, mode=0100644, size=1128076)
Clearing orphaned inode 11411 (uid=0, gid=0, mode=0100644, size=293108)
Setting free inodes count to 406127 (was 408580)
Setting free blocks count to 1305622 (was 1347486)
plush.rootfs: clean, 60209/466336 files, 604906/1910528 blocks (check after next mount)

A subsequent fsck passed clean, the image can be mounted, and fsck -f after that passes as well.

But the filesystem on the card from which the raw block copy image was created still has the same problem -- except that the systemd-fsck which takes place during boot logs the filesystem as "clean". Subsequently though, a proper shutdown, taking the card out, and trying fsck again from another box presents the same error.

Whenever the original is mounted on another machine, syslog notes:

kernel: EXT4-fs (sdc2): 4 orphan inodes deleted
kernel: EXT4-fs (sdc2): recovery complete

Since I have it all backed up, I'm open to trying anything here. I could simply forget about this and reburn the partition from the apparently fixed image, but that does not seem like a very satisfactory solution, since it means assuming fsck cryptically failed at solving a minor looking problem.

I suspect this is going to turn into a "request for official documentation" question regarding stuff like needs recovery_flag (or just plain "What does this mean?" question), so any suggestions along those lines are appreciated.

6
  • Anything in the kernel logs about device errors? This wouldn't be the first time an SD card suddenly became read-only. Dec 3, 2016 at 23:22
  • @MarkPlotnick No, and it is writable. The last thing in the log from before the problem was systemd restarting (the device is headless and became unresponsive following a long apt upgrade). After that it logs a normal boot -- and the systemd-fsck says "clean" (I'll edit that in), but trying fsck outside of that context still fails.
    – goldilocks
    Dec 3, 2016 at 23:36
  • Your fsck on the copy cleared 4 inodes, but fixed the free inode count by reducing it by 2453 inodes! That's enormous. Check that the device is getting enough power.
    – meuh
    Dec 4, 2016 at 11:08
  • @meuh I notice whenever it's mounted on the big box syslog refers to those 4 inodes (edited in above). Some stuff on the fs did turn out to be messed up (updated kernel modules! \O/) so I've burned a new card and will hang on to the old one in case I have a chance to dig into this more. It isn't exactly new -- an unbranded bargain bin class 10 card, in (light duty) use 24/7 for possibly a few years so...I don't think there's any way to verify an SD card is definitively defunct, but I guess it could be that. The power should be okay but might be iffy under certain conditions.
    – goldilocks
    Dec 4, 2016 at 16:48
  • 2
    Doesn't it really suck when the very tool that's supposed to fix your problem, doesn't work because of the nature of the problem? Conclusion: The tool is bad and should be fixed.
    – Marc.2377
    Jul 7, 2017 at 2:42

5 Answers 5

30

I just ran into this same problem. After debugging the issue with the e2fsck maintainer, we realised that the SD card was broken. It was accepting writes without error, but it wasn't actually writing the data to the card. The SD card was effectively read only.

It seems the card had gone into some sort of failsafe mode, where the data could still be read, but nothing written.

The e2fsck message unable to set superblock flags means it tried to write to the superblock to mark the journal as processed, which happened without error, but when it went to read the superblock back again it still indicated that the journal needed to be replayed. In other words, the changes written to the superblock were not saved on the storage medium.

The card I am using that has this problem is a Samsung Evo 16GB microSD, which I mention just in case it's a common problem with these cards.

I was able to test this by using dd to write 4096 bytes from /dev/zero onto the card at block 0, then I read back from the card and instead of getting all zeroes as I should, I still got the original unchanged ext4 superblock.

I'm now in the process of moving the data onto a new card and then seeing if I can get a replacement from Samsung, who appear to offer a 10 year warranty on SD cards.

UPDATE: Samsung replaced the 16GB card with a 32GB one in the same Evo series, so guess I can't complain too much!

6
  • "where the data could still be read, but nothing written" -> The fs was writable.
    – goldilocks
    Oct 1, 2017 at 13:05
  • 2
    @goldilocks: Sounds like your fs superblock may not have been writable though. Also, my fs appeared writable thanks to caching, it was only after unmounting and remounting I noticed any changes were lost.
    – Malvineous
    Oct 2, 2017 at 1:27
  • It was not an illusion due to caching.
    – goldilocks
    Oct 2, 2017 at 12:57
  • 1
    amazing: I have the same issue, with the same card ! looks like it is "read only" from now on, that is weird for sure
    – MoonCactus
    Jun 20, 2020 at 19:37
  • 1
    @Malvineous thanks, this is good to know! I bought from Samsung specifically for the hope I was reducing such a risk, and it ends up the other way. It gets statistically significant enough for me to try something else... but what? May be it was only a batch of these 16GB cards... There might be no extrapolation to be made from one product to another even in the same brand!
    – MoonCactus
    Jun 23, 2020 at 13:24
16

I know this is an old thread, but, I figured I'd offer some insight.

This seems to be the way sd cards die a natural death. The number of read/write cycles sd cards can endure is substantially lower than most other mediums considered 'read/write'. When that has been exhausted, the card will go into read only mode, but won't inform you of such. Lots of things will think they're writing to the card thanks to OS caching, etc., but nothing ever sticks.

A great way to kill an sd card is to mount it as a swap partition or something that's very read/write intensive. You'd be surprised how quickly you can kill a card that way. I've found that running knoppix off of an sd card or usb thumb drive will only last a month or two, depending on the quality of the card and the intensity of knoppix usage. (I've since switched to running knoppix off of a usb SSD drive which has lasted a couple of years now).

2
  • 2
    If you can, please provide a source for it. It would be great.
    – Lyokolux
    Jun 22, 2021 at 12:46
  • It's interesting because after the first Samsung SD card did this for me, I made sure to mount the next one read-only, so that the OS would never write anything to it. Even with no writes beyond the initial set up, the cards still went read-only after a similar amount of time. I'm not really sure what would explain that behaviour, but it certainly wasn't exhausting the number of write cycles. On the plus side, the Raspberry Pi each card was in would still boot and run just fine, because even though the card was read-only, it was configured to be mounted read-only too, so it wasn't a problem.
    – Malvineous
    Mar 30, 2023 at 7:02
2

For me, the problem disappeared after using the card with another microSD-to-USB adapter. Then it worked fine with the first adapter as well. I figure in my case it was a bad contact issue.

1

Had the same problem, unable to mount, unable to read superblock, unable to set superblock flags, unable to use superblock backup.

The problem I had was write lock switch on the card adapter ;)

-1

Twice now I've had similar problems for a microSD and it turned out to be the SD adapter, not the microSD. Changing to a different SD adapter, even when the first one was the same brand as the microSD, fixed the problem of being unable to write superblock flags, and having I/O errors.

2
  • 1
    How is this different from sizif's answer? Dec 24, 2022 at 1:57
  • Similar, but the adapter I was using was to SD, not USB
    – brianc
    Jan 3, 2023 at 16:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .