2

I tried everything I was able to find online. Hours of research since yesterday ;( I found no one struggling with the errors I'm facing, except from GitLab (error code -1 instead of -4 I'm getting), Reddit or this mailing list from 2006. I might give unnecessary details, sorry!

I have this 5 TB WD drive where I already have dozens of files. Decided to build a small NAS from a Raspberry Pi 4. Problem was I wanted LUKS encryption, with BTRFS as the file system; the drive was at that time a 5 TB one-partition EXT4.

I split the drive in 2 partitions (on my main computer) (only 2.3 TB was occupied), creating a LUKS protected BTRFS partition half the drive size: moved everything to the encrypted BTRFS partition, deleting the EXT4 part, growing LUKS, opening the encryption and then grew the BTRFS partition to fill the entire drive, passphrase still worked for LUKS, for a very long time. I thought nothing could happen when I have the LUKS Header Backup. The 5 TB LUKS-BTRFS partition is only protected with a passphrase, no additional slots etc. configured. I was able to unlock the drive and mount it now maybe for 3 weeks without any hiccups and error codes on all my devices (Artix-Linux x86_64, Linuxmint, Debian Aarch64, Parted Magic).

The operating system I chose for the Pi 4 was Debian, not Raspbian OS, since it was lacking the Crypto API/Functionality in the kernel I guess needed for serpent-xts-plain64, my drive encryption cipher. The NAS solution I went with was OpenMediaVault. It does not support unlocking LUKS volumes etc. on its own, so I unlocked it via SSH, mounted the device from the Web UI, created an SMB share, was even able to connect and exchange files for a day.

The other day when I wake up I notice when I connect to the SMB share there are no files?! A quick lsblk made clear the drive was not mounted, and the encryption was already closed. Mounting it was now impossible, tried many distros/kernels, architectures (aarch64 & amd64), tried mounting using GParted on many systems, KDE's own disk mounter etc. but no, guess I'm stuck. Funny thing is I was able to change the passphrase using cryptsetup luksChangeKey /dev/sdd1, it happily accepted my password, then successfully changed it to something else (as far as I know when I restore the header the old password is valid). Like I said before I have the LUKS header backup available, it's the right file I know it, since I heard restoring the wrong header makes things more complicated.

I hope I don't have to reinvent the wheel to decrypt the drive, but if it's necessary, I'll do it :/

As far as I can remember I did luksFormat using this command, it was inside my .zshrc:

cryptsetup -v luksFormat /dev/sdd1 --use-random --verify-passphrase --key-size=512 --hash=whirlpool --cipher=serpent-xts-plain64 --pbkdf=argon2id --type luks2

Here is the output of cryptsetup --debug --verbose luksOpen /dev/sdd1 crypt:

❯ sudo cryptsetup --debug --verbose luksOpen /dev/sdd1 crypt
[sudo] password for user: 
# cryptsetup 2.4.2 processing "cryptsetup --debug --verbose luksOpen /dev/sdd1 crypt"
# Running command open.
# Locking memory.
# Installing SIGINT/SIGTERM handler.
# Unblocking interruption on signal.
# Allocating context for crypt device /dev/sdd1.
# Trying to open and read device /dev/sdd1 with direct-io.
# Initialising device-mapper backend library.
# Trying to load any crypt type from device /dev/sdd1.
# Crypto backend (OpenSSL 1.1.1l  24 Aug 2021) initialized in cryptsetup library version 2.4.2.
# Detected kernel Linux 5.15.8-zen1-1-zen x86_64.
# Loading LUKS2 header (repair disabled).
# Acquiring read lock for device /dev/sdd1.
# Opening lock resource file /run/cryptsetup/L_8:49
# Verifying lock handle for /dev/sdd1.
# Device /dev/sdd1 READ lock taken.
# Trying to read primary LUKS2 header at offset 0x0.
# Opening locked device /dev/sdd1
# Verifying locked device handle (bdev)
# LUKS2 header version 2 of size 16384 bytes, checksum sha256.
# Checksum:cd57d8cf3e5d6bd82e34925c05ac3f84114d564dc1535d443d6003847ede9c03 (on-disk)
# Checksum:cd57d8cf3e5d6bd82e34925c05ac3f84114d564dc1535d443d6003847ede9c03 (in-memory)
# Trying to read secondary LUKS2 header at offset 0x4000.
# Reusing open ro fd on device /dev/sdd1
# LUKS2 header version 2 of size 16384 bytes, checksum sha256.
# Checksum:1fa2c8c216bef143a6841c7e6d7b1e737b39a832e3e8067ce580b103673c67b6 (on-disk)
# Checksum:1fa2c8c216bef143a6841c7e6d7b1e737b39a832e3e8067ce580b103673c67b6 (in-memory)
# Device size 5000946236928, offset 16777216.
# Device /dev/sdd1 READ lock released.
# PBKDF argon2id, time_ms 2000 (iterations 0), max_memory_kb 1048576, parallel_threads 4.
# Activating volume crypt using token (any type) -1.
# dm version   [ opencount flush ]   [16384] (*1)
# dm versions   [ opencount flush ]   [16384] (*1)
# Detected dm-ioctl version 4.45.0.
# Detected dm-crypt version 1.23.0.
# Device-mapper backend running with UDEV support enabled.
# dm status crypt  [ opencount noflush ]   [16384] (*1)
No usable token is available.
# Interactive passphrase entry requested.
Enter passphrase for /dev/sdd1: 
# Activating volume crypt [keyslot -1] using passphrase.
# dm versions   [ opencount flush ]   [16384] (*1)
# dm status crypt  [ opencount noflush ]   [16384] (*1)
# Keyslot 0 priority 1 != 2 (required), skipped.
# Trying to open LUKS2 keyslot 0.
# Running keyslot key derivation.
# Reading keyslot area [0x47000].
# Acquiring read lock for device /dev/sdd1.
# Opening lock resource file /run/cryptsetup/L_8:49
# Verifying lock handle for /dev/sdd1.
# Device /dev/sdd1 READ lock taken.
# Reusing open ro fd on device /dev/sdd1
# Device /dev/sdd1 READ lock released.
# Verifying key from keyslot 0, digest 0.
# Loading key (64 bytes, type logon) in thread keyring.
# dm versions   [ opencount flush ]   [16384] (*1)
# dm status crypt  [ opencount noflush ]   [16384] (*1)
# Calculated device size is 9767440351 sectors (RW), offset 32768.
# DM-UUID is CRYPT-LUKS2-355457dcd03343349b2121f41f3e0a5c-crypt
# Udev cookie 0xd4de97d (semid 4) created
# Udev cookie 0xd4de97d (semid 4) incremented to 1
# Udev cookie 0xd4de97d (semid 4) incremented to 2
# Udev cookie 0xd4de97d (semid 4) assigned to CREATE task(0) with flags DISABLE_LIBRARY_FALLBACK         (0x20)
# dm create crypt CRYPT-LUKS2-355457dcd03343349b2121f41f3e0a5c-crypt [ opencount flush ]   [16384] (*1)
# dm reload   (254:3) [ opencount flush securedata ]   [16384] (*1)
device-mapper: reload ioctl on crypt (254:3) failed: Invalid argument
# Udev cookie 0xd4de97d (semid 4) decremented to 1
# Udev cookie 0xd4de97d (semid 4) incremented to 2
# Udev cookie 0xd4de97d (semid 4) assigned to REMOVE task(2) with flags DISABLE_LIBRARY_FALLBACK         (0x20)
# dm remove crypt  [ opencount flush securedata ]   [16384] (*1)
# Uevent not generated! Calling udev_complete internally to avoid process lock-up.
# Udev cookie 0xd4de97d (semid 4) decremented to 1
# dm versions   [ opencount flush ]   [16384] (*1)
# dm status crypt  [ opencount noflush ]   [16384] (*1)
# Udev cookie 0xd4de97d (semid 4) decremented to 0
# Udev cookie 0xd4de97d (semid 4) waiting for zero
# Udev cookie 0xd4de97d (semid 4) destroyed
# Requesting keyring logon key for revoke and unlink.
# Releasing crypt device /dev/sdd1 context.
# Releasing device-mapper backend.
# Closing read only fd for /dev/sdd1.
# Unlocking memory.
Command failed with code -4 (wrong device or file specified).

The output of fdisk -l:

Disk /dev/sdd: 4.55 TiB, 5000947302400 bytes, 9767475200 sectors
Disk model: My Passport 2627
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 2505C284-7B8A-4EAE-90CB-950187A84D57

Device     Start        End    Sectors  Size Type
/dev/sdd1   2048 9767475166 9767473119  4.5T Linux filesystem

The luksDump, arrently needed too; output of cryptsetup luksDump /dev/sdd1:

❯ sudo cryptsetup luksDump /dev/sdd1
LUKS header information
Version:        2
Epoch:          5
Metadata area:  16384 [bytes]
Keyslots area:  16744448 [bytes]
UUID:           355457dc-d033-4334-9b21-21f41f3e0a5c
Label:          (no label)
Subsystem:      (no subsystem)
Flags:          (no flags)

Data segments:
  0: crypt
        offset: 16777216 [bytes]
        length: (whole device)
        cipher: serpent-xts-plain64
        sector: 4096 [bytes]

Keyslots:
  0: luks2
        Key:        512 bits
        Priority:   normal
        Cipher:     serpent-xts-plain64
        Cipher key: 512 bits
        PBKDF:      argon2id
        Time cost:  5
        Memory:     1048576
        Threads:    4
        Salt:       67 4b ad d5 89 b5 64 b7 b7 46 61 0f a4 9f cb be 
                    52 90 11 99 8c c0 fb 81 be 6a d6 ac 58 f5 3c 12 
        AF stripes: 4000
        AF hash:    sha256
        Area offset:290816 [bytes]
        Area length:258048 [bytes]
        Digest ID:  0
Tokens:
Digests:
  0: pbkdf2
        Hash:       whirlpool
        Iterations: 68985
        Salt:       d7 56 5e 8a d3 7c 7a 86 d3 fc b5 f8 d8 1e 6f 8d 
                    b3 fd 04 34 e7 08 ab 9a 33 92 2f 08 96 4b ff 74 
        Digest:     ed 9c d5 5f 0e df b3 f3 5b 71 95 09 9d f0 a8 b5 
                    9c a5 02 cb d0 1f f7 7b 52 d2 24 29 ee b2 7b 3f 
                    ed bc bd 1d f8 f7 bb 9f f7 c9 68 9b c9 be 86 66 
                    8b 24 5a 3c b7 b2 3e 93 7e d0 42 7c 7e e1 6d ec

S.M.A.R.T. values output using smartctl -a /dev/sdd:

❯ sudo smartctl -a /dev/sdd
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.15.8-zen1-1-zen] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Elements / My Passport (USB, AF)
Device Model:     WDC WD50NDZW-11MR8S1
Serial Number:    WD-WXD1E995WRAF
LU WWN Device Id: 5 0014ee 211f0443e
Firmware Version: 02.01A02
User Capacity:    5,000,947,523,584 bytes [5.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Form Factor:      2.5 inches
TRIM Command:     Available, deterministic
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-3 (minor revision not indicated)
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Fri Dec 17 16:02:40 2021 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      ( 249) Self-test routine in progress...
                                        90% of test remaining.
Total time to complete Offline 
data collection:                ( 2940) seconds.
Offline data collection
capabilities:                    (0x1b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        No Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 776) minutes.
SCT capabilities:              (0x30b5) SCT Status supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       2
  3 Spin_Up_Time            0x0027   253   253   021    Pre-fail  Always       -       4808
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       825
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   098   098   000    Old_age   Always       -       1577
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       321
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       176
193 Load_Cycle_Count        0x0032   198   198   000    Old_age   Always       -       6431
194 Temperature_Celsius     0x0022   119   098   000    Old_age   Always       -       33
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       1

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

Selective Self-tests/Logging not supported

And here's the DMESG output (simply dmesg, due to character limit couldn't post everything):

[   46.940566] wlan0: associated
[   46.989890] wlan0: Limiting TX power to 23 (23 - 0) dBm as advertised by 5c:49:79:56:19:f7
[   50.007552] usb 2-6: new SuperSpeed USB device number 2 using xhci_hcd
[   50.020426] usb 2-6: New USB device found, idVendor=1058, idProduct=2627, bcdDevice=40.08
[   50.020439] usb 2-6: New USB device strings: Mfr=2, Product=3, SerialNumber=1
[   50.020444] usb 2-6: Product: My Passport 2627
[   50.020448] usb 2-6: Manufacturer: Western Digital
[   50.020452] usb 2-6: SerialNumber: 575844314539393557524146
[   50.664550] usb-storage 2-6:1.0: USB Mass Storage device detected
[   50.665002] scsi host4: usb-storage 2-6:1.0
[   50.665220] usbcore: registered new interface driver usb-storage
[   50.676478] usbcore: registered new interface driver uas
[   51.678278] scsi 4:0:0:0: Direct-Access     WD       My Passport 2627 4008 PQ: 0 ANSI: 6
[   51.678667] scsi 4:0:0:1: Enclosure         WD       SES Device       4008 PQ: 0 ANSI: 6
[   51.682041] sd 4:0:0:0: [sdd] Spinning up disk...
[   51.703600] scsi 4:0:0:1: Wrong diagnostic page; asked for 1 got 8
[   51.703603] scsi 4:0:0:1: Failed to get diagnostic page 0x1
[   51.703605] scsi 4:0:0:1: Failed to bind enclosure -19
[   52.701886] ......ready
[   57.822064] sd 4:0:0:0: [sdd] Very big device. Trying to use READ CAPACITY(16).
[   57.822250] sd 4:0:0:0: [sdd] 9767475200 512-byte logical blocks: (5.00 TB/4.55 TiB)
[   57.822255] sd 4:0:0:0: [sdd] 4096-byte physical blocks
[   57.822540] sd 4:0:0:0: [sdd] Write Protect is off
[   57.822544] sd 4:0:0:0: [sdd] Mode Sense: 47 00 10 08
[   57.823041] sd 4:0:0:0: [sdd] No Caching mode page found
[   57.823048] sd 4:0:0:0: [sdd] Assuming drive cache: write through
[   57.983930]  sdd: sdd1
[   57.985534] sd 4:0:0:0: [sdd] Attached SCSI disk
[   57.985680] ses 4:0:0:1: Attached Enclosure device
[  137.355239] nvidia-nvlink: Nvlink Core is being initialized, major device number 507
[  137.355244] NVRM: The NVIDIA probe routine was not called for 1 device(s).
[  137.356116] NVRM: This can occur when a driver such as: 
               NVRM: nouveau, rivafb, nvidiafb or rivatv 
               NVRM: was loaded and obtained ownership of the NVIDIA device(s).
[  137.356117] NVRM: Try unloading the conflicting kernel module (and/or
               NVRM: reconfigure your kernel without the conflicting
               NVRM: driver(s)), then try loading the NVIDIA kernel module
               NVRM: again.
[  137.356118] NVRM: No NVIDIA devices probed.
[  137.356296] nvidia-nvlink: Unregistered the Nvlink Core, major device number 507
[  317.920451] device-mapper: table: 254:3: crypt: Device size is not multiple of sector_size feature
[  317.920455] device-mapper: ioctl: error adding target to table
[ 2685.464145] raid6: skip pq benchmark and using algorithm avx2x4
[ 2685.464148] raid6: using avx2x2 recovery algorithm
[ 2685.468011] xor: automatically using best checksumming function   avx       
[ 2685.528254] Btrfs loaded, crc32c=crc32c-intel, zoned=yes, fsverity=yes
[ 2685.564424] JFS: nTxBlock = 8192, nTxLock = 65536
[ 2685.582407] NILFS version 2 loaded
[ 2685.676402] SGI XFS with ACLs, security attributes, realtime, scrub, repair, quota, no debug enabled
[ 2692.757592]  sda: sda1 sda2 sda3 sda4
[ 2694.215474]  sdd: sdd1
[ 2768.779512] device-mapper: table: 254:3: crypt: Device size is not multiple of sector_size feature
[ 2768.779536] device-mapper: ioctl: error adding target to table
[ 3123.484363] usb 2-6: USB disconnect, device number 2
[ 4886.654141] usb 2-6: new SuperSpeed USB device number 3 using xhci_hcd
[ 4886.667772] usb 2-6: New USB device found, idVendor=1058, idProduct=2627, bcdDevice=40.08
[ 4886.667776] usb 2-6: New USB device strings: Mfr=2, Product=3, SerialNumber=1
[ 4886.667778] usb 2-6: Product: My Passport 2627
[ 4886.667779] usb 2-6: Manufacturer: Western Digital
[ 4886.667780] usb 2-6: SerialNumber: 575844314539393557524146
[ 4886.669555] usb-storage 2-6:1.0: USB Mass Storage device detected
[ 4886.669800] scsi host4: usb-storage 2-6:1.0
[ 4887.692812] scsi 4:0:0:0: Direct-Access     WD       My Passport 2627 4008 PQ: 0 ANSI: 6
[ 4887.693055] scsi 4:0:0:1: Enclosure         WD       SES Device       4008 PQ: 0 ANSI: 6
[ 4887.694634] ses 4:0:0:1: Attached Enclosure device
[ 4887.695784] sd 4:0:0:0: [sdd] Spinning up disk...
[ 4887.696087] ses 4:0:0:1: Wrong diagnostic page; asked for 1 got 8
[ 4887.696090] ses 4:0:0:1: Failed to get diagnostic page 0x1
[ 4887.696092] ses 4:0:0:1: Failed to bind enclosure -19
[ 4888.716288] ......ready
[ 4893.836679] sd 4:0:0:0: [sdd] Very big device. Trying to use READ CAPACITY(16).
[ 4893.836793] sd 4:0:0:0: [sdd] 9767475200 512-byte logical blocks: (5.00 TB/4.55 TiB)
[ 4893.836795] sd 4:0:0:0: [sdd] 4096-byte physical blocks
[ 4893.837071] sd 4:0:0:0: [sdd] Write Protect is off
[ 4893.837072] sd 4:0:0:0: [sdd] Mode Sense: 47 00 10 08
[ 4893.837383] sd 4:0:0:0: [sdd] No Caching mode page found
[ 4893.837385] sd 4:0:0:0: [sdd] Assuming drive cache: write through
[ 4893.996397]  sdd: sdd1
[ 4893.997502] sd 4:0:0:0: [sdd] Attached SCSI disk
[ 4951.411265] device-mapper: table: 254:3: crypt: Device size is not multiple of sector_size feature
[ 4951.411286] device-mapper: ioctl: error adding target to table

1 Answer 1

1

This is a problem with the device size of your partition.

Your partition is an odd number of 512-byte sectors large (9767473119 sectors as shown by fdisk). Your LUKS header is set to use 4096-byte sectors (sector: 4096 [bytes] shown by cryptsetup luksDump). So that leaves 7 sectors on the partition that can not be used.

Unfortunately, instead of just ignoring the surplus sectors, the device mapper crypt target takes offense, resulting in such error messages:

[ 8243.293778] device-mapper: table: 253:49: crypt: Device size is not multiple of sector_size feature (-EINVAL)
[ 8243.293781] device-mapper: ioctl: error adding target to table

In this case you have to make the partition size 4K aligned, i.e. a multiple of 8 512-byte sectors. You can do that with parted resizepart or any other partition tool of your choice. Just make sure the start sector of the partition does not change.

1
  • 1
    Thanks a lot, it really solved the problem and made it possible to mount the partition after doing resizepart <part-number> <multiplier-of-4096>s (after the number comes 's' without space which indicates that you entered a sector), but this method leads to an unmountable BTRFS partition. I solved that by again growing the shrinked partition-end in gparted and now it's mountable.
    – mercus
    May 29 at 1:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.