I am working on my yocto distribution including cryptsetup in the 2.3.2 version

I am running such distribution on a board with 1 GB RAM and I am incurring in an "out of memory" error trying to open an encrypted partition that I am not able to properly debug. Any ideas?

My distro runs from an mSD with 3 partitions; the third one (30 MB) is the encrypted one.

I used the steps described on the ArchLinux guide to encrypt that partition, with ext3 instead of ext4

# cryptsetup -y -v luksFormat /dev/sda2
# cryptsetup open /dev/sda2 cryptroot
# mkfs.ext3 /dev/mapper/cryptroot

But trying to open that partition on my board raises an error:

cryptsetup --debug open /dev/mmcblk0p3  cryptroot
# cryptsetup 2.3.2 processing "cryptsetup --debug open /dev/mmcblk0p3 cryptroot"
# Running command open.
# Locking memory.
# Installing SIGINT/SIGTERM handler.
# Unblocking interruption on signal.
# Allocating context for crypt device /dev/mmcblk0p3.
# Trying to open and read device /dev/mmcblk0p3 with direct-io.
# Initialising device-mapper backend library.
# Trying to load any crypt type from device /dev/mmcblk0p3.
# Crypto backend (OpenSSL 1.1.1k  25 Mar 2021) initialized in cryptsetup library version 2.3.2.
# Detected kernel Linux 4.1.35-rt41 ppc.
# Loading LUKS2 header (repair disabled).
# Acquiring read lock for device /dev/mmcblk0p3.
# Opening lock resource file /run/cryptsetup/L_179:3
# Verifying lock handle for /dev/mmcblk0p3.
# Device /dev/mmcblk0p3 READ lock taken.
# Trying to read primary LUKS2 header at offset 0x0.
# Opening locked device /dev/mmcblk0p3
# Veryfing locked device handle (bdev)
# LUKS2 header version 2 of size 16384 bytes, checksum sha256.
# Checksum:43e122216ab19330fdfb6d2f9d7b586c4e5189884aef24be884e7159228e9ee5 (on-disk)
# Checksum:43e122216ab19330fdfb6d2f9d7b586c4e5189884aef24be884e7159228e9ee5 (in-memory)
# Trying to read secondary LUKS2 header at offset 0x4000.
# Reusing open ro fd on device /dev/mmcblk0p3
# LUKS2 header version 2 of size 16384 bytes, checksum sha256.
# Checksum:4ed9a44c22fde04c4b59a638c20eba6da3a13e591a6a1cfe7e0fec4437dc14cc (on-disk)
# Checksum:4ed9a44c22fde04c4b59a638c20eba6da3a13e591a6a1cfe7e0fec4437dc14cc (in-memory)
# Device size 32505856, offset 16777216.
# Device /dev/mmcblk0p3 READ lock released.
# Only 1 active CPUs detected, PBKDF threads decreased from 4 to 1.
# Not enough physical memory detected, PBKDF max memory decreased from 1048576kB to 255596kB.
# PBKDF argon2i, time_ms 2000 (iterations 0), max_memory_kb 255596, parallel_threads 1.
# Activating volume cryptroot using token -1.
# Interactive passphrase entry requested.
Enter passphrase for /dev/mmcblk0p3:
# Activating volume cryptroot [keyslot -1] using passphrase.
device-mapper: ioctl: 4.31.0-ioctl (2015-3-12) initialised: dm-devel@redhat.com
# dm version   [ opencount flush ]   [16384] (*1)
# dm versions   [ opencount flush ]   [16384] (*1)
# Detected dm-ioctl version 4.31.0.
# Device-mapper backend running with UDEV support enabled.
# dm status cryptroot  [ opencount noflush ]   [16384] (*1)
# Keyslot 0 priority 1 != 2 (required), skipped.
# Trying to open LUKS2 keyslot 0.
# Keyslot 0 (luks2) open failed with -12.
Not enough available memory to open a keyslot.
# Releasing crypt device /dev/mmcblk0p3 context.
# Releasing device-mapper backend.
# Closing read only fd for /dev/mmcblk0p3.
# Unlocking memory.
Command failed with code -3 (out of memory).

LUKS2 uses Argon2i key derivation function which is memory-hard -- meaning it requires a lot of memory to open the device to prevent (or at least make it harder) brute force attacks using GPUs. You can check how much memory you need to open your device using cryptsetup luksDump /dev/sda2, look for the line Memory: 755294 under Keyslots.

When creating the device, cryptsetup checks how much memory is available and adjusts the amount required for opening it accordingly, but if you did create the LUKS device from a different computer (for example when formatting the SD card on a desktop) or even on the same machine with more memory available, it's possible you simply don't have enough memory now. And we are talking only about RAM, swap is not used in this case.

I recommend re-creating the LUKS device with --pbkdf pbkdf2 to switch to the "old" (used to be default in LUKS1) key derivation function PBKDF2 which doesn't use extra memory. Alternatively you can also use --pbkdf-memory <num> to force lower amount of memory for the default Argon2i.

  • Solved changing the partition encrypting command, from cryptsetup open /dev/sda2 cryptroot to cryptsetup -y -v luksFormat --type luks1 /dev/sda2, so the "--type luks1" addition did the job. I suppose it was due to the fact that partition is created on a more powerful PC and the less powerful board could not decrypt it using che luks2 format
    – EagleOne
    May 4 at 6:52
  • I can't edit the comment but the first part of the command is wrong. I changed from cryptsetup -y -v luksFormat /dev/sda2 to cryptroot to cryptsetup -y -v luksFormat --type luks1 /dev/sda2 (I did not change the open command as a typographic error would make think), as I said, adding the --type luks1 directive
    – EagleOne
    May 4 at 10:18

Vojtech Trefny already explained that this happens because of Argon2i requiring too much memory. Indeed, from man cryptsetup:

For PBKDF2, only time cost (number of iterations) applies. For Argon2i/id, there is also memory cost (memory required during the process of key derivation) and parallel cost (number of threads that run in parallel during the key derivation).

But, as long as you have access to hardware with sufficient memory1, it is overkill to re-create the device.

Instead, using a computer with sufficient memory, just add a key to your encrypted device with pbkdf2,

cryptsetup luksAddKey -S 1 --pbkdf pbkdf2 /dev/sdxy

which assumes that the key slot 1 is free (you can find free key slots by inspecting cryptsetup luksDump /dev/sdxy).

Then, in your less powerful computer, unlock the device with

cryptsetup luksOpen -S 1 /dev/sdxy name

The -S 1 is essential, otherwise the more expensive key may be tried and the OOM killer triggered all the same.

1: Remember that you can always keep around a bootable USB with a minimal shell and core utilities (a Debian ISO cut it for me), and you will be able to boot it in most computers and access your encrypted volume.


System "freezes" are often caused by running too many, too large programs and running out of available memory. Use free to see if you have swap space, read man mkswap swapon fstab to create some. Traditionally, swap space of 1.5 × RAM has been recommended, but YMMV. If you don't plan to hibernate your system, you can have less than 1.0 × RAM.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.