I'm experiencing very poor read performance over raid1/crypt/lvm. In the same time, write speeds are about 2x+ faster on the same setup. On another raid1 setup on the same machine I get normal read speeds (maybe because I'm not using cryptsetup).

OS related disks: sda + sdb. I have raid1 configuration with two disks, both are in place. I'm using LVM over the RAID. No encryption. Both disks are WD Green, 5400 rpm.

IO test results on this raid1:

dd if=/dev/zero of=/tmp/output.img3 bs=8k count=256k conv=fsync
 - 2147483648 bytes (2.1 GB) copied, 22.3392 s, 96.1 MB/s

echo 3 > /proc/sys/vm/drop_caches
dd if=/tmp/output.img3 of=/dev/null bs=8k
 - 2147483648 bytes (2.1 GB) copied, 15.9 s, 135 MB/s

And here is the problematic setup (on the same machine). Currently I have only one sdc (WD Green, 5400rpm) configured in software raid1 + crypt (luks, serpent-xts-plain) + lvm. Tomorrow I will attach another disk (sdd) to complete this two-disk raid1 setup.

IO tests results on this raid1:

dd if=/dev/zero of=output.img3 bs=8k count=256k conv=fsync
2147483648 bytes (2.1 GB) copied, 17.7235 s, 121 MB/s

echo 3 > /proc/sys/vm/drop_caches
dd if=output.img3 of=/dev/null bs=8k
2147483648 bytes (2.1 GB) copied, 36.2454 s, 59.2 MB/s

We can see that the read performance is very very bad (59MB/s compared to 135MB/s when using no encryption). Nothing is using the disks during benchmark. I can confirm this because I checked with iostat and dstat.

Details on the hardware:

  • disks: all are WD green, 5400rpm, 64mb cache.
  • cpu: FX-8350 at stock speed
  • ram: 4x4GB at 1066Mhz.

Details on the software:

  • OS: Debian Wheezy 7, amd64
  • mdadm: v3.2.5 - 18th May 2012
  • LVM version: 2.02.95(2) (2012-03-06)
  • LVM Library version: 1.02.74 (2012-03-06)
  • LVM Driver version: 4.22.0
  • cryptsetup: 1.4.3

Here is how I configured the slow raid1+crypt+lvm setup:

parted /dev/sdc

  1. mklabel gpt
  2. type: ext4
  3. start: 2048s
  4. end: -1

Now the raid, crypt and the lvm configuration:

  1. mdadm --create /dev/md1 --level=1 --raid-disks=2 missing /dev/sdc
  2. cryptsetup --cipher serpent-xts-plain luksFormat /dev/md1
  3. cryptsetup luksOpen /dev/md1 md1_crypt
  4. vgcreate vg_sql /dev/mapper/md1_crypt
  5. lvcreate -l 100%VG vg_sql -n lv_sql
  6. mkfs.ext4 /dev/mapper/vg_sql-lv-sql
  7. mount /dev/mapper/vg_sql-lv_sql /sql

So guys, can you help me identify the reason and fix it? It has to be something with the cryptsetup as there is no such read slowdown on the other setup (sda+sdb) where no encryption is present. But I have no idea what to do.


  • "Both disks are WD Green, 5400 rpm." That'll be your problem. May 27, 2014 at 15:04
  • 3
    I find it odd that WD Green drives are used but a power-hungry AMD is also used...
    – Nathan C
    May 27, 2014 at 15:08
  • That's just like having a bowl of french fries, and loads of ketchup, because ketchup comes from tomatoes, and they're healthy, right? May 27, 2014 at 15:25
  • 1
    What I expect is the crypted raid to be as fast on reading as the non-crypted raid. Why? Why would you expect that? May 27, 2014 at 15:38
  • 2
    @Android5360 Except that encryption (WRT symetric block ciphers) isn't slower than decryption. They're mathematically equivalent operations, the same speed in each direction. You're seeing the impact of caches, buffers and seek times - disk operations - not anything inherent in encryption of decryption (of symmetric block ciphers). May 27, 2014 at 15:54

2 Answers 2


It should be obvious that encryption adds considerable overhead, but disregarding that:

You're also using non-AES encryption which means you don't get the acceleration from your processor (AES hardware acceleration). You're also using a desktop-quality processor and extremely slow consumer drives.

This setup is exactly why you're having such poor performance. Put decent drives in along with a server-quality processor (is this even a server?) and use AES for LUKS.

  • Regarding that: decryption usually is faster than encryption, isn't it?..
    – poige
    May 27, 2014 at 15:00
  • @poige Not always as it depends on caching. Unlike reads, writes are cached by the drive's controller and reported written when the drive doesn't have the data written to it yet. Reads are cached by the OS, but the OP dropped those caches before reading. Encryption/decryption are typically similar in speeds.
    – Nathan C
    May 27, 2014 at 15:05
  • @poige Encryption is generally symetrical - equally fast in both directions. If you're experiencing faster decryption than encryption, it's probably actually an issue of your disk read being faster than your disk write. May 27, 2014 at 15:06
  • you guys awesome. serverfault.com/a/598838/67675
    – poige
    May 27, 2014 at 15:11
  • 1
    "Raid 1 is just a mirror, so performance isn't improved in this case" again you're terribly wrong. OMG, study the subject before trying to teach others. kernelnewbies.org/KernelProjects/Raid1ReadBalancing
    – poige
    May 27, 2014 at 16:16

It's usual not correctly done testing. Loading Linux Software RAID with 8k read operation is the way to ask it beahave poorly (you did). Try using larger bs.

I'm also adding this output of cryptsetup benchmark for ppl in comments below

#  Algorithm | Key |  Encryption |  Decryption
     aes-cbc   128b   476.6 MiB/s  1891.3 MiB/s
 serpent-cbc   128b    79.1 MiB/s   222.6 MiB/s
 twofish-cbc   128b   135.0 MiB/s   158.7 MiB/s
     aes-cbc   256b   194.7 MiB/s   892.5 MiB/s
 serpent-cbc   256b    46.2 MiB/s   211.4 MiB/s
 twofish-cbc   256b   144.6 MiB/s    93.3 MiB/s
     aes-xts   256b  1542.0 MiB/s  1727.0 MiB/s
 serpent-xts   256b   114.0 MiB/s   200.2 MiB/s
 twofish-xts   256b   111.4 MiB/s   131.7 MiB/s
     aes-xts   512b   573.6 MiB/s   829.3 MiB/s
 serpent-xts   512b   199.8 MiB/s   191.1 MiB/s
 twofish-xts   512b    81.8 MiB/s   107.3 MiB/s

-- clearly seen (not always, but often) decryption is faster.


You must log in to answer this question.