Why is RAMFS much slower than Ram?

Question

I have a 64GB DDR4 3200MHz memory installed on my PC. When I run sysbench, I get the following results:

# sysbench memory --memory-block-size=1M --memory-total-size=10G run
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time


Running memory speed test with the following options:
  block size: 1024KiB
  total size: 10240MiB
  operation: write
  scope: global

Initializing worker threads...

Threads started!

Total operations: 10240 (26329.53 per second)

10240.00 MiB transferred (26329.53 MiB/sec)


General statistics:
    total time:                          0.3876s
    total number of events:              10240

Latency (ms):
         min:                                    0.04
         avg:                                    0.04
         max:                                    0.08
         95th percentile:                        0.04
         sum:                                  386.04

Threads fairness:
    events (avg/stddev):           10240.0000/0.00
    execution time (avg/stddev):   0.3860/0.00

It indicates it works at up to 26 GB/s. So far so good. But when I mount ramfs and try a similar test, this number drops considerably:

# mount -t ramfs -o size=11G ramfs /mnt/ramfs/
# dd if=/dev/zero of=/mnt/ramfs/zero.img bs=1G count=10  conv=fdatasync
10+0 records in
10+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 3.51899 s, 3.1 GB/s

Indicating I'm only getting around 3GB/s writing speed. I understand the file system has some overhead but a drop from 26GB/s to 3GB/s would be a really big overhead.

UPDATE 1 - Another test:

# time head -c 11G /dev/zero > /mnt/ramfs/zero.img

real    0m3.046s
user    0m0.225s
sys     0m2.808s

Am I doing something wrong? Is there a way to increase performance on RAMFS? Why is RAMFS so much slower than RAM itself?

Answer 1

While that factor of 8 is not that great, I'm not so surprised: your benchmark first allocates memory, and then accesses it. There's no file-system logic going on here, sure, you will need page faults to actually map that memory, but sysbench uses hugetlb to minimize the overhead of that and so on:

Writing to ramfs is really not a benchmark of your memory, but of your kernel's ability to quickly change entries in large tables and find free memory pages.