Linux Page Cache Performance versus memcpy

Question

I am benchmarking the write performance in the case that Linux writes the data to the page cache without throttling the process or syncing the data to the disk. The simple experiment I am doing looks like the following:

long size = 1024;
int fd = open ("file", O_CREAT | O_RDWR | O_TRUNC, ...);
char *buffer = (char *) malloc (size)
**start_time = time.now ();**
write (fd, buffer, size);
**end_time = time.now ();**
close (fd);
printf ("write duration %d\n", end_time - start_time);

Here I expect to effectively observe the memory bandwidth since the data is only copied to the page cache of the OS, and nothing is synced to the disk. Also, the OS has not started with the background flushes or throttling the process, since, the dirty rate is way lower than the background_dirty_ratio. Though, when I compare it to the memory bandwidth (memcpy cost of the same size), it is way more expensive than a memcpy:

char *buffer1 = (char *) malloc (size)
char *buffer2 = (char *) malloc (size)
**start_time = time.now ();**
memcpy (buffer1, buffer2, size);
**end_time = time.now ();**
printf ("memcpy duration %d\n", end_time - start_time);

For instance on my system (Linux kernel version 4.2, CentOS), I see a memcpy bandwidth of almost 60GB/s, and the write bandwidth is almost 2GB/s. To my understanding, when write system call is invoked, it merely copies the data in the page cache (in memory) and returns as soon as the copy is done. So I expect to see a bandwidth near to memory bandwidth. I also tested the same experiment with larger data (before the OS starts to throttle the process) to mitigate the cost of making the system call. But still, I see almost the same results. Does anyone know why I am not observing the memory bandwidth when performing write on the page cache?

Answer 1

The main factor here is that memcpy is handled by the C library, or even directly by the C compiler, whereas write is a system call, handled by the kernel.

So your memcpy runs in-process, possibly without even the (tiny) overhead of a function call. write, on the other hand, has all the overhead of a system call. In your test especially, since the write size is small, the cost of the copy itself is probably dwarfed by the cost of the system call. Even with your larger tests, it’s likely that the system call cost is still the main factor.

To reduce the weight of the system calls, try comparing with even larger sizes, changing the dirty ratio configurations if necessary, or writing to a tmpfs to avoid the cost of writing to a disk. You might also want to disable KPTI and other mitigations which increase the cost of system calls, and perhaps look into io_uring.

To get a better idea of the work involved in handling a write, you can trace the appropriate system call, __x64_sys_write on x86-64; this will show the call chain and time spent, e.g.

  9)               |  __x64_sys_write() {
  9)               |    ksys_write() {
  9)               |      __fdget_pos() {
  9)               |        __fget_light() {
  9)   0.465 us    |          __fget_files();
  9)   0.943 us    |        }
  9)   1.155 us    |      }
  9)               |      vfs_write() {
  9)               |        rw_verify_area() {
  9)               |          security_file_permission() {
  9)               |            selinux_file_permission() {
  9)               |              __inode_security_revalidate() {
  9)               |                _cond_resched() {
  9)   0.034 us    |                  rcu_all_qs();
  9)   0.241 us    |                }
  9)   0.649 us    |              }
  9)               |              file_has_perm() {
  9)   0.034 us    |                bpf_fd_pass();
  9)               |                inode_has_perm() {
  9)   0.133 us    |                  avc_has_perm();
  9)   0.376 us    |                }
  9)   0.808 us    |              }
  9)   2.126 us    |            }
  9)   0.032 us    |            bpf_lsm_file_permission();
  9)   2.616 us    |          }
  9)   2.815 us    |        }
  9)               |        __vfs_write() {
  9)               |          new_sync_write() {
  9)               |            pipe_write() {
  9)               |              mutex_lock() {
  9)               |                _cond_resched() {
  9)   0.034 us    |                  rcu_all_qs();
  9)   0.236 us    |                }
  9)   0.566 us    |              }
  9)               |              _cond_resched() {
  9)   0.036 us    |                rcu_all_qs();
  9)   0.232 us    |              }
  9)   0.036 us    |              mutex_unlock();
  9)               |              __wake_up_sync_key() {
  9)               |                __wake_up_common_lock() {
  9)   0.036 us    |                  _raw_spin_lock_irqsave();
  9)               |                  __wake_up_common() {
  9)               |                    pollwake() {
  9)               |                      default_wake_function() {
  9)               |                        try_to_wake_up() {
  9)   0.178 us    |                          _raw_spin_lock_irqsave();
  9)               |                          select_task_rq_fair() {
  9)   0.033 us    |                            available_idle_cpu();
  9)   0.032 us    |                            available_idle_cpu();
  9)   0.040 us    |                            cpus_share_cache();
  9)   0.058 us    |                            available_idle_cpu();
  9)   1.061 us    |                          }
  9)   0.036 us    |                          ttwu_queue_wakelist();
  9)   0.036 us    |                          _raw_spin_lock();
  9)   0.079 us    |                          update_rq_clock();
  9)               |                          ttwu_do_activate() {
  9)               |                            enqueue_task_fair() {
  9)               |                              enqueue_entity() {
  9)   0.040 us    |                                update_curr();
  9)   0.088 us    |                                __update_load_avg_se();
  9)   0.070 us    |                                __update_load_avg_cfs_rq();
  9)   0.032 us    |                                update_cfs_group();
  9)   0.055 us    |                                __enqueue_entity();
  9)   1.347 us    |                              }
  9)               |                              enqueue_entity() {
  9)   0.038 us    |                                update_curr();
  9)   0.077 us    |                                __update_load_avg_se();
  9)   0.050 us    |                                __update_load_avg_cfs_rq();
  9)               |                                update_cfs_group() {
  9)   0.047 us    |                                  reweight_entity();
  9)   0.289 us    |                                }
  9)   0.035 us    |                                __enqueue_entity();
  9)   1.469 us    |                              }
  9)   0.033 us    |                              hrtick_update();
  9)   3.546 us    |                            }
  9)               |                            ttwu_do_wakeup() {
  9)               |                              check_preempt_curr() {
  9)   0.046 us    |                                resched_curr();
  9)   0.279 us    |                              }
  9)   0.671 us    |                            }
  9)   4.653 us    |                          }
  9)   0.038 us    |                          _raw_spin_unlock_irqrestore();
  9)   7.458 us    |                        }
  9)   7.652 us    |                      }
  9)   7.924 us    |                    }
  9)   8.865 us    |                  }
  9)   0.045 us    |                  _raw_spin_unlock_irqrestore();
  9)   9.501 us    |                }
  9)   9.703 us    |              }
  9)   0.033 us    |              kill_fasync();
  9)   0.055 us    |              __sb_start_write();
  9)               |              file_update_time() {
  9)               |                current_time() {
  9)   0.037 us    |                  ktime_get_coarse_real_ts64();
  9)   0.039 us    |                  timestamp_truncate();
  9)   0.454 us    |                }
  9)               |                __mnt_want_write_file() {
  9)   0.057 us    |                  __mnt_want_write();
  9)   0.289 us    |                }
  9)               |                generic_update_time() {
  9)   0.089 us    |                  __mark_inode_dirty();
  9)   0.321 us    |                }
  9)   0.039 us    |                __mnt_drop_write_file();
  9)   1.904 us    |              }
  9)   0.037 us    |              __sb_end_write();
  9) + 14.315 us   |            }
  9) + 14.620 us   |          }
  9) + 14.840 us   |        }
  9)   0.166 us    |        __fsnotify_parent();
  9)   0.095 us    |        fsnotify();
  9) + 18.702 us   |      }
  9)               |      fput() {
  9)   0.035 us    |        fput_many();
  9)   0.238 us    |      }
  9) + 20.668 us   |    }
  9) + 20.907 us   |  }

This is a rather extreme example but it illustrates that the system call can end up doing an awful lot more than copying memory.

Answer 2

If you add appropriately a call to sync(2) or to fsync(2) in your code you should expect the Linux kernel to send some data to your disk.

But in 2022 disks are often SSD, and the disk itself has some buffer....

See also time(7), sd(4), use clock_gettime(2), and probably some (unknown to me) ioctl(2) or fcntl(2) or io_submit(2) to force your hard disk to write data at the physical level....

Maybe you'll need to code some kernel module....