I am curious about how data is copied back and forth between CPU and GPU memories when I run a CUDA program. Specifically, I want to know how the Linux kernel is involved in this process.

I had a few possible assumptions:

  1. The CUDA user library sees the GPU as a file and calls read(2) and write(2) for every transaction.
  2. The CUDA user library asks Linux to mmap(2) the relevant control registers (DMA registers, PIO registers and some other MMIO registers) into the user space, and then operates the the GPU however it wants in user state.
  3. Something else?

I eliminated assumption #1 by running simple CUDA programs that copy data back and forth for 1000 times (and launches some empty kernels in between), where strace(1) did not observe any calls to write(2) and read(2).

Assumption #2 seem possible, since I observed with time(1) that the amount of data transfered seem to scale with user time instead of sys time. So the program seem to be copying data in user state.

But it seems a bit weird. How could a user program be allowed to manipulate such important I/O control registers by itself?

I would be grateful for some professional ideas on this topic.


The application asks the kernel to mmap a set of buffers at startup, creating this mapping is a privileged operation.

Normal operation just fills these buffers with data (such as textures, vertices or commands), and finally makes a single kernel call to start the submitted command queue. This start strobe is the only register access performed, everything else is shared memory.

The GPU has its own rudimentary MMU to make sure commands cannot reference data that belongs to another context, except where desired (e.g. a compositor that combines the render target from a game with the render target from an overlay and writes the result to the on-screen buffer).

For compute-only workloads, the same mechanism works fine, the command queue just doesn't end with "send data to screen" but with "return data to host".

  • After I start the commands, does the kernel need to perform any non-O(1) operations on the data buffer before handing it over to the DMA? (In other words, does the kernel state time scale with the data length transferred?) – Azuresonance Mar 16 at 7:08
  • 1
    No, validating the command buffer would take too long, so there is no trust boundary crossed for command buffer submission and there is no need to copy the data to a buffer inaccessible to the application, or access it at all. – Simon Richter Mar 16 at 7:27
  • Thank you. By the way, by any chance do you know which system call is used to start the commands? – Azuresonance Mar 16 at 8:58
  • 1
    I think it should be an ioctl call on /dev/dri/card0, but that is just a convention between the kernel driver and the userspace helper library loaded into the process, so there is no fixed interface. – Simon Richter Mar 16 at 9:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.