How can my PCI device driver remap PCI memory to userspace?

Question

I am trying to implement a PCI device driver for a virtual PCI device on QEMU. The device defines a BAR region as RAM, and the driver can do ioremap() this region and access it without any issues. The next step is to assign this region (or a fraction of it) to a user application. To do this, I have also implemented an .mmap function as part of my driver file operations. This mmap is simply using remap_pfn_range, but it also passes the pfn of the memory pointer returned by the ioremap() earlier.

However, upon running the user space application, the mmap is successful, but when the app tries to access the memory, it is killed and I get the following dmesg errors.

[ 1502.402970] a.out: Corrupted page table at address 7f911b79f000

[ 1502.404085] PGD 13926d067 P4D 13926d067 PUD 1317aa067 PMD 1326d9067 PTE 800026d901000227

[ 1502.404085] Bad pagetable: 000f [#1] SMP NOPTI

[ 1502.404085] Modules linked in: edu_driver(OE) ppdev kvm_amd kvm irqbypass input_leds parport_pc serio_raw parport mac_hid qemu_fw_cfg sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear psmouse pata_acpi floppy e1000 i2c_piix4

[ 1502.404085] CPU: 0 PID: 1988 Comm: a.out Tainted: G OE 4.15.0-55-generic #60-Ubuntu

[ 1502.404085] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014

[ 1502.404085] RIP: 0033:0x55d687642811 [ 1502.404085] RSP: 002b:00007ffe16c38da0 EFLAGS: 00000213

[ 1502.404085] RAX: 00007f911b79f000 RBX: 0000000000000000 RCX: 00007f911b2a1813

[ 1502.404085] RDX: 0000000000000003 RSI: 0000000000001000 RDI: 0000000000000000

[ 1502.404085] RBP: 00007ffe16c38dc0 R08: 0000000000000003 R09: 0000000000000000

[ 1502.404085] R10: 0000000000008001 R11: 0000000000000246 R12: 000055d687642660

[ 1502.404085] R13: 00007ffe16c38ea0 R14: 0000000000000000 R15: 0000000000000000

[ 1502.404085] FS: 00007f911b7984c0(0000) GS:ffff97237fc00000(0000) knlGS:0000000000000000

[ 1502.404085] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

[ 1502.404085] CR2: 00007f911b79f000 CR3: 0000000132cd8000 CR4: 00000000000006f0

[ 1502.404085] RIP: 0x55d687642811 RSP: 00007ffe16c38da0

[ 1502.404085] ---[ end trace 6b088b58eb816baf ]---

Does anyone know what have I done wrong? Did I missed a step? Or it could be an error specific to QEMU? I am running x86_softmmu as my QEMU configuration and my kernel is the 4.14

Answer 1

I've solved this issue and managed to map PCI memory to userspace via the driver. I've changed the pfn input of the remap_pfn_range function I was using in my custom .mmap

The original was:

io_remap_pfn_range(vma, vma->vm_start, pfn, vma->vm_end - vma->vm_start, vma->vm_page_prot ))

where the pfn was the result of the buffer pointer return from the ioremap(). I changed the pfn to:

pfn = pci_resource_start(pdev, BAR) >> PAGE_SHIFT .

That basically points to the actual starting address pointed by the BAR.

My working remap_pfn_range function is now:

io_remap_pfn_range(vma, vma->vm_start, pci_resource_start(pdev, BAR) >> PAGE_SHIFT, vma->vm_end - vma->vm_start,vma->vm_page_prot )

I confirmed that it works by doing some dummy writes to the buffer pointer in my driver, then picking up the reads and doing some wirtes in my userspace app.