1

EDIT: Is there a better stackexchange site I should put this on?

I am trying use the DDR4 connected to the FPGA side of a zynq ultrascale chip, specifically on Xilinx's devboard zcu102. I am using Vivado's MIG to generate the controller and connect to the HPM0 AXI port of the main processor. The memory is 0x2000_0000 long and mapped to physical address 0x4_0000_0000.

The problem I am having is when accessing memory unaligned. I believe it to be some misconfiguration in the MMU due to a missing detail in the device tree.

One thing I know is that this arm (A53) is supposed to be able to turn unaligned accesses into multiple access to the memory where necessary. My getting a bus error is telling me that the MMU is misconfigured in some way but I'm not sure how.

The device tree is too long to put here, so I will put what I think are the most important parts directly in here and the rest as a pastebin.

    amba_pl@0 {
            #address-cells = <0x2>;
            #size-cells = <0x2>;
            compatible = "simple-bus";
            ranges;

            ddr4@400000000 {
                    compatible = "xlnx,ddr4-2.2";
                    reg = <0x4 0x0 0x0 0x20000000>;
                    alignment = <0x10>;
            };

            PERIPHERAL@ff380000 {
                    compatible = "xlnx,PERIPHERAL-1.0";
                    reg = <0x0 0xff380000 0x0 0x80000>;
            };

            PERIPHERAL@ff990000 {
                    compatible = "xlnx,PERIPHERAL-1.0";
                    reg = <0x0 0xff990000 0x0 0x10000>;
            };
    };

    aliases {
            ethernet0 = "/amba/ethernet@ff0e0000";
            i2c0 = "/amba/i2c@ff020000";
            i2c1 = "/amba/i2c@ff030000";
            serial0 = "/amba/serial@ff000000";
            serial1 = "/amba/serial@ff010000";
            spi0 = "/amba/spi@ff0f0000";
    };

    memory {
            #address-cells = <0x2>;
            #size-cells = <0x2>;
            device_type = "memory";
            alignment = <0x10>;
            reg = <0x4 0x0 0x0 0x20000000>;
    };

I tried setting the alignment on both /memory and the ddr controller on the /amba_pl, but it doesn't seem to make a difference. In fact, it seems to be ignoring the /memory section, as it still boots off of main system memory and not the memory on the fpga.

I am able to access memory using mmap and devmem as such:

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <unistd.h>
#include <stdint.h>
#include <math.h>

#define ARRAY_TYPE uint16_t

int main(int argc, char *argv[]) {
    if (argc != 2){
        printf("Wrong arg count\n");
        return -1;
    }
    volatile ARRAY_TYPE * ps = malloc(sizeof(ARRAY_TYPE)*10);
    if (NULL==ps){
        printf("Failed to malloc for ps\n");
        return -1;
    }


    int fd = open("/dev/mem", O_RDWR | O_SYNC );
    volatile ARRAY_TYPE *pl = mmap((void*)0x400000000, 0x20000000, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0x400000000);
    if (pl == MAP_FAILED) {
        perror("Can't map memory for pl");
        printf("FAIL\n");
        return -1;
    }


    //force them out of alignment by argv[1] bytes.
    volatile ARRAY_TYPE *ps_unaligned = (volatile ARRAY_TYPE *)(((uint8_t *)ps) + atoi(argv[1]));
    volatile ARRAY_TYPE *pl_unaligned = (volatile ARRAY_TYPE *)(((uint8_t *)pl) + atoi(argv[1]));

    printf("trying ps with offset %d\n", atoi(argv[1]));
    ps_unaligned[0] = 1;
    printf("trying pl with offset %d\n", atoi(argv[1]));
    pl_unaligned[0] = 1;

    munmap(pl, 0x20000000);
    free(ps);
    close(fd);

    return 0;
}

Whenever I try to access memory out of alignment on the main system memory it works fine. However when I access the memory hanging off the FPGA unaligned I get a bus error.

root@zynq: ~
17:22:08 $ ./memtest 0
trying ps with offset 0
trying pl with offset 0
root@zynq: ~
17:22:12 $ ./memtest 1 
trying ps with offset 1
trying pl with offset 1
Bus error
root@zynq: ~
17:22:13 $ ./memtest 2
trying ps with offset 2
trying pl with offset 2
root@zynq: ~
17:22:14 $ ./memtest 3
trying ps with offset 3
trying pl with offset 3
Bus error

I've also looked at the virtual memory mappings of the memory region by looking at /proc/$pid/smaps during a gdb session: Here's the section containing the system memory allocated with devmem/mmap:

400000000-420000000 rw-s 400000000 00:06 1054                            /dev/mem
Size:             524288 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:                   0 kB
Pss:                   0 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:         0 kB
Referenced:            0 kB
Anonymous:             0 kB
LazyFree:              0 kB
AnonHugePages:         0 kB
ShmemPmdMapped:        0 kB
Shared_Hugetlb:        0 kB
Private_Hugetlb:       0 kB
Swap:                  0 kB
SwapPss:               0 kB
Locked:                0 kB
VmFlags: rd wr sh mr mw me ms pf io de dd 

Here's the section for the memory mapped from the FPGA:

5555567000-5555588000 rw-p 00000000 00:00 0                              [heap]
Size:                132 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:                   4 kB
Pss:                   4 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:         4 kB
Referenced:            4 kB
Anonymous:             4 kB
LazyFree:              0 kB
AnonHugePages:         0 kB
ShmemPmdMapped:        0 kB
Shared_Hugetlb:        0 kB
Private_Hugetlb:       0 kB
Swap:                  0 kB
SwapPss:               0 kB
Locked:                0 kB
VmFlags: rd wr mr mw me ac 

I'm not sure if any of the differences in flags explains what is going on but I'm including them in case someone more proficient in memory mapping knows something.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.