3

Quoting the linux kernel documentation for boot parameters :

pcie_bus_perf : Set device MPS to the largest allowable MPS based on its parent bus. Also set MRRS (Max Read Request Size) to the largest supported value (no larger than the MPS that the device or bus can support) for best performance.

I fail to understand why the MRRS should not be larger than the MPS "for best performance".

I mean if a device can do MPS=X and MRRS=4.X then read requests could, in numbers, be less hence the bus less busy compared to a MRRS=X situation, even if the satisfaction of the request needs to be split in 4.

Would the split induce some significant overhead somewhere ?

BTW, I know the concept of "fair sharing" and understand the impact of large MRRS on that sharing but I never understood fair sharing synonymous to best performance.

1 Answer 1

3
+50

I hope you found the answer but I found some information for this which could help.

This kernel-mailing-list-discussion and this article are mentioning this issue and the explanation is that by setting the MRRS you are ensuring that the devices are not sending out read requests where the completion-packet-size (the answer) is bigger than the MPS of the device sending out the read request. If you ensure that, every node is able to have the MPS of the node above as its own MPS (or the highest supported by the device if it is lower than the MPS of the node above). So one node with a very low MPS cannot slow down the whole bus.

This schematic from the discussion helped me a lot to understand the problem:

normal:

        root (MPS=128)
         |
   ------------------
  /                  \
 bridge0 (MPS=128)    bridge1 (MPS=128)
/                      \
EP0 (MPS=128)           EP1 (MPS=128)

perf:

        root (MPS=256)
         |
   ------------------
  /                  \
 bridge0 (MPS=256)    bridge1 (MPS=128)
/                      \
EP0 (MPS=256)           EP1 (MPS=128)

Where every node is able to have a higher MPS than 128 bytes except the EP1.

3
  • No ! I had not progressed in my understanding. So thanks a lot for your finding. At least can I read that the issue was not obvious to many. So, If I understand correctly when the linux kernel documentation writes "no larger than the MPS that the device or bus can support for best performance." on should read for best write performances at the expense of read performances.
    – MC68020
    Commented Jul 6, 2022 at 22:08
  • No, it is the best performance overall. If you use the default-settings a very slow device could slow you down because every node has only the MPS of the slowest device. But if you use the perf-setting the nodes above the device and other devices are able to have higher MPS. I added a schema which helped me understanding where the perf aspect is.
    – Tom W.
    Commented Jul 7, 2022 at 10:03
  • Fair enough ! OK for "best overall write performances"… to the expenses of read performances on the slowest node. Anyway… more kudos coming for your contribution.
    – MC68020
    Commented Jul 7, 2022 at 10:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .