Showing posts with label hpc. Show all posts
Showing posts with label hpc. Show all posts

Sunday, March 20, 2016

How are lanes managed on PCIe 3.0 controller embedded on recent Xeon processors?

Leave a Comment

I'm using several PCIe 3.0 extension cards (GPUs and Infiniband interconnects). I'm wondering how lanes are actually managed and if I may optimize my devices by changing ports or by using some adpaters (16x -> 8x). Intel Haswell-EP may manage 40 lanes PCIe 3.0. On Intel's schematics, the PCIe 3.0 controller seems to be split in two x16 and one x8 sub-bridges.

Are all devices connected to a main PCIe bridge (and quantity of lanes automatically négociatiated for each device), or do the motherboard connect the devices directly to one of the supposedly 3 sub-bridges 16x, 16x and 8x (quantity of lane are then negociated for each of those sub-bridges)?

I do not have a direct access to the motherbard to see how devices are connected, but I suspect that the lanes of the supposedly 8x sub-bridge are not utilized. Also, I would like to know if by using a 16x to 8x adapter, I could harness more lanes and increase my total PCIe bandwidth (eventough the maximum theoretical bandwidth would be divided by two for that device).

0 Answers

Read More