Versal CCIX-PCIe Module (CPM) Root port Linux driver

This page provides details for using the Root Port Linux Driver for the Versal Integrated Block for PCIe (CPM).

Table of Contents

Introduction

The Versal ACAP devices include CCIX-PCIe Module (CPM). The integrated block for CPM PCIe A along with the integrated bridge can function as PCIe Root Port with up to x16 Gen4 link configuration.

There are two integrated PCIe controllers (each capable of x8 maximum link width) and only one of them has access to the integrated bridge required for root mode. This page discusses the use of integrated PCIe controller with integrated bridge in CPM in root mode.

The integrated bridge has connectivity to memory via NoC and to PS via AXI-MM interface.

The interrupts (legacy/MSI/MSI-X) received from endpoints are routed to PS. The GIC Interrupt translation services (ITS) feature is enabled to process MSI/MSI-X interrupts.

Please refer PG346 and PG347 for more details.

Features Supported in HW

  • Supports ECAM for enumeration

  • Supports Non Prefetchable memory (32-bit)

  • Supports Prefetchable memory (64-bit)

  • Supports legacy interrupts

  • Supports MSI/MSI-X via GIC ITS feature

Known Issues:

Please refer AR 72972.

Features supported in driver

  • Enumeration of PCIe hirearchy

  • Assign Non Prefetchable memory

  • Assign Prefetchable memory

  • Handle legacy interrupts

Kernel Configuration

Enable CONFIG_PCIE_XDMA_PL option

Device tree

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 cpm_pciea:@fca10000 { #address-cells = <3>; #interrupt-cells = <1>; #size-cells = <2>; compatible = "xlnx,versal-cpm-host-1.00"; interrupt-map = <0 0 0 1 &pcie_intc_0 1>, <0 0 0 2 &pcie_intc_0 2>, <0 0 0 3 &pcie_intc_0 3>, <0 0 0 4 &pcie_intc_0 4>; interrupt-map-mask = <0 0 0 7>; interrupt-parent = <&gic>; interrupt-names = "misc"; interrupts = <0 72 4>; ranges = <0x02000000 0x00000000 0xE0000000 0x0 0xE0000000 0x00000000 0x10000000>, <0x43000000 0x00000080 0x00000000 0x00000080 0x00000000 0x00000000 0x80000000>; msi-map = <0x0 &its_gic 0x0 0x10000>; reg = <0x6 0x00000000 0x0 0x1000000>, <0x0 0xFCA10000 0x0 0x1000>; reg-names = "cfg", "cpm_slcr"; pcie_intc_0: pci-interrupt-controller { #address-cells = <0>; #interrupt-cells = <1>; interrupt-controller ; }; };

Tested End Points

  1. Samsung NVMe SSD

  2. Intel NVMe SSD

  3. Intel NIC card


Test Procedure

Note: This depends on end point functionality and use case.

The following test procedure shows NVMe device testing (Enable BLK_DEV_NVME in kernel configuration)

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 root@xilinx:~# lspci 00:00.0 PCI bridge: Xilinx Corporation Device a03f 01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981 root@xilinx:~# mount /dev/nvme0n1 /mnt/ root@xilinx:~# cd /mnt root@xilinx:~# dd if=/dev/zero of=tmp.txt bs=8192 count=200000 200000+0 records in 200000+0 records out root@xilinx:~# cat /proc/interrupts CPU0 CPU1 3: 2741 8093 GICv3 30 Level arch_timer 9: 0 0 GICv3 92 Level zynqmp-dma 10: 0 0 GICv3 93 Level zynqmp-dma 11: 0 0 GICv3 94 Level zynqmp-dma 12: 0 0 GICv3 95 Level zynqmp-dma 13: 0 0 GICv3 96 Level zynqmp-dma 14: 0 0 GICv3 97 Level zynqmp-dma 15: 0 0 GICv3 98 Level zynqmp-dma 16: 0 0 GICv3 99 Level zynqmp-dma 21: 0 0 GICv3 47 Level cdns-i2c 22: 0 0 GICv3 174 Level f12a0000.rtc 23: 0 0 GICv3 175 Level f12a0000.rtc 24: 0 0 GICv3 160 Level mmc0 25: 1114 0 GICv3 50 Level uart-pl011 27: 3 0 GICv3 157 Level f1030000.spi 28: 0 0 GICv3 104 Level xilinx-pcie 29: 0 0 GICv3 62 Level zynqmp_ipi 31: 10 0 ITS-MSI 524288 Edge nvme0q0 32: 730 0 ITS-MSI 524289 Edge nvme0q1 33: 0 1741 ITS-MSI 524290 Edge nvme0q2 IPI0: 1881 1446 Rescheduling interrupts IPI1: 1289 465 Function call interrupts IPI2: 0 0 CPU stop interrupts IPI3: 0 0 CPU stop (for crash dump) interrupts IPI4: 0 0 Timer broadcast interrupts IPI5: 0 0 IRQ work interrupts IPI6: 0 0 CPU wake-up interrupts Err: 0