Solution Versal PL Programming

Introduction‍

The Versal Programmable Logic (PL) can be programmed either using U-Boot or through Linux.
This page provides the details about programming the PL from Linux world using Linux FPGA Manager framework.

Flow:

 

 

HW IP Features

  • Full Bitstream and Partial Bitstream/PDI loading

  • Encrypted and Authenticated Full/Partial Bitstream/PDI loading

  • Readback of Configuration Data

Features supported in the driver

  • Full Bitstream Support (DFX flow)

    • Non Secure Bitstream/PDI loading 

    • Encrypted  Bitstream/PDI loading with device-key

    • Authenticated Bitstream/PDI 

    • Authenticated and Encrypted Bitstream/PDI with device-key

  • Partial Bitstream Support 

    • Non Secure Bitstream/PDI loading 

    • Encrypted Partial Bitstream/PDI loading with device-key

    • Authenticated Partial Bitstream/PDI

    • Authenticated and Encrypted Partial Bitstream/PDI with device-key

Unsupported features in the driver

  • Full Bitstream Support 

    • Encrypted  Bitstream/PDI loading with user-key

    • Authenticated and Encrypted Bitstream/PDI  loading with User-key

  • Partial Bitstream Support 

    • Encrypted Partial Bitstream/PDI with user-key

    • Authenticated and Encrypted Partial Bitstream/PDI  with user-key

  •  Read Back Support

    • Configuration Data Readback.

Missing Features, Known Issues and Limitations in the driver

The Xilinx device tree generator (https://github.com/Xilinx/device-tree-xlnx ) currently lacks automated support for device tree overlay generation for Partial Reconfiguration / DFX designs.  For systems requiring runtime device tree overlay (eg, Linux device driver support) support, hand-crafted device trees can be deployed and loaded. See the "Working with Device Tree Overlay (DTBO)" section below for details on how to load these at runtime.
Few considerations that user should be aware of in programming the PL are as follows

  • For the Versal platform, it is expected that the base PDI will always be part of BOOT PDI and will be loaded by PLM at boot time. Later from Linux, users can load either the Full/partial PDI from Linux. The term Full bitstream/PDI in Versal implies the whole PL region programming except the configuration that's part of base PDI.

  • Using overlays we can add a new node or add/update the existing node properties. But it will not allow replacing the existing nodes which are already part of the live tree. 

  • If the PL design is having axi-intc IP, the relevant DT node should be the top node in the overlay. dtsi or pl.dtsi file before its actually being referenced in any other nodes.

    • Also, IRQCHIP Xilinx Intc driver module support must be enabled in the kernel configuration.x

  • Apply overlay change sets will occur from top to bottom and Removal of overlay change sets will occur in the opposite order of apply(the most recently applied overlay change set must be removed first). So if the overlay has any internal dependency one on another those need to be taken care of while creating the overlay files. (Like: Interrupt controllers, clock nodes.....etc).

Example DT node:

/dts-v1/; /plugin/; / { fragment@0 { target = <&fpga>; __overlay__ { #address-cells = <2>; #size-cells = <2>; firmware-name = "system.pdi"; }; }; fragment@1 { target = <&amba>; __overlay__ { #address-cells = <2>; #size-cells = <2>; axi_intc_1: interrupt-controller@a00d0000 { /* Top node */ #interrupt-cells = <2>; clock-names = "s_axi_aclk"; clocks = <&clk_wiz_0 0>; compatible = "xlnx,axi-intc-4.1", "xlnx,xps-intc-1.00.a"; interrupt-controller ; interrupt-names = "irq"; interrupt-parent = <&axi_intc_0>; interrupts = <1 2>; reg = <0x0 0xa00d0000 0x0 0x10000>; xlnx,kind-of-intr = <0x360>; xlnx,num-intr-inputs = <0xe>; }; axi_iic_0: i2c@a0000000 { #address-cells = <1>; #size-cells = <0>; clock-names = "s_axi_aclk"; clocks = <&clk_wiz_0 0>; compatible = "xlnx,axi-iic-2.1", "xlnx,xps-iic-2.00.a"; interrupt-names = "iic2intc_irpt"; interrupt-parent = <&gic>; interrupts = <0 91 4>; reg = <0x0 0xa0000000 0x0 0x1000>; }; axi_iic_1: i2c@a0008000 { #address-cells = <1>; #size-cells = <0>; clock-names = "s_axi_aclk"; clocks = <&clk_wiz_0 0>; compatible = "xlnx,axi-iic-2.1", "xlnx,xps-iic-2.00.a"; interrupt-names = "iic2intc_irpt"; interrupt-parent = <&axi_intc_1>; interrupts = <1 2>; reg = <0x0 0xa0008000 0x0 0x1000>; }; }; }; };

Kernel Configuration

The following config options are to be enabled in order to use FPGA Manager, Please note that these options are enabled by default through xilinx_versal_defconfig except for the option. If the user wants to test a feature they have it.

Versal FPGA Manager Configuration:

Select: Device Drivers → FPGA Configuration Framework →   Xilinx Versal FPGA 

 

In order to test the feature, user needs to enable FPGA debug: 

Select: Device Drivers →  FPGA Configuration Framework -->   FPGA debug  fs

 

DT overlay ConfigFS interface Configuration:
In order to load Bitstream with DTBO user needs to enable below options:

Select: Device Drivers --> Device Tree and Open Firmware support

 

Contiguous Memory Allocator Configuration:

CONFIG_CMA

Select: Kernel Features --> Contiguous Memory Allocator

CONFIG_DMA_CMA

Select: Device Drivers --> Generic Driver Options → DMA Contiguous Memory Allocator 

 

Devicetree

versal_fpga: versal_fpga { compatible = "xlnx,versal-fpga"; }; fpga: fpga { compatible = "fpga-region"; fpga-mgr = <&versal_fpga>; #address-cells = <2>; #size-cells = <2>; };

Note:

  • power-domains:  Property is optional. If this property is present it controls the PM domain specifier as defined by bindings of the power controller specified by phandle.
    The PL_PD power domain will be turned on before loading the bitstream and turned off while removing/unloading the bitstream using overlays

  • From 2022.1 onwards, the Xilinx Device Tree Generator (DTG) tool generates extra pid and uid properties that are not being used by Linux; users can ignore these. The same applies to Full/Partial(DFX) Overlays.

DFX flow - Required DT overlays syntax

The base FPGA Region (static) with one PR region. This classic example has one PR region which has two RM's in it.

Configure Base Image (also called the "static image"):

An FPGA image is targeted to do a full reconfiguration of the FPGA. A base image may be set up, a set of partial reconfiguration regions that May later be reprogrammed.

Configure the PR Images:

An FPGA set up with a base image that created a PR region. The contents of each PR may have multiple Reconfigurable Modules. These RMs (PR0-RM0, PR0-RM1) are reprogrammed independently while the rest of the system continues to function.

Typical DFX topology 

  • -- xilinx `-- dfx-static |-- dfx-static.dtbo |-- dfx-static.pdi |---PR0 | `-- rm0         | `-- dfx-rm0       | |-- dfx-rm0.dtsi | |-- dfx-rm0.dtbo | `-- dfx-rm0.pdi        | `-- rm1 | `-- dfx-rm1 | |-- dfx-rm1.dtsi | |-- dfx-rm1.dtbo | |-- dfx-rm1.pdi `

Required DT syntax for dfx-static - Full Image

dfx-static image configuration

/dts-v1/; /plugin/; / { fragment@0 { target = <&fpga>; #address-cells = <2>; #size-cells = <2>; __overlay__ { #address-cells = <2>; #size-cells = <2>; firmware-name = "dfx-static.pdi"; fpga_PR0: fpga_PR0 { #address-cells = <2>; #size-cells = <2>; compatible = "fpga-region"; ranges ; }; }; }; fragment@1 { target = <&amba>; #address-cells = <2>; #size-cells = <2>; __overlay__ { #address-cells = <2>; #size-cells = <2>; /* Add PL node which are only part of dfx-static region(don't add PR related PL nodes here) */ }; }; }; Example fragement@1 fragment@1 { target = <&amba>; #address-cells = <2>; #size-cells = <2>; __overlay__ { #address-cells = <2>; #size-cells = <2>; / * static PL IP nodes */ dfx_axi_shutdown_man_0: dfx_axi_shutdown_manager@000a4000000 { compatible = "xlnx,dfx-axi-shutdown-manager-1.00", "xlnx,dfx-axi-shutdown-manager"; reg = <0x0 0xa4000000 0x0 0x1000>; clocks = <&misc_clk_0>; clock-names = "aclk"; }; misc_clk_0: misc_clk_0 { #clock-cells = <0>; clock-frequency = <99999001>; compatible = "fixed-clock"; }; }; };

dfx-static image has already been configured prior to OS boot up.

/dts-v1/; /plugin/; / { fragment@0 { target = <&fpga>; #address-cells = <2>; #size-cells = <2>; __overlay__ { #address-cells = <2>; #size-cells = <2>; external-fpga-config; /* Image has already been configured prior to OS boot up */ fpga_PR0: fpga_PR0 { #address-cells = <2>; #size-cells = <2>; compatible = "fpga-region"; ranges ; }; }; }; fragment@1 { target = <&amba>; #address-cells = <2>; #size-cells = <2>; __overlay__ { #address-cells = <2>; #size-cells = <2>; /* Add PL node which are only part of dfx-static region(don't add PR related PL nodes here) */ }; }; }; Example fragement@1 fragment@1 { target = <&amba>; #address-cells = <2>; #size-cells = <2>; __overlay__ { #address-cells = <2>; #size-cells = <2>; / * static PL IP nodes */ dfx_axi_shutdown_man_0: dfx_axi_shutdown_manager@000a4000000 { compatible = "xlnx,dfx-axi-shutdown-manager-1.00", "xlnx,dfx-axi-shutdown-manager"; reg = <0x0 0xa4000000 0x0 0x1000>; clocks = <&misc_clk_0>; clock-names = "aclk"; }; misc_clk_0: misc_clk_0 { #clock-cells = <0>; clock-frequency = <99999001>; compatible = "fixed-clock"; }; }; };

Required DT syntax for dfx-PR0 - Partial

dfx-rm0 image configuration

/dts-v1/; /plugin/; / { fragment@0 { target = <&fpga_PR0>; __overlay__ { #address-cells = <2>; #size-cells = <2>; firmware-name = "dfx-rm0.pdi"; fpga-bridges = <&dfx_axi_shutdown_man_0>; /* This propery is only needed if the the pr-decoupler/Axi-shutdown manager IP is part of dfx-static */ partial-fpga-config; }; }; fragment@1 { target = <&amba>; __overlay__ { #address-cells = <2>; #size-cells = <2>; /* Add PL node which are only part of dfx-rm0 (pr0-rm0) region(don't add static/any other PR/RM related PL nodes here) */ }; }; }; Example fragment@1: fragment@1 { target = <&amba>; __overlay__ { #address-cells = <2>; #size-cells = <2>; axi_gpio_0: gpio@000a4010000 { #gpio-cells = <2>; compatible = "xlnx,xps-gpio-1.00.a"; gpio-controller ; reg = <0x00000000 0xa4010000 0x0 0x10000>; xlnx,all-inputs = <0x0>; xlnx,all-inputs-2 = <0x1>; xlnx,all-outputs = <0x1>; xlnx,all-outputs-2 = <0x0>; xlnx,dout-default = <0x00000000>; xlnx,dout-default-2 = <0x00000000>; xlnx,gpio-width = <0x1>; xlnx,gpio2-width = <0x1>; xlnx,interrupt-present = <0x0>; xlnx,is-dual = <0x1>; xlnx,tri-default = <0xFFFFFFFF>; xlnx,tri-default-2 = <0xFFFFFFFF>; }; }; };

dfx-rm1 image configuration

/dts-v1/; /plugin/; / { fragment@0 { target = <&fpga_PR0>; __overlay__ { #address-cells = <2>; #size-cells = <2>; firmware-name = "dfx-rm1.pdi"; fpga-bridges = <&dfx_axi_shutdown_man_0>; /* This propery is only needed if the the pr-decoupler/Axi-shutdown manager IP is part of dfx-static */ partial-fpga-config; }; }; fragment@1 { target = <&amba>; __overlay__ { #address-cells = <2>; #size-cells = <2>; /* Add PL node which are only part of dfx-rm1 (pr0-rm1) region(don't add static/any other PR/RM related PL nodes here) */ }; }; }; Example fragment@1: fragment@1 { target = <&amba>; __overlay__ { #address-cells = <2>; #size-cells = <2>; axi_gpio_0: gpio@000a4020000 { #gpio-cells = <2>; compatible = "xlnx,xps-gpio-1.00.a"; gpio-controller ; reg = <0x00000000 0xa4020000 0x0 0x10000>; xlnx,all-inputs = <0x0>; xlnx,all-inputs-2 = <0x1>; xlnx,all-outputs = <0x1>; xlnx,all-outputs-2 = <0x0>; xlnx,dout-default = <0x00000000>; xlnx,dout-default-2 = <0x00000000>; xlnx,gpio-width = <0x1>; xlnx,gpio2-width = <0x1>; xlnx,interrupt-present = <0x0>; xlnx,is-dual = <0x1>; xlnx,tri-default = <0xFFFFFFFF>; xlnx,tri-default-2 = <0xFFFFFFFF>; }; }; };

FPGA Programming

Steps for programming the Bitstream/PDI using sysfs 

Load the Bitstream/PDI

  • mkdir -p /lib/firmware

  • cp /media/Bitstream.pdi /lib/firmware/

  • echo Bitstream.pdi > /sys/class/fpga_manager/fpga0/firmware

Steps for programming the Bistream/PDI using overlay

There is no out-of-box tool flow support for loading Bitstreams/PDI that need PL drivers. The creation of DTO files needed for this approach is not supported in the tool flow (DTG, Petalinux, Yocto).

Users can still achieve their goal of programming the bitstreams that need PL driver support by using hand-written DTSi files.

Copy the Bitstream/PDI (.pdi) and pl.dtbo files into lib/firmware folder

  • mkdir /configfs

  • mount -t configfs configfs /configfs

  • cd /configfs/device-tree/overlays/

  • mkdir Bitstream

  • echo -n "pl.dtbo" > Bitstream/path

Steps to remove the drivers got added as part of DTO

  • rmdir /configfs/device-tree/overlays/Bitstream


Compiling pl.dtsi (object)

dtc -O dtb -o pl.dtbo -b 0 -@ pl.dtsi

Device-tree overlay file contents example:

//Device Tree Example // Enable the axi-gpio interface /dts-v1/; /plugin/; / {     fragment@0 { /* Bitstream  Fragment */         target = <&fpga>;         overlay0: __overlay__ {             #address-cells = <2>;             #size-cells = <2>;             firmware-name = "Bitstream.pdi";         };     };     fragment@2 { /* PL Drivers Fragment */         target = <&amba>;         overlay2: __overlay__ {             axi_gpio_0: gpio@a0000000 {                 #gpio-cells = <3>;                 #interrupt-cells = <2>;                 clock-names = "s_axi_aclk";                 clocks = <&clk 71>;                 compatible = "xlnx,axi-gpio-2.0", "xlnx,xps-gpio-1.00.a";                 gpio-controller ;                 -controller ;                 interrupt-names = "ip2intc_irpt";                 interrupt-parent = <&gic>;                 interrupts = <0 89 4>;                 reg = <0x0 0xa0000000 0x0 0x1000>;                 all-inputs = <0x1>;                 all-inputs-2 = <0x0>;                 all-outputs = <0x0>;                 all-outputs-xlnx,2 = <0x1>;                 dout-xlnx,default = <0x00000000>;                 dout-default-2 = <0xAAAAAAAA>;                 gpio-width = <0x8>;                 xlnx,gpio2-width = <0x8>;                 xlnx,interrupt-present = <0x1>;                 xlnx,is-dual = <xlnx,xlnx,0x1>;                 xlnx,tri-xlnx,xlnx,default = <0xFFFFFFFF>;                 xlnx,tri-xlnx,xlnx,default-2 = <0xFFFFFFFF>;             };             psu_ctrl_ipi: PERIPHERAL@ff380000 {                 /* This is a place holder node for a custom IP, user may need to update the entries */                 compatible =compatible "PERIPHERAL-1.0",xlnx,xlnx;                 reg = <0x0 0xff380000 0x0 0x80000>;             };             psu_message_buffers: PERIPHERAL@ff990000 {                 /* This is a place holder node for a custom IP, user may need to update the entries */                 compatible =compatible "xlnx,PERIPHERAL-1.0"xlnx,;                 reg = <0x0 0xff990000 0x0 0x10000>;             };         };     }; };

Steps for programming the Bitstream/PDI using Debug fs

  • mount -t debugfs none /sys/kernel/debug/

  • mkdir -p /lib/firmware

  • cp /media/Bitstream.pdi /lib/firmware/

  • dd bs=16M if=/lib/firmware/Bitstream.pdi of=/sys/kernel/debug/fpga/fpga0/load (where Bitstream.pdi < 16M)

Steps for programming the Bitstream/PDI using fpgautil 

root@xilinx:~# fpgautil -h fpgautil: FPGA Utility for Loading/reading PL Configuration in zynqMP Usage: fpgautil -b <bin file path> -o <dtbo file path> Options: -b <binfile> (Bin file path) -o <dtbofile> (DTBO file path) -f <flags> Optional: <Bitstream type flags> f := <Full | Partial > Default: <Full> Examples: (Load Full Bitstream using Overlay) fpgautil -b top.bit.bin -o can.dtbo (Load Partial Bitstream through the sysfs interface) fpgautil -b top.bit.bin -f Partial

Fpgautil usage for DFX (aka PR/RP)use cases

Typical topology: FPGA has the base FPGA Region(static) with two PR/DFX regions (each PR region has two RMs).  The relevant files can be seen as below.

-- xilinx `-- static |-- static.dtbo |-- static.bin |---PR0 | `-- PR0-RM0         | |-- pr0-rm0.dtsi | |-- pr0-rm0.dtbo | |-- pr0-rm0.bin        | `-- PR0-RM1 | |-- pr0-rm1.dtsi | |-- pr0-rm1.dtbo | |-- pr0-rm1.bin |---PR1 | `-- PR1-RM0 | |-- pr1-rm0.dtsi | |-- pr1-rm0.dtbo | |-- pr1-rm0.bin | `-- PR0-RM1 | |-- pr1-rm1.dtsi | |-- pr1-rm1.dtbo | |-- pr1-rm1.bin
  • Configure/Reconfigure Static region using fpgautil: fpgautil -b <full/static bitstream image> -o <Full/static dtbo file> -f Full -n <Fpga region info>

    • Usage example: fpgautil -b static.bin -o static.dtbo -f Full -n Full

  • PR0 region configuration/Reconfigure using fpgautil:

    • Command to configure the PR0-RM0 Image: fpgautil -b <PR0-RM0 Bitstream image> -o <PR0-RM0 dtbo file> -f Partial -n <Fpga region info>

      • Usage example: fpgautil PR0-RM0.bin -o PR0-RM0.dtbo -f Partial -n PR0

    • Command to Remove Images(PR0-RM0) from the PR0 region: fpgautil -R -n  -n <Fpga region info> (Note: Before Reconfiguring the same region with a different Bitstream user needs to remove the existing dtbo relevant to it)

      • Usage example: fpgautil -R -n PR0

    • Command to configure the PR0-RM1 Image:: fpgautil -b <PR0-RM1 Bitstream image> -o <PR0-RM1 dtbo file> -f Partial -n <Fpga region info>

      • Usage example: fpgautil PR0-RM1.bin -o PR0-RM1.dtbo -f Partial -n PR0

    • Command to Remove Images(PR0-RM1) from the PR0 region: fpgautil -R -n <Fpga region info>

      • Usage example: fpgautil -R -n PR0

  • PR1 region configuration/reconfiguration using fpgautil:

    • Command to configure the PR0-RM0 Image: fpgautil -b <PR1-RM0 Bitstream image> -o <PR1-RM0 dtbo file> -f Partial -n <Fpga region info>

      • Usage example: fpgautil PR1-RM0.bin -o PR1-RM0.dtbo -f Partial -n PR1

    • Command to Remove Images(PR1-RM0) from the PR1 region: fpgautil -R -n <Fpga region info> (Note: Before Reconfiguring the same region with a different Bitstream user needs to remove the existing dtbo relevant to it)

      • Usage example: fpgautil -R -n PR1

    • Command to configure the PR1-RM1 Image:: fpgautil -b <PR1-RM1 Bitstream image> -o <PR1-RM1 dtbo file> -f Partial -n <Fpga region info>

      • Usage example: fpgautil PR1-RM1.bin -o PR1-RM1.dtbo -f Partial -n PR0

    • Command to Remove Images(PR0-RM1) from the PR0 region: fpgautil -R -n <Fpga region info>

      • Usage example: fpgautil -R -n PR1

 

  • Command to remove the images from the static region: fpgautil -R -n <Fpga region info>

    • Usage example: fpgautil -R -n Full

 

Notes:

  • Before reconfiguring the same region (whether it is a static/ partial region) with a different Bitstream the user needs to remove the existing dtbo relevant to it.

  • Fpgautil -R is responsible for only removing the dtbo file from the livetree. it will not remove the PL logic from the FPGA region. If the user wants to remove the complete configuration logic from the FPGA region he/she should reconfigure the target configuration region with a greybox reconfigurable module.

Exercising FPGA programming using Libdfx

https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/1959854475

Mainline Status

  • This driver is currently not available in the mainline kernel

Release history

2023.2

  • None

2023.1

  • FPGA Manager framework upgraded to 6.1v

2022.2

  • fpga: versal-fpga: Remove unnecessary priv structure

  • fpga: versal-fpga: Remove empty functions

  • fpga: fpga-mgr: versal: Simplify registration

2022.1

  • FPGA Manager framework upgraded to 5.15v

2021.2

  • None.

2021.1

  • FPGA Manager framework upgraded to 5.10v

2020.2

Summary:

  • fpga: versal: Initialized variables before using it

  • fpga: versal: Use the scatterlist interface

  • fpga: versal: Use 32-bit DMA addressing

  • fpga: versal-fpga: Add versal fpga manager driver

 

References

https://github.com/Xilinx/linux-xlnx/blob/master/drivers/fpga/versal-fpga.c

https://github.com/Xilinx/linux-xlnx/blob/master/Documentation/fpga/fpga-mgr.txt

https://github.com/Xilinx/linux-xlnx/blob/master/Documentation/devicetree/bindings/fpga/fpga-region.txt











© Copyright 2019 - 2022 Xilinx Inc. Privacy Policy