Solution Versal PL Programming

Owned by navam
Last updated: Jun 18, 2024
24 min read

Introduction‍

The Versal Programmable Logic (PL) can be programmed either using U-Boot or through Linux.
This page provides the details about programming the PL from Linux world using Linux FPGA Manager framework.

Flow:

 

 

HW IP Features

  • Full Bitstream and Partial Bitstream/PDI loading

  • Encrypted and Authenticated Full/Partial Bitstream/PDI loading

  • Readback of Configuration Data

Features supported in the driver

  • Full Bitstream Support (DFX flow)

    • Non Secure Bitstream/PDI loading 

    • Encrypted  Bitstream/PDI loading with device-key

    • Authenticated Bitstream/PDI 

    • Authenticated and Encrypted Bitstream/PDI with device-key

  • Partial Bitstream Support 

    • Non Secure Bitstream/PDI loading 

    • Encrypted Partial Bitstream/PDI loading with device-key

    • Authenticated Partial Bitstream/PDI

    • Authenticated and Encrypted Partial Bitstream/PDI with device-key

Unsupported features in the driver

  • Full Bitstream Support 

    • Encrypted  Bitstream/PDI loading with user-key

    • Authenticated and Encrypted Bitstream/PDI  loading with User-key

  • Partial Bitstream Support 

    • Encrypted Partial Bitstream/PDI with user-key

    • Authenticated and Encrypted Partial Bitstream/PDI  with user-key

  •  Read Back Support

    • Configuration Data Readback.

Missing Features, Known Issues and Limitations in the driver

The Xilinx device tree generator (https://github.com/Xilinx/device-tree-xlnx ) currently lacks automated support for device tree overlay generation for Partial Reconfiguration / DFX designs.  For systems requiring runtime device tree overlay (eg, Linux device driver support) support, hand-crafted device trees can be deployed and loaded. See the "Working with Device Tree Overlay (DTBO)" section below for details on how to load these at runtime.
Few considerations that user should be aware of in programming the PL are as follows

  • For the Versal platform, it is expected that the base PDI will always be part of BOOT PDI and will be loaded by PLM at boot time. Later from Linux, users can load either the Full/partial PDI from Linux. The term Full bitstream/PDI in Versal implies the whole PL region programming except the configuration that's part of base PDI.

  • Using overlays we can add a new node or add/update the existing node properties. But it will not allow replacing the existing nodes which are already part of the live tree. 

  • If the PL design is having axi-intc IP, the relevant DT node should be the top node in the overlay. dtsi or pl.dtsi file before its actually being referenced in any other nodes.

    • Also, IRQCHIP Xilinx Intc driver module support must be enabled in the kernel configuration.x

  • Apply overlay change sets will occur from top to bottom and Removal of overlay change sets will occur in the opposite order of apply(the most recently applied overlay change set must be removed first). So if the overlay has any internal dependency one on another those need to be taken care of while creating the overlay files. (Like: Interrupt controllers, clock nodes.....etc).

  • If the design has AXI INTC IP, the PL configuration stress test may lead to the kernel crash due to the memory leak issues that exist with the AXI INT driver.

  • The existing Overlay fame-work allows the removal of overlay' only if it's the top-most one. So please follow the LIFO order to remove overlays from live tree.

Example DT node:

/dts-v1/; /plugin/; / { fragment@0 { target = <&fpga>; __overlay__ { #address-cells = <2>; #size-cells = <2>; firmware-name = "system.pdi"; }; }; fragment@1 { target = <&amba>; __overlay__ { #address-cells = <2>; #size-cells = <2>; axi_intc_1: interrupt-controller@a00d0000 { /* Top node */ #interrupt-cells = <2>; clock-names = "s_axi_aclk"; clocks = <&clk_wiz_0 0>; compatible = "xlnx,axi-intc-4.1", "xlnx,xps-intc-1.00.a"; interrupt-controller ; interrupt-names = "irq"; interrupt-parent = <&axi_intc_0>; interrupts = <1 2>; reg = <0x0 0xa00d0000 0x0 0x10000>; xlnx,kind-of-intr = <0x360>; xlnx,num-intr-inputs = <0xe>; }; axi_iic_0: i2c@a0000000 { #address-cells = <1>; #size-cells = <0>; clock-names = "s_axi_aclk"; clocks = <&clk_wiz_0 0>; compatible = "xlnx,axi-iic-2.1", "xlnx,xps-iic-2.00.a"; interrupt-names = "iic2intc_irpt"; interrupt-parent = <&gic>; interrupts = <0 91 4>; reg = <0x0 0xa0000000 0x0 0x1000>; }; axi_iic_1: i2c@a0008000 { #address-cells = <1>; #size-cells = <0>; clock-names = "s_axi_aclk"; clocks = <&clk_wiz_0 0>; compatible = "xlnx,axi-iic-2.1", "xlnx,xps-iic-2.00.a"; interrupt-names = "iic2intc_irpt"; interrupt-parent = <&axi_intc_1>; interrupts = <1 2>; reg = <0x0 0xa0008000 0x0 0x1000>; }; }; }; };

Kernel Configuration

The following config options are to be enabled in order to use FPGA Manager, Please note that these options are enabled by default through xilinx_versal_defconfig except for the option. If the user wants to test a feature they have it.

Versal FPGA Manager Configuration:

Select: Device Drivers → FPGA Configuration Framework →   Xilinx Versal FPGA 

 

DT overlay ConfigFS interface Configuration:
To load Bitstream with DTBO, the user needs to enable below options:

Select: Device Drivers --> Device Tree and Open Firmware support

Contiguous Memory Allocator Configuration:

CONFIG_CMA

Select: Kernel Features --> Contiguous Memory Allocator

CONFIG_DMA_CMA

Select: Device Drivers --> Generic Driver Options → DMA Contiguous Memory Allocator 

Devicetree

Note:

  • power-domains:  Property is optional. If this property is present it controls the PM domain specifier as defined by bindings of the power controller specified by phandle.
    The PL_PD power domain will be turned on before loading the bitstream and turned off while removing/unloading the bitstream using overlays

  • From 2022.1 onwards, the Xilinx Device Tree Generator (DTG) tool generates extra pid and uid properties that are not being used by Linux; users can ignore these. The same applies to Full/Partial(DFX) Overlays.

DFX flow - Required DT overlays syntax

The base FPGA Region (static) with one PR region. This classic example has one PR region which has two RM's in it.

Configure Base Image (also called the "static image"):

An FPGA image is targeted to do a full reconfiguration of the FPGA. A base image may be set up, a set of partial reconfiguration regions that May later be reprogrammed.

Configure the PR Images:

An FPGA set up with a base image that created a PR region. The contents of each PR may have multiple Reconfigurable Modules. These RMs (PR0-RM0, PR0-RM1) are reprogrammed independently while the rest of the system continues to function.

Typical DFX topology 

Required DT syntax for dfx-static - Full Image

dfx-static image configuration

dfx-static image has already been configured prior to OS boot up.

Required DT syntax for dfx-PR0 - Partial

dfx-rm0 image configuration

dfx-rm1 image configuration

FPGA Programming

Steps for programming the Bitstream/PDI using sysfs 

Load the Bitstream/PDI

  • mkdir -p /lib/firmware

  • cp /media/Bitstream.pdi /lib/firmware/

  • echo Bitstream.pdi > /sys/class/fpga_manager/fpga0/firmware

Steps for programming the Bistream/PDI using overlay

There is no out-of-box tool flow support for loading Bitstreams/PDI that need PL drivers. The creation of DTO files needed for this approach is not supported in the tool flow (DTG, Petalinux, Yocto).

Users can still achieve their goal of programming the bitstreams that need PL driver support by using hand-written DTSi files.

Copy the Bitstream/PDI (.pdi) and pl.dtbo files into lib/firmware folder

  • mkdir /configfs

  • mount -t configfs configfs /configfs

  • cd /configfs/device-tree/overlays/

  • mkdir Bitstream

  • echo -n "pl.dtbo" > Bitstream/path

Steps to remove the drivers got added as part of DTO

  • rmdir /configfs/device-tree/overlays/Bitstream


Compiling pl.dtsi (object)

Device-tree overlay file contents example:

Steps for programming the Bitstream/PDI using Debug fs

  • mount -t debugfs none /sys/kernel/debug/

  • mkdir -p /lib/firmware

  • cp /media/Bitstream.pdi /lib/firmware/

Steps for programming the Bitstream/PDI using fpgautil 

Fpgautil usage for DFX (aka PR/RP)use cases

Typical topology: FPGA has the base FPGA Region(static) with two PR/DFX regions (each PR region has two RMs).  The relevant files can be seen as below.

  • Configure/Reconfigure Static region using fpgautil: fpgautil -b <full/static bitstream image> -o <Full/static dtbo file> -f Full -n <Fpga region info>

    • Usage example: fpgautil -b static.bin -o static.dtbo -f Full -n Full

  • PR0 region configuration/Reconfigure using fpgautil:

    • Command to configure the PR0-RM0 Image: fpgautil -b <PR0-RM0 Bitstream image> -o <PR0-RM0 dtbo file> -f Partial -n <Fpga region info>

      • Usage example: fpgautil PR0-RM0.bin -o PR0-RM0.dtbo -f Partial -n PR0

    • Command to Remove Images(PR0-RM0) from the PR0 region: fpgautil -R -n  -n <Fpga region info> (Note: Before Reconfiguring the same region with a different Bitstream user needs to remove the existing dtbo relevant to it)

      • Usage example: fpgautil -R -n PR0

    • Command to configure the PR0-RM1 Image:: fpgautil -b <PR0-RM1 Bitstream image> -o <PR0-RM1 dtbo file> -f Partial -n <Fpga region info>

      • Usage example: fpgautil PR0-RM1.bin -o PR0-RM1.dtbo -f Partial -n PR0

    • Command to Remove Images(PR0-RM1) from the PR0 region: fpgautil -R -n <Fpga region info>

      • Usage example: fpgautil -R -n PR0

  • PR1 region configuration/reconfiguration using fpgautil:

    • Command to configure the PR0-RM0 Image: fpgautil -b <PR1-RM0 Bitstream image> -o <PR1-RM0 dtbo file> -f Partial -n <Fpga region info>

      • Usage example: fpgautil PR1-RM0.bin -o PR1-RM0.dtbo -f Partial -n PR1

    • Command to Remove Images(PR1-RM0) from the PR1 region: fpgautil -R -n <Fpga region info> (Note: Before Reconfiguring the same region with a different Bitstream user needs to remove the existing dtbo relevant to it)

      • Usage example: fpgautil -R -n PR1

    • Command to configure the PR1-RM1 Image:: fpgautil -b <PR1-RM1 Bitstream image> -o <PR1-RM1 dtbo file> -f Partial -n <Fpga region info>

      • Usage example: fpgautil PR1-RM1.bin -o PR1-RM1.dtbo -f Partial -n PR0

    • Command to Remove Images(PR0-RM1) from the PR0 region: fpgautil -R -n <Fpga region info>

      • Usage example: fpgautil -R -n PR1

 

  • Command to remove the images from the static region: fpgautil -R -n <Fpga region info>

    • Usage example: fpgautil -R -n Full

 

Notes:

  • Before reconfiguring the same region (whether it is a static/ partial region) with a different Bitstream the user needs to remove the existing dtbo relevant to it.

  • Fpgautil -R is responsible for only removing the dtbo file from the livetree. it will not remove the PL logic from the FPGA region. If the user wants to remove the complete configuration logic from the FPGA region he/she should reconfigure the target configuration region with a greybox reconfigurable module.

Exercising FPGA programming using Libdfx