Introduction‍

The Zynq UltraScale+ MPSoC Programmable Logic (PL) can be programmed either using First Stage Boot-loader(FSBL), U-Boot or through Linux.
This page provides details about programming the PL from the Linux world using the Linux FPGA Manager framework.

Flow:

HW IP Features

Full Bitstream and Partial Bitstream loading
Encrypted and Authenticated Full/Partial Bitstream loading
Readback of Configuration Registers
Readback of Bitstream(Configuration Data)
Compressed Bitstream

Features supported in the driver

Full Bitstream Support
- Non Secure Bitstream
- Encrypted Bitstream
- Authenticated Bitstream
- Authenticated and Encrypted Bitstream
- Compressed Bitstream
- Loading Bitstream using Devicetree overlay
Partial Bitstream Support that doesn't require PL drivers
- Non Secure Partial Bitstream
- Encrypted Partial Bitstream
- Authenticated Partial Bitstream
- Authenticated and Encrypted Partial Bitstream
Readback
- Configuration Registers Readback
- Configuration Data Readback

Known Issues for PL Programming

A few considerations that users should be aware of in programming the PL/FPGA are as follows.

If the user tries to load the un-aligned bit/bin file the PL configuration takes a longer time when compared with aligned(word-aligned) bit/bin files.
If the design has AXI INTC IP, the PL configuration stress test may lead to the kernel crash due to the memory leak issues that exist with the AXI INT driver.
Using overlays we can add a new node or add/update the existing node properties. But it will not allow replacing the existing nodes which is already part of the live tree.
If the PL design has axi-intc IP, the relevant DT node should be the top node in the overlay.dtsi or pl.dtsi file before it is actually being referenced in any other nodes.
- Also, IRQCHIP Xilinx Intc driver module support must be enabled in the kernel configuration.
Apply overlay change sets will occur from top to bottom and Removal of overlay change sets will occur in the opposite order of apply(the most recently applied overlay change set must be removed first). So if the overlay has any internal dependency one on another those need to be taken care of while creating the overlay files. (Like: Interrupt controllers, clock nodes.....etc).
The existing Overlay fame-work allows the removal of overlay' only if it's the top-most one. So please follow the LIFO order to remove overlays from live tree.
- for more details ref this link: https://elixir.bootlin.com/linux/v6.6.32/source/drivers/of/overlay.c#L1137

Example DT node.

/dts-v1/;
/plugin/;
/ {
	fragment@0 {
		target = <&fpga_full>;
		overlay0: __overlay__ {
			#address-cells = <2>;
			#size-cells = <2>;
			firmware-name = "system.bin";
			resets = <&zynqmp_reset 116>;
		};
	};
	fragment@1 {
		target = <&amba>;
		overlay1: __overlay__ {
			afi0: afi0 {
				compatible = "xlnx,afi-fpga";
				config-afi = < 0 0>, <1 0>, <2 0>, <3 0>, <4 0>, <5 0>, <6 0>, <7 0>, <8 0>, <9 0>, <10 0>, <11 0>, <12 0>, <13 0>, <14 0xa00>, <15 0x000>;
			};
		};
	};
	fragment@2 {
		target = <&amba>;
		overlay2: __overlay__ {
			#address-cells = <2>;
			#size-cells = <2>;
			axi_intc_1: interrupt-controller@a00d0000 { /* Top node */
				#interrupt-cells = <2>;
				clock-names = "s_axi_aclk";
				clocks = <&clk_wiz_0 0>;
				compatible = "xlnx,axi-intc-4.1", "xlnx,xps-intc-1.00.a";
				interrupt-controller ;
				interrupt-names = "irq";
				interrupt-parent = <&axi_intc_0>;
				interrupts = <1 2>;
				reg = <0x0 0xa00d0000 0x0 0x10000>;
				xlnx,kind-of-intr = <0x360>;
				xlnx,num-intr-inputs = <0xe>;
			};
			axi_iic_0: i2c@a0000000 {
				#address-cells = <1>;
				#size-cells = <0>;
				clock-names = "s_axi_aclk";
				clocks = <&clk_wiz_0 0>;
				compatible = "xlnx,axi-iic-2.1", "xlnx,xps-iic-2.00.a";
				interrupt-names = "iic2intc_irpt";
				interrupt-parent = <&gic>;
				interrupts = <0 91 4>;
				reg = <0x0 0xa0000000 0x0 0x1000>;
			};
			axi_iic_1: i2c@a0008000 {
				#address-cells = <1>;
				#size-cells = <0>;
				clock-names = "s_axi_aclk";
				clocks = <&clk_wiz_0 0>;
				compatible = "xlnx,axi-iic-2.1", "xlnx,xps-iic-2.00.a";
				interrupt-names = "iic2intc_irpt";
				interrupt-parent = <&axi_intc_1>;
				interrupts = <1 2>;
				reg = <0x0 0xa0008000 0x0 0x1000>;
			};
		};
	};
};

Summary Table for Partial Reconfiguration / Dynamic Function Exchange Feature Support

Partial Reconfiguration / Dynamic Function Exchange (DFX) is an advanced feature of Xilinx Zynq UltraScale+ MPSoC devices. Refer to the table below for details. For PetaLinux releases prior to 2018.3 or for building Linux system images manually, please refer to the Linux OSL Flow page.

Release \ Feature	Vivado RTL Design Flow	Vivado IPI Design Flow	Linux Device Tree Generator	Linux Programming Framework
<= 2018.3	Supported	Not Supported	Automation Not Supported - Hand Crafted	FPGA Manager
2019.1 - 2020.2	Supported	DFX Lounge	DFX Lounge	FPGA Manager / fpgautil

Kernel Configuration

The following config options have to be enabled in order to use FPGA Manager, Please note that these options are enabled by default through xilinx_zynqmp_defconfig except for the fpga debugfs option. If the user wants to test the readback feature they have to enable it.

Zynq UltraScale+ MPSoC FPGA Manager Configuration:

Select: Device Drivers → FPGA Configuration Framework → Xilinx Zynq UltraScale+ MPSoC FPGA

In order to test readback feature user needs to enable FPGA debug fs

Select: Device Drivers → FPGA Configuration Framework --> FPGA debug fs

DT overlay ConfigFS Interface Configuration:
In order to load Bitstream with DTBO user needs to enable the below options

Select: Device Drivers --> Device Tree and Open Firmware support

Contiguous Memory Allocator Configuration:

CONFIG_CMA

Select: Kernel Features --> Contiguous Memory Allocator

CONFIG_DMA_CMA

Select: Device Drivers --> Generic Driver Options → DMA Contiguous Memory Allocator

Devicetree

pcap {
       compatible = "xlnx,zynqmp-pcap-fpga";
       clocks = <&clk 41>;
};

fpga_full: fpga-full {
       compatible = "fpga-region";
       fpga-mgr = <&pcap>;
       #address-cells = <2>;
       #size-cells = <2>;
	   power-domains = <&zynqmp_firmware PL_PD>;
};

For more details about devicetree bindings Reference the Below link: Devicetree

Note: The above devicetree node is by default present in the zynqmp.dtsi file

Bitstream Format

From the 2018.3 release onwards FPGA Manager supports loading of vivado and bootgen generated Bitstream and Bin files vivadobootgen[1]

Note: For releases earlier to 2018.3 FPGA Manager was capable of loading only bootgen generated bin files. bootgen

.Bit Format

Generated by Vivado

.Bin Format

Generated by Vivado
Generated by Bootgen by converting vivado generated bit file
Bootgen Command to generate bin file:
# bootgen -image Bitstream.bif -arch zynqmp -o ./Bitstream.bin -w (2018.1 or later releases)
# bootgen -image Bitstream.bif -arch zynqmp -process_Bitstream bin (2017.4 and earlier releases)
Bitstream.bif file should contains the below lines:

all:
{
        [destination_device = pl] Full.bit (or)  Partial.bit /* Bitstream file name */
}

Building Software Images

Xilinx tools (Petalinux, Yocto) provide an option with the name FPGA Manager which if enabled builds the device tree overlay fragments automatically and copies the Bitstream and DTBO files into root file system.

PetaLinux Flow

PetaLinux setup and build: PetaLinux

Source the PetaLinux tool.source /opt/petalinux/petalinux-v<petalinux-version>/settings.sh

Create a Versal template project or bsp project.

petalinux-create -t project -n zynqmp-dfx  --template versal

petalinux-create -t project -s <bsp path> -n zynqmp-dfx

Go to the project
```
cd zynqmp-dfx
```

Configure the project with static.xsa/base.xsa

petalinux-config --get-hw-description <base.xsa/static.xsa>

Enable FPGA manager using the following command
```
petalinux-config -> FPGA MANAGER
```
Create the static/base application using the following command from static/base xsa
```
petalinux-create -t apps --template fpgamanager_dtg -n <static-app> --enable --srcuri "<static xsa>"
```
The previous command creates and packages the static dtbo and pdi files into the rootfs (/lib/firmware/xilinx/) using fpgamanager_dtg template.
Create the partial application to configure the partial region using the rm xsa in the following command. You should point static pl app name as --static-pn command line option to define the relation between base and partial.
```
petalinux-create -t apps --template fpgamanager_dtg_dfx -n <rm-app> --enable --srcuri <rm.xsa>" --static-pn <static-app>
```
This command generates and packages the rm dtbo, pdi files into the rootfs (/lib/firmware/xilinx/<static-app>/<rm-app>).
Execute
```
 petalinux-build
```
The command generates the rootfs containing both static and rprm dtbos, and respective pdi files as mentioned in the step 5 and step 6.
Once the base boot images ready with the previous step, if you want to build only DFX apps, use the commands mentioned in step 8.

Execute the following command:

petalinux-build -c <static-app>
petalinux-build -c <rm-app>
in below path you will see with the below names.
in <TMPDIR>/deploy/rpm you will see <static-app>.rpm and <rm-app>.rpm

For more details relevant to the above steps refer to this link: Petalinux-DFX-Support

Once the build is complete, binaries are available at images/linux directory.

Boot the hardware with newly built images
- Directory /lib/firmware in the rootfs contains the Bitstreams and dtbo files

Yocto Flow

Yocto setup and build: yocto

Once Yocto environment is set

Edit <yocto-proj-root>/build/conf/local.conf add below lines of code

EXTRA_IMAGE_FEATURES += " fpga-manager"
EXTRA_HDF = "<PATH_TO_HDF_or_XSA>"

Note:

By default, it will pack base hdf Bitstream and dtbo to the /lib/firmware/base in the rootfs
Extra hdf path can contain multiple HDF's

2. Build the required images (run the below command)

$ bitbake petalinux-image-minimal

Once the build is complete, binaries are available at ${DEPLOY_DIR_IMAGE} (${TMPDIR}/deploy/images/${MACHINE}/) directory

3. Boot the hardware with newly built images

- Directory /lib/firmware in the rootfs contains the Bitstreams and dtbo files

OSL Flow

Setup and Build: OSL

Images required for testing the FPGA Manager

Bin/Bit file
DTBO file

DTBO File Generation:

Steps to Create the DT-Overlay Fragment
1. Clone the device-tree-xlnx repo:
  git clone device-tree-xlnx.git
2. Helper script: dt_overlay.tcl
  1. Command: # xsct dt_overaly.tcl system.hdf psu_cortexa53_0 device-tree-xlnx output_dir (or) xsct dt_overaly.tcl system.xsa psu_cortexa53_0 device-tree-xlnx output_dir

Ex: xsct dt_overaly.tcl system.hdf psu_cortexa53_0 ${DTS_REPO}/device-tree-xlnx overlay (or) xsct dt_overaly.tcl system.xsa psu_cortexa53_0 ${DTS_REPO}/device-tree-xlnx overlay

2. Create DeviceTree Overlay Blob (.dtbo) file from the pl.dtsi file

# dtc -O dtb -o pl.dtbo -b 0 -@ pl.dtsi

Ex: ./scripts/dtc/dtc -O dtb -o pl.dtbo -b 0 -@ pl.dtsi

3. Copy the generated bit and dtbo files to target

Note: PL nodes should not be the part of Base Device-tree (system.dtb).

If the PL nodes are part of static dtb (system.dtb), these nodes cannot be removed at run-time.

FPGA programming

Linux FPGA Manager framework provides sysfs (Bitstream loading), debugfs (readback), configfs (Bitstream loading along with DTBO for PL drivers) attributes.
Alternatively, users can opt for Xilinx developed fpgautil. This utility provides an easy-to-use interface for programmers for all FPGA Manager use cases.

Exercising FPGA programming using fpgautil

Source code for the fpgautil is available in below meta layers.

Note: In 2022.1 release fpgautil source code is moved from meta-xilinx-tools to meta-xilinx/meta-xilinx-core layer.

2022.1 and later release: https://github.com/Xilinx/meta-xilinx/blob/master/meta-xilinx-core/recipes-bsp/fpga-manager-script/files/fpgautil.c

2021.2 and earlier release: https://github.com/Xilinx/meta-xilinx-tools/blob/rel-v2021.2/recipes-bsp/fpga-manager-script/files/fpgautil.c

root@xilinx-zcu102-2018_3:~# fpgautil -h

fpgautil: FPGA Utility for Loading/reading PL Configuration in zynqMP

Usage:  fpgautil -b <bin file path> -o <dtbo file path>

Options: -b <binfile>           (Bin file path)
         -o <dtbofile>          (DTBO file path)
         -f <flags>             Optional: <Bitstream type flags>
                                   f := <Full | Partial > 
                                Default: <Full>
          -s <secure flags>     Optional: <Secure flags>
                                   s := <AuthDDR | AuthOCM | EnUsrKey | EnDevKey | AuthEnUsrKeyDDR | AuthEnUsrK>
          -k <AesKey>           Optional: <AES User Key>
          -r <Readback>         Optional: <file name>
                                Default: By default Read back contents will be stored in readback.bin file
          -t                    Optional: <Readback Type>
                                   0 - Configuration Register readback
                                   1 - Configuration Data Frames readback
                                Default: 0 (Configuration register readback)
          -R                    Optional: Remove overlay from a live tree
 
Examples:
(Load Full Bitstream using Overlay)
fpgautil -b top.bin -o can.dtbo
(Load Partial Bitstream through the sysfs interface)
fpgautil -b top.bin -f Partial 
(Load Authenticated Bitstream through the sysfs interface)
fpgautil -b top.bin -f Full -s AuthDDR 
(Load Parital Encrypted Userkey Bitstream using Overlay)
fpgautil -b top.bin -o pl.dtbo -f Partial -s EnUsrKey -k <32byte key value>
(Read PL Configuration Registers)
fpgautil -b top.bin -r

Fpgautil usage for DFx (aka PR/RP)use cases

Typical topology: FPGA has the base FPGA Region(static) with two PR/DFx regions (Each PR region has two RMs). The relevant files can be looked as below.

-- xilinx
    `-- static
        |-- static.dtbo
        |-- static.bin
        |---PR0
        |   `-- PR0-RM0 
        |        |-- pr0-rm0.dtsi
        |        |-- pr0-rm0.dtbo
        |        |-- pr0-rm0.bin
        |    `-- PR0-RM1
        |        |-- pr0-rm1.dtsi
        |        |-- pr0-rm1.dtbo
        |        |-- pr0-rm1.bin
        |---PR1
        |   `-- PR1-RM0
        |       |-- pr1-rm0.dtsi
        |       |-- pr1-rm0.dtbo
        |       |-- pr1-rm0.bin
        |   `-- PR0-RM1
        |       |-- pr1-rm1.dtsi
        |       |-- pr1-rm1.dtbo
        |       |-- pr1-rm1.bin

Configure/Reconfigure Static region using fpgautil: fpgautil -b <full/static bitstream image> -o <Full/static dtbo file> -f Full -n <Fpga region info>
- Usage example: fpgautil -b static.bin -o static.dtbo -f Full -n Full

PR0 region configuration/Reconfigure using fpgautil:
- Command to configure the PR0-RM0 Image: fpgautil -b <PR0-RM0 Bitstream image> -o <PR0-RM0 dtbo file> -f Partial -n <Fpga region info>
  - Usage example: fpgautil PR0-RM0.bin -o PR0-RM0.dtbo -f Partial -n PR0
- Command to Remove Images(PR0-RM0) from the PR0 region: fpgautil -R -n -n <Fpga region info> (Note: Before Reconfiguring the same region with a different Bitstream user needs to remove the existing dtbo relevant to it)
  - Usage example: fpgautil -R -n PR0
- Command to configure the PR0-RM1 Image:: fpgautil -b <PR0-RM1 Bitstream image> -o <PR0-RM1 dtbo file> -f Partial -n <Fpga region info>
  - Usage example: fpgautil PR0-RM1.bin -o PR0-RM1.dtbo -f Partial -n PR0
- Command to Remove Images(PR0-RM1) from the PR0 region: fpgautil -R -n <Fpga region info>
  - Usage example: fpgautil -R -n PR0

PR1 region configuration/reconfiguration using fpgautil:
- Command to configure the PR0-RM0 Image: fpgautil -b <PR1-RM0 Bitstream image> -o <PR1-RM0 dtbo file> -f Partial -n <Fpga region info>
  - Usage example: fpgautil PR1-RM0.bin -o PR1-RM0.dtbo -f Partial -n PR1
- Command to Remove Images(PR1-RM0) from the PR1 region: fpgautil -R -n <Fpga region info> (Note: Before Reconfiguring the same region with a different Bitstream user needs to remove the existing dtbo relevant to it)
  - Usage example: fpgautil -R -n PR1
- Command to configure the PR1-RM1 Image:: fpgautil -b <PR1-RM1 Bitstream image> -o <PR1-RM1 dtbo file> -f Partial -n <Fpga region info>
  - Usage example: fpgautil PR1-RM1.bin -o PR1-RM1.dtbo -f Partial -n PR0
- Command to Remove Images(PR0-RM1) from the PR0 region: fpgautil -R -n <Fpga region info>
  - Usage example: fpgautil -R -n PR1

Command to remove the images from the static region: fpgautil -R -n <Fpga region info>
- Usage example: fpgautil -R -n Full

Notes:

Before Reconfiguring the same region(whether it is a static/ partial region) with a different Bitstream user needs to remove the existing dtbo relevant to it
Fpgautil -R is responsible for only removing the dtbo file from the livetree. it will not remove the PL logic from the FPGA region. If the user wants to remove the complete configuration logic from the FPGA region he should reconfigure the target configuration region with grey-box logic.

Loading an overlay along with it's Bitstream

Usage: fpgautil -b <bin file path> -o <overlay file path>

root@xilinx-zcu102-2018_3:~#
root@xilinx-zcu102-2018_3:~# fpgautil -b /lib/firmware/design_1_wrapper/design_1_wrapper.bin -o /lib/firmwaree/design_1_wrapper/design_1_wrapper.dtbo 
[   43.642776] fpga_manager fpga0: writing design_1_wrapper.bin to Xilinx ZynqMP FPGA Manager
[   43.871149] XGpio: /amba/gpio@a0000000: registered, base is 298
[   43.877420] XGpio: /amba/gpio@a0000000: dual channel registered, base is 290
Time taken to load DTBO is 244.000000 Milli Seconds
DTBO loaded through zynqMP FPGA manager successfully
root@xilinx-zcu102-2018_3:~# 
root@xilinx-zcu102-2018_3:/sys/class/gpio# dmesg |grep gpio
[    5.399518] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[   43.871149] XGpio: /amba/gpio@a0000000: registered, base is 298
[   43.877420] XGpio: /amba/gpio@a0000000: dual channel registered, base is 290
root@xilinx-zcu102-2018_3:/sys/class/gpio# dmesg |grep gpio
[    5.399518] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[   43.871149] XGpio: /amba/gpio@a0000000: registered, base is 298
[   43.877420] XGpio: /amba/gpio@a0000000: dual channel registered, base is 290
root@xilinx-zcu102-2018_3:/sys/class/gpio# echo 290 > export 
root@xilinx-zcu102-2018_3:/sys/class/gpio# cd gpio290
root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290# ls
active_low  device      direction   edge        power       subsystem   uevent      value
root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290# echo 1 out > direction 
root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290# echo 0 > value 
root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290# echo 1 > value 
root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290#

Removing an overlay

Usage: fpgautil -R

root@xilinx-zcu102-2018_3:~# fpgautil -R 
root@xilinx-zcu102-2018_3:~#

Loading Bitstream only

Usage: fpgautil -b <bit/bin file absolute path>

root@xilinx-zcu102-2018_3:~# fpgautil  -b /mnt/design_1_wrapper.bit
Time taken to load BIN is 213.000000 Milli Seconds
BIN FILE loaded through zynqMP FPGA manager successfully

Loading Partial Bitstream that doesn't require PL drivers

Usage: fpgautil -b <bit/bin file full path> -f Partial

root@xilinx-zcu102-2018_3:~# fpgautil -b pr1rm1.bin -f Partial
Flags : 0x1
Time taken to load BIN is 56.000000 Milli Seconds
BIN FILE loaded through zynqMP FPGA manager successfully

Loading Secure Authenticated Bitstream

Usage: fpgautil -b <bit/bin file full path> -f <flags> -s AuthDDR

root@xilinx-zcu102-2018_3:~# fpgautil -b Authentication_system_wrapper.bin -f Full -s AuthDDR
Flags : 0x0
Flags : 0x2
[  467.885628] fpga_manager fpga0: writing Authentication_system_wrapper.bin to Xilinx ZynqMP FPGA Manager
Time taken to load BIN FILE is 263.000000 Milli Seconds
BIN FILE loaded through zynqMP FPGA manager successfully

Read back of FPGA Configuration Registers

Usage: fpgautil -r

root@xilinx-zcu102-2018_3:~#fpgautil -r
Readback contents are stored in the file readback.bin
root@xilinx-zcu102-2018_3:~# vi readback.bin 
zynqMP FPGA Configuration register contents are
CRC --> 	 0 	
FAR --> 	 7fc0000 	
FDRI --> 	 0 	
FDRO --> 	 effffffe 	
CMD --> 	 d 	
CTRL0 --> 	 101 	
MASK --> 	 0 	
STAT --> 	 16907ffc 	
LOUT --> 	 0 	
COR0 --> 	 38003fe5 	
MFWR --> 	 0 	
CBC --> 	 0 	
IDCODE --> 	 1484a093 	
AXSS --> 	 0 	
COR1 --> 	 400000 	
WBSTR --> 	 0 	
TIMER --> 	 0 	
BOOTSTS --> 	 1 	
CTRL1 --> 	 0

Read back of FPGA Configuration Data (Frames)

Usage: fpgautil -r <filename> -t 1

root@xilinx-zcu102-2018_3:~# fpgautil -r readbackdata -t 1
root@xilinx-zcu102-2018_3:~# hexdump readbackdata.bin > data.txt

Verification of Read-back Data

To verify the readback data stream, compare it to the RBD golden readback file and masking readback bits with the MSD file. This approach is simple because there is a 1:1 correspondence between the start of the readback data stream and the start of the RBD and MSD files, making the task of aligning readback, mask, and expected data easier.
The RBD and MSD files contain an ASCII representation of the readback and mask data along with a file header that lists the file name, etc. This header information should be ignored or deleted. The ASCII 1s and 0s in the RBD and MSD files correspond to the binary readback data from the device.
For generating RBD and MSD refer UG908 (Vivado Design Suite User Guide)
By compiling the below utility(verify_readback.c file) user can verify the read back contents.
Source code: verify_readback

Note:

utility compares the readback with the RBD golden file, by masking readback bits with the MSD file. It is expected from user to remove/strip the header in the msd and rdb files and also remove the print statement in the readback contents file(readback.bin).

Example

$./verify_readback -msd design_1_wrapper.msd -rbd design_1_wrapper.rbd -data reabackdata_2.bin 
=============================================
Starting verification of readback data...
 
>>>> Readback data is correct! <<<<
=============================================

Exercising FPGA programming using fpga framework attributes

FPGA programming using sysfs attributes

Linux FPGA Manager driver creates /sys/class/fpga_manager/<fpga> folder, which contains file attributes that provides control and state.

firmware: /sys/class/fpga_manager/<fpga>/firmware
firmware attribute requests an fpga image using the firmware class, then write output to the FPGA
flags: /sys/class/fpga_manager/<fpga>/flags
flags attribute determines the type of Bitstream
0 - Full Bitstream 1 - Partial Bitstream (default: 0)
state: /sys/class/fpga_manager/<fpga>/state
state attribute is a superset of FPGA states and FPGA Manager driver states
name: /sys/class/fpga_manager/<fpga>/name
name attribute is name of the low level FPGA Manager driver
key: /sys/class/fpga_manager/<fpga>/key
key attribute stores the key value useful for Encrypted Bitstream loading to read the userkey

Steps for programming the Full Bitstream

Set flags for Full Bitstream

echo 0 > /sys/class/fpga_manager/fpga0/flags

Load the Bitstream

mkdir -p /lib/firmware
cp /media/design_1_wrapper.bin /lib/firmware/
echo design_1_wrapper.bin > /sys/class/fpga_manager/fpga0/firmware

Steps for programming the Partial Bitstream

Set flags for Partial Bitstream

echo 1 > /sys/class/fpga_manager/fpga0/flags

Load the Bitstream Partial Bitstream

mkdir -p /lib/firmware
cp /media/partail_wrapper.bin /lib/firmware/
echo partail_wrapper.bin > /sys/class/fpga_manager/fpga0/firmware

FPGA Readback using debugfs attributes

Linux FPGA Manager driver creates /sys/kernel/debug/fpga/ folder, which contains image attributes that provides readback contents.

readback_type: /sys/module/zynqmp_fpga/parameters/readback_type
readback_type is a module parameter which tells the readback type (default : 0)
0 – Configuration Register read 1 - Configuration Data read
image: /sys/kernel/debug/fpga/<fpga>/image
image is a debugfs attribute which provides the readback information based on the readback_type module param

Steps for Readback of Configuration Registers

Set flags for readback type

echo 0 > /sys/module/zynqmp_fpga/parameters/readback_type

Perform Readback operation by reading the image debug attribute

cat /sys/kernel/debug/fpga/fpga0/image

Example:

root@xilinx-zcu102-2018_3:~# echo 0 > /sys/module/zynqmp_fpga/parameters/readback_type
root@xilinx-zcu102-2018_3:~# cat /sys/kernel/debug/fpga/fpga0/image
zynqMP FPGA Configuration register contents are
CRC -->   0 
FAR -->   7fc0000   
FDRI -->      0 
FDRO -->      effffffe  
CMD -->   d 
CTRL0 -->     101   
MASK -->      0 
STAT -->      16907ffc  
LOUT -->      0 
COR0 -->      38003fe5  
MFWR -->      0 
CBC -->   0 
IDCODE -->    1484a093  
AXSS -->      0 
COR1 -->      400000    
WBSTR -->     0 
TIMER -->     0 
BOOTSTS -->   1 
CTRL1 -->     0

Steps for Readback of Configuration DataFrames

Set flags for readback type

echo 1 > /sys/module/zynqmp_fpga/parameters/readback_type

Perform Readback operation by reading the image debug attribute

cat /sys/kernel/debug/fpga/fpga0/image

Example:

root@xilinx-zcu102-2018_3:~# echo 1 > /sys/module/zynqmp_fpga/parameters/readback_type
root@xilinx-zcu102-2018_3:~# cat /sys/kernel/debug/fpga/fpga0/image > reabackdata.bin
root@xilinx-zcu102-2018_3:~# hexdump readbackdata.bin > data.txt

FPGA programming using Device Tree Overlay (DTO)

The Device Tree Overlay (DTO) is used to reprogram an FPGA while Linux is running. The DTO overlay will add the child node and the fragments from the .dtbo file to the base device tree,, The newly added device node/drivers will be probed after Bitstream programming

DTO contains:

Target FPGA Region
- "target-path" or "target" - The insertion point where the the contents of the overlay will go into the live tree.
target-path is a full path, while target is a phandle.
FPGA Image firmware file name
- "firmware-name" - Specifies the name of the FPGA image file on the firmware search path.
The search path is described in the firmware class documentation.
Image specific information
- external-fpga-config : boolean, set if the FPGA has already been configured prior to Linux boot up.
Child devices
- child nodes corresponding to hardware that will be loaded in this region of the FPGA.

Notes:

If the user tries to load the un-aligned bit/bin file the PL configuration takes a longer time when compared with aligned(word-aligned) bit/bin files.
Using overlay’s we can add a new node or add/update the existing node properties. But it will not allow replacing the existing nodes which are already part of the live tree.

Steps for programming the Full Bitstream using overlay

Copy the Full Bitstream (.bin) and pl.dtbo files into firmware folder

mkdir /configfs
mount -t configfs configfs /configfs
cd /configfs/device-tree/overlays/
mkdir full
echo -n "pl.dtbo" > full/path

Steps to remove the drivers got added as part of DTO

rmdir /configfs/device-tree/overlays/full

Devicetree overlay file contents example:

//Device Tree Example: Full Reconfiguration along with PS PL Configuration (Clock, resets and AFI)
// HSI Generated overlay/pl.dtsi file.
// Enable the axi-gpio interface
/dts-v1/;
/plugin/;
/ {
	fragment@0 { /* Bitstream  Fragment */ 
		target = <&fpga_full>;
		overlay0: __overlay__ {
			#address-cells = <2>;
			#size-cells = <2>;
			firmware-name = "design_1_wrapper.bin";
			resets = <&rst 116>;
		};
	};
	fragment@1 { /* PS-PL configuration Fragment */
		target = <&amba>;
		overlay1: __overlay__ {
			afi0: afi0 {
				compatible = "xlnx,afi-fpga";
				config-afi = < 0 0>, <1 0>, <2 0>, <3 0>, <4 0>, <5 0>, <6 0>, <7 0>, <8 0>, <9 0>, <10 0>, <11 0>, <12 0>, <13 0>, <14 0>, <14 0>, <15 0x000>;
			};
			clocking0: clocking0 {
				#clock-cells = <0>;
				assigned-clock-rates = <99990005>;
				assigned-clocks = <&clk 71>;
				clock-output-names = "fabric_clk";
				clocks = <&clk 71>;
				compatible = "xlnx,fclk";
			};
		};
	};
	fragment@2 { /* PL Drivers Fragment */
		target = <&amba>;
		overlay2: __overlay__ {
			axi_gpio_0: gpio@a0000000 {
				#gpio-cells = <3>;
				#interrupt-cells = <2>;
				clock-names = "s_axi_aclk";
				clocks = <&clk 71>;
				compatible = "xlnx,axi-gpio-2.0", "xlnx,xps-gpio-1.00.a";
				gpio-controller ;
				interrupt-controller ;
				interrupt-names = "ip2intc_irpt";
				interrupt-parent = <&gic>;
				interrupts = <0 89 4>;
				reg = <0x0 0xa0000000 0x0 0x1000>;
				xlnx,all-inputs = <0x1>;
				xlnx,all-inputs-2 = <0x0>;
				xlnx,all-outputs = <0x0>;
				xlnx,all-outputs-2 = <0x1>;
				xlnx,dout-default = <0x00000000>;
				xlnx,dout-default-2 = <0xAAAAAAAA>;
				xlnx,gpio-width = <0x8>;
				xlnx,gpio2-width = <0x8>;
				xlnx,interrupt-present = <0x1>;
				xlnx,is-dual = <0x1>;
				xlnx,tri-default = <0xFFFFFFFF>;
				xlnx,tri-default-2 = <0xFFFFFFFF>;
			};
			psu_ctrl_ipi: PERIPHERAL@ff380000 {
				/* This is a place holder node for a custom IP, user may need to update the entries */
				compatible = "xlnx,PERIPHERAL-1.0";
				reg = <0x0 0xff380000 0x0 0x80000>;
			};
			psu_message_buffers: PERIPHERAL@ff990000 {
				/* This is a place holder node for a custom IP, user may need to update the entries */
				compatible = "xlnx,PERIPHERAL-1.0";
				reg = <0x0 0xff990000 0x0 0x10000>;
			};
		};
	};
};

Working with Device Tree Overlay (DTBO)

Copy the Partial Bitstream (.bin) and rm.dtbo files into lib/firmware folder

mkdir /configfs
mount -t configfs configfs /configfs
cd /configfs/device-tree/overlays/
mkdir partial
echo 1 > /sys/class/fpga_manager/fpga0/flags
echo -n "pl.dtbo" > partial/path

To remove a device tree overlay:

rmdir /configfs/device-tree/overlays/partial

Exercising FPGA programming using Libdfx

Refer this link: libdfx - Linux User Space Solution for FPGA Programming

Mainline Status

The current driver availble in the xilinx linux git is in sync with the open source 6.6 kernel driver except for the following

Encrypted and Authenticated Full/Partial Bitstream loading
Readback of Configuration Registers
Readback of Bitstream(Configuration Data)
Use the scatterlist interface

Release history

2024.2

None.

2024.1

FPGA Manager framework upgraded to 6.6v

2023.2

Revert "fpga: zynqmp: Initialized variables before using it"

2023.1

FPGA Manager framework upgraded to 6.1v

2022.3

None.

2022.2

fpga: zynqmp: Add feature-list validation check
firmware: xilinx: Update the zynqmp_pm_fpga_load() API
fpga: zynqmp-fpga: Remove empty function
fpga: zynqmp: Make word align the configuration data

2022.1

FPGA Manager framework upgraded to 5.15v

2021.2

None

2021.1

FPGA Manager framework upgraded to 5.10v

2020.2

Summary:

fpga: zynqmp: Use the scatterlist interface
fpga: zynqmp: Initialized variables before using it
fpga: zynqmp: Fix incorrect variables type

Commits:

4823227 fpga: zynqmp: Use the scatterlist interface
aac8be7 fpga: zynqmp: Initialized variables before using it
2899bc8 fpga: zynqmp: Fix incorrect variables type

2020.1

None

2019.2

Summary:

fpga: zynqmp-fpga: Adds status interface

Commits:

8e85861 fpga: zynqmp-fpga: Adds status interface

2019.1

Summary:

fpga: Fix bitstream typo error
Merge tag 'v4.19' into master
fpga: ZynqMP: Adds support for Authentication of bitstreams usning User-key
drivers: xilinx: Reorganize firmware driver for zynqmp
drivers: Defer probe if firmware is not ready
fpga: zynqmp: Revert Authentication of bitstreams using User-key changes
fpga: zynqmp: Use SPDX license header

Commits:

e732840 fpga: Fix bitstream typo error
26a8a4d Merge tag 'v4.19' into master
c609186 fpga: ZynqMP: Adds support for Authentication of bitstreams usning User-key
3f9e46d drivers: xilinx: Reorganize firmware driver for zynqmp
6a63448 drivers: Defer probe if firmware is not ready
8d43780 fpga: zynqmp: Revert Authentication of bitstreams using User-key changes
3384284 fpga: zynqmp: Use SPDX license header

2018.3

Summary:

Added support for vivado generated bit and bin file loading
Added support for PL configuration readback
Added support for clock framework

Commits:

daca3d fpga: zynqmp: Adds support to load vivado generated .bit and .bin files
bd1f10 fpga: zynqmp-fpga: Add support for pl configuration readback
097ea7 fpga: zynqmp-fpga: Add support for clock framework

2018.2
Summary:

Added Support for Partial Reconfiguration

Commits:

No Changes in the driver (Changes are done in the xilfpga library).

2018.1
Summary:

soc: zynqmp: Define EEMI ops structure
soc: zynqmp: Use new firmware APIs
fpga: zynqmp: sync driver with xilfpga library enhancements
fpga: zynqmp: Remove useless blank line in zynqmp_fpga_remove

Commits:

31c2a5e soc: zynqmp: Define EEMI ops structure
0b7483c soc: zynqmp: Use new firmware APIs
45ec97f fpga: zynqmp: sync driver with xilfpga library enhancements
fbf0102 fpga: zynqmp: Remove useless blank line in zynqmp_fpga_remove

2017.4

None

2017.3
Summary:

zynqmp: Use new firmware.h instead of pm.h
Revert "fpga manager: Adopted Authenticated Bitstream loading support for Xilinx"

Commits:

5e81ba5 zynqmp: Use new firmware.h instead of pm.h
a7fbcf3 Revert "fpga manager: Adopted Authenticated Bitstream loading support for Xilinx"

2017.2

None

2017.1
Summary:

fpga manager: Adopted Encrypted Bitstream loading support for Xilinx ZynqMp.
fpga: zynqmp: Fix ZynqMP name in print.
fpga manager: Adopted Authenticated Bitstream loading support for Xilinx.
fpga: zynqmp: Add fpga image information struct.

Commits:

a17a6e1 fpga manager: Adopted Encrypted Bitstream loading support for Xilinx ZynqMp.
7130ff7 fpga: zynqmp: Fix ZynqMP name in print.
ed5a141 fpga manager: Adopted Authenticated Bitstream loading support for Xilinx.
b8bc48c fpga: zynqmp: Add fpga image information struct.

2016.4

None

2016.3
Summary:

fpga manager Adding FPGA Manager support for Xilinx zynqmp Soc
fpga: Removed warning from zynqmp-fpga.c compilation.

Commits:

280ca3f fpga manager: Adding FPGA Manager support for Xilinx zynqmp Soc
ad53092 fpga: Removed -warning from zynqmp-fpga.c compilation.

References

https://github.com/Xilinx/linux-xlnx/blob/master/drivers/fpga/zynqmp-fpga.c

https://github.com/Xilinx/linux-xlnx/blob/master/Documentation/driver-api/fpga

https://github.com/Xilinx/embeddedsw/tree/master/lib/sw_services/xilfpga/doc

Introduction‍

HW IP Features

Features supported in the driver

Known Issues for PL Programming

Summary Table for Partial Reconfiguration / Dynamic Function Exchange Feature Support

Kernel Configuration

Devicetree

Bitstream Format

Building Software Images

PetaLinux Flow

Yocto Flow

OSL Flow

FPGA programming

Exercising FPGA programming using fpgautil

Fpgautil usage for DFx (aka PR/RP)use cases

Loading an overlay along with it's Bitstream

Removing an overlay

Loading Bitstream only

Loading Partial Bitstream that doesn't require PL drivers

Loading Secure Authenticated Bitstream

Read back of FPGA Configuration Registers

Read back of FPGA Configuration Data (Frames)

Verification of Read-back Data

Exercising FPGA programming using fpga framework attributes

FPGA programming using sysfs attributes

Steps for programming the Full Bitstream

Steps for programming the Partial Bitstream

FPGA Readback using debugfs attributes

Steps for Readback of Configuration Registers

Steps for Readback of Configuration DataFrames

FPGA programming using Device Tree Overlay (DTO)

Steps for programming the Full Bitstream using overlay

Steps to remove the drivers got added as part of DTO

Working with Device Tree Overlay (DTBO)

Exercising FPGA programming using Libdfx

Mainline Status

Release history

See also

References