The Zynq UltraScale+ MPSoC Programmable Logic (PL) can be programmed either using First Stage Boot-loader(FSBL), U-Boot or through Linux.
This page provides details about programming the PL from the Linux world using the Linux FPGA Manager framework.
Flow:
A few considerations that users should be aware of in programming the PL/FPGA are as follows.
Example DT node.
/dts-v1/; /plugin/; / { fragment@0 { target = <&fpga_full>; overlay0: __overlay__ { #address-cells = <2>; #size-cells = <2>; firmware-name = "system.bin"; resets = <&zynqmp_reset 116>; }; }; fragment@1 { target = <&amba>; overlay1: __overlay__ { afi0: afi0 { compatible = "xlnx,afi-fpga"; config-afi = < 0 0>, <1 0>, <2 0>, <3 0>, <4 0>, <5 0>, <6 0>, <7 0>, <8 0>, <9 0>, <10 0>, <11 0>, <12 0>, <13 0>, <14 0xa00>, <15 0x000>; }; }; }; fragment@2 { target = <&amba>; overlay2: __overlay__ { #address-cells = <2>; #size-cells = <2>; axi_intc_1: interrupt-controller@a00d0000 { /* Top node */ #interrupt-cells = <2>; clock-names = "s_axi_aclk"; clocks = <&clk_wiz_0 0>; compatible = "xlnx,axi-intc-4.1", "xlnx,xps-intc-1.00.a"; interrupt-controller ; interrupt-names = "irq"; interrupt-parent = <&axi_intc_0>; interrupts = <1 2>; reg = <0x0 0xa00d0000 0x0 0x10000>; xlnx,kind-of-intr = <0x360>; xlnx,num-intr-inputs = <0xe>; }; axi_iic_0: i2c@a0000000 { #address-cells = <1>; #size-cells = <0>; clock-names = "s_axi_aclk"; clocks = <&clk_wiz_0 0>; compatible = "xlnx,axi-iic-2.1", "xlnx,xps-iic-2.00.a"; interrupt-names = "iic2intc_irpt"; interrupt-parent = <&gic>; interrupts = <0 91 4>; reg = <0x0 0xa0000000 0x0 0x1000>; }; axi_iic_1: i2c@a0008000 { #address-cells = <1>; #size-cells = <0>; clock-names = "s_axi_aclk"; clocks = <&clk_wiz_0 0>; compatible = "xlnx,axi-iic-2.1", "xlnx,xps-iic-2.00.a"; interrupt-names = "iic2intc_irpt"; interrupt-parent = <&axi_intc_1>; interrupts = <1 2>; reg = <0x0 0xa0008000 0x0 0x1000>; }; }; }; }; |
Partial Reconfiguration / Dynamic Function Exchange (DFX) is an advanced feature of Xilinx Zynq UltraScale+ MPSoC devices. Refer to the table below for details. For PetaLinux releases prior to 2018.3 or for building Linux system images manually, please refer to the Linux OSL Flow page.
Release \ Feature | Vivado RTL Design Flow | Vivado IPI Design Flow | Linux Device Tree Generator | Linux Programming Framework |
---|---|---|---|---|
<= 2018.3 | Supported | Not Supported | Automation Not Supported - Hand Crafted | FPGA Manager |
2019.1 - 2020.2 | Supported | DFX Lounge | DFX Lounge | FPGA Manager / fpgautil |
The following config options have to be enabled in order to use FPGA Manager, Please note that these options are enabled by default through xilinx_zynqmp_defconfig except for the fpga debugfs option. If the user wants to test the readback feature they have to enable it.
Zynq UltraScale+ MPSoC FPGA Manager Configuration:
Select: Device Drivers → FPGA Configuration Framework → Xilinx Zynq UltraScale+ MPSoC FPGA
In order to test readback feature user needs to enable FPGA debug fs
Select: Device Drivers → FPGA Configuration Framework --> FPGA debug fs
DT overlay ConfigFS Interface Configuration:
In order to load Bitstream with DTBO user needs to enable the below options
Select: Device Drivers --> Device Tree and Open Firmware support
Contiguous Memory Allocator Configuration:
CONFIG_CMA
Select: Kernel Features --> Contiguous Memory Allocator
CONFIG_DMA_CMA
Select: Device Drivers --> Generic Driver Options → DMA Contiguous Memory Allocator
pcap { compatible = "xlnx,zynqmp-pcap-fpga"; clocks = <&clk 41>; }; fpga_full: fpga-full { compatible = "fpga-region"; fpga-mgr = <&pcap>; #address-cells = <2>; #size-cells = <2>; power-domains = <&zynqmp_firmware PL_PD>; }; |
For more details about devicetree bindings Reference the Below link: Devicetree
Note: The above devicetree node is by default present in the zynqmp.dtsi file
From the 2018.3 release onwards FPGA Manager supports loading of vivado and bootgen generated Bitstream and Bin files vivadobootgen[1]
Note: For releases earlier to 2018.3 FPGA Manager was capable of loading only bootgen generated bin files. bootgen
.Bit Format
.Bin Format
all: { [destination_device = pl] Full.bit (or) Partial.bit /* Bitstream file name */ } |
Xilinx tools (Petalinux, Yocto) provide an option with the name FPGA Manager which if enabled builds the device tree overlay fragments automatically and copies the Bitstream and DTBO files into root file system.
PetaLinux setup and build: PetaLinux
petalinux-create -t project -n zynqmp-dfx --template versal |
petalinux-create -t project -s <bsp path> -n zynqmp-dfx |
cd zynqmp-dfx |
petalinux-config --get-hw-description <base.xsa/static.xsa> |
petalinux-config -> FPGA MANAGER |
petalinux-create -t apps --template fpgamanager_dtg -n <static-app> --enable --srcuri "<static xsa>" |
The previous command creates and packages the static dtbo and pdi files into the rootfs (/lib/firmware/xilinx/) using fpgamanager_dtg template.
--static-pn
command line option to define the relation between base and partial.petalinux-create -t apps --template fpgamanager_dtg_dfx -n <rm-app> --enable --srcuri <rm.xsa>" --static-pn <static-app> |
This command generates and packages the rm dtbo, pdi files into the rootfs (/lib/firmware/xilinx/<static-app>/<rm-app>).
petalinux-build |
The command generates the rootfs containing both static and rprm dtbos, and respective pdi files as mentioned in the step 5 and step 6.
Once the base boot images ready with the previous step, if you want to build only DFX apps, use the commands mentioned in step 8.
petalinux-build -c <static-app> petalinux-build -c <rm-app> in below path you will see with the below names. in <TMPDIR>/deploy/rpm you will see <static-app>.rpm and <rm-app>.rpm |
For more details relevant to the above steps refer to this link: Petalinux-DFX-Support
Once the build is complete, binaries are available at images/linux directory.
Yocto setup and build: yocto
Once Yocto environment is set
EXTRA_IMAGE_FEATURES += " fpga-manager" EXTRA_HDF = "<PATH_TO_HDF_or_XSA>" |
Note:
2. Build the required images (run the below command)
$ bitbake petalinux-image-minimal |
Once the build is complete, binaries are available at ${DEPLOY_DIR_IMAGE} (${TMPDIR}/deploy/images/${MACHINE}/) directory
3. Boot the hardware with newly built images
Setup and Build: OSL
Images required for testing the FPGA Manager
DTBO File Generation:
Ex: xsct dt_overaly.tcl system.hdf psu_cortexa53_0 ${DTS_REPO}/device-tree-xlnx overlay
2. Create DeviceTree Overlay Blob (.dtbo) file from the pl.dtsi file
# dtc -O dtb -o pl.dtbo -b 0 -@ pl.dtsi
Ex: ./scripts/dtc/dtc -O dtb -o pl.dtbo -b 0 -@ pl.dtsi
3. Copy the generated bit and dtbo files to target
Note: PL nodes should not be the part of Base Device-tree (system.dtb).
Source code for the fpgautil is available in below meta layers.
Note: In 2022.1 release fpgautil source code is moved from meta-xilinx-tools to meta-xilinx/meta-xilinx-core layer. |
2022.1 and later release: https://github.com/Xilinx/meta-xilinx/blob/master/meta-xilinx-core/recipes-bsp/fpga-manager-script/files/fpgautil.c
2021.2 and earlier release: https://github.com/Xilinx/meta-xilinx-tools/blob/rel-v2021.2/recipes-bsp/fpga-manager-script/files/fpgautil.c
root@xilinx-zcu102-2018_3:~# fpgautil -h fpgautil: FPGA Utility for Loading/reading PL Configuration in zynqMP Usage: fpgautil -b <bin file path> -o <dtbo file path> Options: -b <binfile> (Bin file path) -o <dtbofile> (DTBO file path) -f <flags> Optional: <Bitstream type flags> f := <Full | Partial > Default: <Full> -s <secure flags> Optional: <Secure flags> s := <AuthDDR | AuthOCM | EnUsrKey | EnDevKey | AuthEnUsrKeyDDR | AuthEnUsrK> -k <AesKey> Optional: <AES User Key> -r <Readback> Optional: <file name> Default: By default Read back contents will be stored in readback.bin file -t Optional: <Readback Type> 0 - Configuration Register readback 1 - Configuration Data Frames readback Default: 0 (Configuration register readback) -R Optional: Remove overlay from a live tree Examples: (Load Full Bitstream using Overlay) fpgautil -b top.bin -o can.dtbo (Load Partial Bitstream through the sysfs interface) fpgautil -b top.bin -f Partial (Load Authenticated Bitstream through the sysfs interface) fpgautil -b top.bin -f Full -s AuthDDR (Load Parital Encrypted Userkey Bitstream using Overlay) fpgautil -b top.bin -o pl.dtbo -f Partial -s EnUsrKey -k <32byte key value> (Read PL Configuration Registers) fpgautil -b top.bin -r |
Typical topology: FPGA has the base FPGA Region(static) with two PR/DFx regions (Each PR region has two RMs). The relevant files can be looked as below.
-- xilinx `-- static |-- static.dtbo |-- static.bin |---PR0
| `-- PR0-RM0
| |-- pr0-rm0.dtsi | |-- pr0-rm0.dtbo | |-- pr0-rm0.bin
| `-- PR0-RM1
| |-- pr0-rm1.dtsi
| |-- pr0-rm1.dtbo
| |-- pr0-rm1.bin
|---PR1
| `-- PR1-RM0
| |-- pr1-rm0.dtsi
| |-- pr1-rm0.dtbo
| |-- pr1-rm0.bin
| `-- PR0-RM1
| |-- pr1-rm1.dtsi
| |-- pr1-rm1.dtbo
| |-- pr1-rm1.bin
Notes:
Fpgautil -R is responsible for only removing the dtbo file from the livetree. it will not remove the PL logic from the FPGA region. If the user wants to remove the complete configuration logic from the FPGA region he should reconfigure the target configuration region with grey-box logic.
Usage: fpgautil -b <bin file path> -o <overlay file path>
root@xilinx-zcu102-2018_3:~# root@xilinx-zcu102-2018_3:~# fpgautil -b /lib/firmware/design_1_wrapper/design_1_wrapper.bin -o /lib/firmwaree/design_1_wrapper/design_1_wrapper.dtbo [ 43.642776] fpga_manager fpga0: writing design_1_wrapper.bin to Xilinx ZynqMP FPGA Manager [ 43.871149] XGpio: /amba/gpio@a0000000: registered, base is 298 [ 43.877420] XGpio: /amba/gpio@a0000000: dual channel registered, base is 290 Time taken to load DTBO is 244.000000 Milli Seconds DTBO loaded through zynqMP FPGA manager successfully root@xilinx-zcu102-2018_3:~# root@xilinx-zcu102-2018_3:/sys/class/gpio# dmesg |grep gpio [ 5.399518] input: gpio-keys as /devices/platform/gpio-keys/input/input0 [ 43.871149] XGpio: /amba/gpio@a0000000: registered, base is 298 [ 43.877420] XGpio: /amba/gpio@a0000000: dual channel registered, base is 290 root@xilinx-zcu102-2018_3:/sys/class/gpio# dmesg |grep gpio [ 5.399518] input: gpio-keys as /devices/platform/gpio-keys/input/input0 [ 43.871149] XGpio: /amba/gpio@a0000000: registered, base is 298 [ 43.877420] XGpio: /amba/gpio@a0000000: dual channel registered, base is 290 root@xilinx-zcu102-2018_3:/sys/class/gpio# echo 290 > export root@xilinx-zcu102-2018_3:/sys/class/gpio# cd gpio290 root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290# ls active_low device direction edge power subsystem uevent value root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290# echo 1 out > direction root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290# echo 0 > value root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290# echo 1 > value root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290# |
Usage: fpgautil -R
root@xilinx-zcu102-2018_3:~# fpgautil -R root@xilinx-zcu102-2018_3:~# |
Usage: fpgautil -b <bit/bin file absolute path>
root@xilinx-zcu102-2018_3:~# fpgautil -b /mnt/design_1_wrapper.bit Time taken to load BIN is 213.000000 Milli Seconds BIN FILE loaded through zynqMP FPGA manager successfully |
Usage: fpgautil -b <bit/bin file full path> -f Partial
root@xilinx-zcu102-2018_3:~# fpgautil -b pr1rm1.bin -f Partial Flags : 0x1 Time taken to load BIN is 56.000000 Milli Seconds BIN FILE loaded through zynqMP FPGA manager successfully |
Usage: fpgautil -b <bit/bin file full path> -f <flags> -s AuthDDR
root@xilinx-zcu102-2018_3:~# fpgautil -b Authentication_system_wrapper.bin -f Full -s AuthDDR Flags : 0x0 Flags : 0x2 [ 467.885628] fpga_manager fpga0: writing Authentication_system_wrapper.bin to Xilinx ZynqMP FPGA Manager Time taken to load BIN FILE is 263.000000 Milli Seconds BIN FILE loaded through zynqMP FPGA manager successfully |
Usage: fpgautil -r
root@xilinx-zcu102-2018_3:~#fpgautil -r Readback contents are stored in the file readback.bin root@xilinx-zcu102-2018_3:~# vi readback.bin zynqMP FPGA Configuration register contents are CRC --> 0 FAR --> 7fc0000 FDRI --> 0 FDRO --> effffffe CMD --> d CTRL0 --> 101 MASK --> 0 STAT --> 16907ffc LOUT --> 0 COR0 --> 38003fe5 MFWR --> 0 CBC --> 0 IDCODE --> 1484a093 AXSS --> 0 COR1 --> 400000 WBSTR --> 0 TIMER --> 0 BOOTSTS --> 1 CTRL1 --> 0 |
Usage: fpgautil -r <filename> -t 1
root@xilinx-zcu102-2018_3:~# fpgautil -r readbackdata -t 1 root@xilinx-zcu102-2018_3:~# hexdump readbackdata.bin > data.txt |
To verify the readback data stream, compare it to the RBD golden readback file and masking readback bits with the MSD file. This approach is simple because there is a 1:1 correspondence between the start of the readback data stream and the start of the RBD and MSD files, making the task of aligning readback, mask, and expected data easier.
The RBD and MSD files contain an ASCII representation of the readback and mask data along with a file header that lists the file name, etc. This header information should be ignored or deleted. The ASCII 1s and 0s in the RBD and MSD files correspond to the binary readback data from the device.
For generating RBD and MSD refer UG908 (Vivado Design Suite User Guide)
By compiling the below utility(verify_readback.c file) user can verify the read back contents.
Source code: verify_readback
Note:
utility compares the readback with the RBD golden file, by masking readback bits with the MSD file. It is expected from user to remove/strip the header in the msd and rdb files and also remove the print statement in the readback contents file(readback.bin).
Example
$./verify_readback -msd design_1_wrapper.msd -rbd design_1_wrapper.rbd -data reabackdata_2.bin ============================================= Starting verification of readback data... >>>> Readback data is correct! <<<< ============================================= |
Linux FPGA Manager driver creates /sys/class/fpga_manager/<fpga> folder, which contains file attributes that provides control and state.
Set flags for Full Bitstream
Load the Bitstream
mkdir -p /lib/firmware
cp /media/design_1_wrapper.bin /lib/firmware/
echo design_1_wrapper.bin > /sys/class/fpga_manager/fpga0/firmware
Set flags for Partial Bitstream
Load the Bitstream Partial Bitstream
mkdir -p /lib/firmware
cp /media/partail_wrapper.bin /lib/firmware/
echo partail_wrapper.bin > /sys/class/fpga_manager/fpga0/firmware
Linux FPGA Manager driver creates /sys/kernel/debug/fpga/ folder, which contains image attributes that provides readback contents.
Set flags for readback type
echo 0 > /sys/module/zynqmp_fpga/parameters/readback_type
Perform Readback operation by reading the image debug attribute
cat /sys/kernel/debug/fpga/fpga0/image
Example:
root@xilinx-zcu102-2018_3:~# echo 0 > /sys/module/zynqmp_fpga/parameters/readback_type root@xilinx-zcu102-2018_3:~# cat /sys/kernel/debug/fpga/fpga0/image zynqMP FPGA Configuration register contents are CRC --> 0 FAR --> 7fc0000 FDRI --> 0 FDRO --> effffffe CMD --> d CTRL0 --> 101 MASK --> 0 STAT --> 16907ffc LOUT --> 0 COR0 --> 38003fe5 MFWR --> 0 CBC --> 0 IDCODE --> 1484a093 AXSS --> 0 COR1 --> 400000 WBSTR --> 0 TIMER --> 0 BOOTSTS --> 1 CTRL1 --> 0 |
Set flags for readback type
echo 1 > /sys/module/zynqmp_fpga/parameters/readback_type
Perform Readback operation by reading the image debug attribute
cat /sys/kernel/debug/fpga/fpga0/image
Example:
root@xilinx-zcu102-2018_3:~# echo 1 > /sys/module/zynqmp_fpga/parameters/readback_type root@xilinx-zcu102-2018_3:~# cat /sys/kernel/debug/fpga/fpga0/image > reabackdata.bin root@xilinx-zcu102-2018_3:~# hexdump readbackdata.bin > data.txt |
The Device Tree Overlay (DTO) is used to reprogram an FPGA while Linux is running. The DTO overlay will add the child node and the fragments from the .dtbo file to the base device tree,, The newly added device node/drivers will be probed after Bitstream programming
DTO contains:
Notes:
|
Copy the Full Bitstream (.bin) and pl.dtbo files into firmware folder
mkdir /configfs
mount -t configfs configfs /configfs
cd /configfs/device-tree/overlays/
mkdir full
echo -n "pl.dtbo" > full/path
Devicetree overlay file contents example:
//Device Tree Example: Full Reconfiguration along with PS PL Configuration (Clock, resets and AFI) // HSI Generated overlay/pl.dtsi file. // Enable the axi-gpio interface /dts-v1/; /plugin/; / { fragment@0 { /* Bitstream Fragment */ target = <&fpga_full>; overlay0: __overlay__ { #address-cells = <2>; #size-cells = <2>; firmware-name = "design_1_wrapper.bin"; resets = <&rst 116>; }; }; fragment@1 { /* PS-PL configuration Fragment */ target = <&amba>; overlay1: __overlay__ { afi0: afi0 { compatible = "xlnx,afi-fpga"; config-afi = < 0 0>, <1 0>, <2 0>, <3 0>, <4 0>, <5 0>, <6 0>, <7 0>, <8 0>, <9 0>, <10 0>, <11 0>, <12 0>, <13 0>, <14 0>, <14 0>, <15 0x000>; }; clocking0: clocking0 { #clock-cells = <0>; assigned-clock-rates = <99990005>; assigned-clocks = <&clk 71>; clock-output-names = "fabric_clk"; clocks = <&clk 71>; compatible = "xlnx,fclk"; }; }; }; fragment@2 { /* PL Drivers Fragment */ target = <&amba>; overlay2: __overlay__ { axi_gpio_0: gpio@a0000000 { #gpio-cells = <3>; #interrupt-cells = <2>; clock-names = "s_axi_aclk"; clocks = <&clk 71>; compatible = "xlnx,axi-gpio-2.0", "xlnx,xps-gpio-1.00.a"; gpio-controller ; interrupt-controller ; interrupt-names = "ip2intc_irpt"; interrupt-parent = <&gic>; interrupts = <0 89 4>; reg = <0x0 0xa0000000 0x0 0x1000>; xlnx,all-inputs = <0x1>; xlnx,all-inputs-2 = <0x0>; xlnx,all-outputs = <0x0>; xlnx,all-outputs-2 = <0x1>; xlnx,dout-default = <0x00000000>; xlnx,dout-default-2 = <0xAAAAAAAA>; xlnx,gpio-width = <0x8>; xlnx,gpio2-width = <0x8>; xlnx,interrupt-present = <0x1>; xlnx,is-dual = <0x1>; xlnx,tri-default = <0xFFFFFFFF>; xlnx,tri-default-2 = <0xFFFFFFFF>; }; psu_ctrl_ipi: PERIPHERAL@ff380000 { /* This is a place holder node for a custom IP, user may need to update the entries */ compatible = "xlnx,PERIPHERAL-1.0"; reg = <0x0 0xff380000 0x0 0x80000>; }; psu_message_buffers: PERIPHERAL@ff990000 { /* This is a place holder node for a custom IP, user may need to update the entries */ compatible = "xlnx,PERIPHERAL-1.0"; reg = <0x0 0xff990000 0x0 0x10000>; }; }; }; }; |
Copy the Partial Bitstream (.bin) and rm.dtbo files into lib/firmware folder
mkdir /configfs
mount -t configfs configfs /configfs
cd /configfs/device-tree/overlays/
mkdir partial
echo -n "pl.dtbo" > partial/path
To remove a device tree overlay:
Refer this link: libdfx - Linux User Space Solution for FPGA Programming
The current driver availble in the xilinx linux git is in sync with the open source 6.6 kernel driver except for the following
2023.2
2023.1
2022.3
2022.2
2022.1
FPGA Manager framework upgraded to 5.15v
2021.2
2021.1
FPGA Manager framework upgraded to 5.10v
2020.2
Summary:
Commits:
2020.1
2019.2
Summary:
Commits:
2019.1
Summary:
Commits:
2018.3
Summary:
Commits:
2018.2
Summary:
Commits:
2018.1
Summary:
Commits:
2017.4
2017.3
Summary:
Commits:
2017.2
2017.1
Summary:
Commits:
2016.4
2016.3
Summary:
Commits:
https://github.com/Xilinx/linux-xlnx/blob/master/drivers/fpga/zynqmp-fpga.c
https://github.com/Xilinx/linux-xlnx/blob/master/Documentation/fpga/fpga-mgr.txt
https://github.com/Xilinx/embeddedsw/tree/master/lib/sw_services/xilfpga/doc