The Zynq UltraScale+ MPSoC Programmable Logic (PL) can be programmed either using First Stage Boot-loader(FSBL), U-Boot or through Linux.
This page provides the details about programming the PL from Linux world using Linux FPGA Manager framework.
Flow:
The Xilinx device tree generator (https://github.com/Xilinx/device-tree-xlnx) currently lacks automated support for device tree overlay generation for Partial Reconfiguration / DFX designs. For systems requiring runtime device tree overlay (eg, Linux device driver support) support, hand-crafted device trees can be deployed and loaded. See the "Working with Device Tree Overlay (DTBO)" section below for details on how to load these at runtime.
Occasionally, bare-metal applications relying on the XilFPGA library may produce an error getting the DONE status. See Answer Record 70504 for more details - https://www.xilinx.com/support/answers/70504.html
Partial Reconfiguration / Dynamic Function Exchange (DFX) is an advanced feature of Xilinx Zynq UltraScale+ MPSoC devices. Refer to the table below for details. For PetaLinux releases prior to 2018.3 or for building Linux system images manually, please refer to the Linux OSL Flow page.
Release \ Feature | Vivado RTL Design Flow | Vivado IPI Design Flow | Linux Device Tree Generator | Linux Programming Framework |
---|---|---|---|---|
<= 2018.3 | Supported | Not Supported | Automation Not Supported - Hand Crafted | FPGA Manager |
2019.1 - 2020.2 | Supported | DFX Lounge | DFX Lounge | FPGA Manager / fpgautil |
The following config options has to be enabled in order to use FPGA Manager, Please note that these options are enabled by default through xilinx_zynqmp_defconfig except for the fpga debugfs option. If user wants to test readback feature they have enable it.
Zynq UltraScale+ MPSoC FPGA Manager Configuration:
Select: Device Drivers → FPGA Configuration Framework → Xilinx Zynq UltraScale+ MPSoC FPGA
In-order to test readback feature user needs to enable FPGA debug fs
Select: Device Drivers → FPGA Configuration Framework --> FPGA debug fs
DT overlay ConfigFS interface Configuration:
In-order to load Bitstream with DTBO user needs to enable below options
Select: Device Drivers --> Device Tree and Open Firmware support
Contiguous Memory Allocator Configuration:
CONFIG_CMA
Select: Kernel Features --> Contiguous Memory Allocator
CONFIG_DMA_CMA
Select: Device Drivers --> Generic Driver Options → DMA Contiguous Memory Allocator
pcap { compatible = "xlnx,zynqmp-pcap-fpga"; clocks = <&clk 41>; }; fpga_full: fpga-full { compatible = "fpga-region"; fpga-mgr = <&pcap>; #address-cells = <2>; #size-cells = <2>; }; |
For more details about devicetree bindings Reference Below link: Devicetree
Note: Above devicetree node is by default present in the zynqmp.dtsi file
From 2018.3 release onwards FPGA Manager supports loading of vivado and bootgen generated Bitstream and Bin files vivadobootgen[1]
Note: For releases earlier to 2018.3 FPGA Manager was capable of loading only bootgen generated bin files. bootgen
.Bit Format
.Bin Format
all: { [destination_device = pl] Full.bit (or) Partial.bit /* Bitstream file name */ } |
Xilinx tools (Petalinux, Yocto) provide an option with the name FPGA Manager which if enabled builds the device tree overlay fragments automatically and copies the Bitstream and DTBO files into root file system.
Petalinux setup and build: Petalinux
# petalinux-config
(OR)
Edit file build/conf/plnxtool.conf
Add below line
IMAGE_FEATURES += " fpga-manager"
2. Select FPGA Manager user
3. Select the Specify hw directory path, in the prompt provide HDF path
Note:
4. Use petalinux-build command to build the required images
# petalinux-build
Once build is complete, binaries are available at images/linux directory
5. Boot the hardware with newly built images
Yocto setup and build: yocto
Once Yocto environment is set
IMAGE_FEATURES += " fpga-manager"
EXTRA_HDF = "Path to the HDF"
Note:
2. Build the required images (run below command)
# bitbake core-image-minimal
Once build is complete, binaries are available at images/linux directory
3. Boot the hardware with newly built images
Setup and Build: OSL
Images required for testing the FPGA Manager
DTBO File Generation:
Ex: xsct dt_overaly.tcl system.hdf psu_cortexa53_0 ${DTS_REPO}/device-tree-xlnx overlay
2. Create Device Tree Overlay Blob (.dtbo) file from the pl.dtsi file
# dtc -O dtb -o pl.dtbo -b 0 -@ pl.dtsi
Ex: ./scripts/dtc/dtc -O dtb -o pl.dtbo -b 0 -@ pl.dtsi
3. Copy the generated bit and dtbo files to target
Note: PL nodes should not be the part of Base Device-tree (system.dtb).
Source code for the fpgautil is available here fpgautil.c
root@xilinx-zcu102-2018_3:~# fpgautil -h fpgautil: FPGA Utility for Loading/reading PL Configuration in zynqMP Usage: fpgautil -b <bin file path> -o <dtbo file path> Options: -b <binfile> (Bin file path) -o <dtbofile> (DTBO file path) -f <flags> Optional: <Bitstream type flags> f := <Full | Partial > Default: <Full> -s <secure flags> Optional: <Secure flags> s := <AuthDDR | AuthOCM | EnUsrKey | EnDevKey | AuthEnUsrKeyDDR | AuthEnUsrK> -k <AesKey> Optional: <AES User Key> -r <Readback> Optional: <file name> Default: By default Read back contents will be stored in readback.bin file -t Optional: <Readback Type> 0 - Configuration Register readback 1 - Configuration Data Frames readback Default: 0 (Configuration register readback) -R Optional: Remove overlay from a live tree Examples: (Load Full Bitstream using Overlay) fpgautil -b top.bit.bin -o can.dtbo (Load Partial Bitstream through the sysfs interface) fpgautil -b top.bit.bin -f Partial (Load Authenticated Bitstream through the sysfs interface) fpgautil -b top.bit.bin -f Full -s AuthDDR (Load Parital Encrypted Userkey Bitstream using Overlay) fpgautil -b top.bit.bin -o pl.dtbo -f Partial -s EnUsrKey -k <32byte key value> (Read PL Configuration Registers) fpgautil -b top.bit.bin -r |
Usage: fpgautil -b <bin file path> -o <overlay file path>
root@xilinx-zcu102-2018_3:~# root@xilinx-zcu102-2018_3:~# fpgautil -b /lib/firmware/design_1_wrapper/design_1_wrapper.bit.bin -o /lib/firmwaree/design_1_wrapper/design_1_wrapper.dtbo [ 43.642776] fpga_manager fpga0: writing design_1_wrapper.bit.bin to Xilinx ZynqMP FPGA Manager [ 43.871149] XGpio: /amba/gpio@a0000000: registered, base is 298 [ 43.877420] XGpio: /amba/gpio@a0000000: dual channel registered, base is 290 Time taken to load DTBO is 244.000000 Milli Seconds DTBO loaded through zynqMP FPGA manager successfully root@xilinx-zcu102-2018_3:~# root@xilinx-zcu102-2018_3:/sys/class/gpio# dmesg |grep gpio [ 5.399518] input: gpio-keys as /devices/platform/gpio-keys/input/input0 [ 43.871149] XGpio: /amba/gpio@a0000000: registered, base is 298 [ 43.877420] XGpio: /amba/gpio@a0000000: dual channel registered, base is 290 root@xilinx-zcu102-2018_3:/sys/class/gpio# dmesg |grep gpio [ 5.399518] input: gpio-keys as /devices/platform/gpio-keys/input/input0 [ 43.871149] XGpio: /amba/gpio@a0000000: registered, base is 298 [ 43.877420] XGpio: /amba/gpio@a0000000: dual channel registered, base is 290 root@xilinx-zcu102-2018_3:/sys/class/gpio# echo 290 > export root@xilinx-zcu102-2018_3:/sys/class/gpio# cd gpio290 root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290# ls active_low device direction edge power subsystem uevent value root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290# echo 1 out > direction root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290# echo 0 > value root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290# echo 1 > value root@xilinx-zcu102-2018_3:/sys/class/gpio/gpio290# |
Usage: fpgautil -R
root@xilinx-zcu102-2018_3:~# fpgautil -R root@xilinx-zcu102-2018_3:~# |
Usage: fpgautil -b <bit/bin file absolute path>
root@xilinx-zcu102-2018_3:~# fpgautil -b /mnt/design_1_wrapper.bit Time taken to load BIN is 213.000000 Milli Seconds BIN FILE loaded through zynqMP FPGA manager successfully |
Usage: fpgautil -b <bit/bin file full path> -f Partial
root@xilinx-zcu102-2018_3:~# fpgautil -b pr1rm1.bit.bin -f Partial Flags : 0x1 Time taken to load BIN is 56.000000 Milli Seconds BIN FILE loaded through zynqMP FPGA manager successfully |
Usage: fpgautil -b <bit/bin file full path> -f <flags> -s AuthDDR
root@xilinx-zcu102-2018_3:~# fpgautil -b Authentication_system_wrapper.bin -f Full -s AuthDDR Flags : 0x0 Flags : 0x2 [ 467.885628] fpga_manager fpga0: writing Authentication_system_wrapper.bin to Xilinx ZynqMP FPGA Manager Time taken to load BIN FILE is 263.000000 Milli Seconds BIN FILE loaded through zynqMP FPGA manager successfully |
Usage: fpgautil -r
root@xilinx-zcu102-2018_3:~#fpgautil -r Readback contents are stored in the file readback.bin root@xilinx-zcu102-2018_3:~# vi readback.bin zynqMP FPGA Configuration register contents are CRC --> 0 FAR --> 7fc0000 FDRI --> 0 FDRO --> effffffe CMD --> d CTRL0 --> 101 MASK --> 0 STAT --> 16907ffc LOUT --> 0 COR0 --> 38003fe5 MFWR --> 0 CBC --> 0 IDCODE --> 1484a093 AXSS --> 0 COR1 --> 400000 WBSTR --> 0 TIMER --> 0 BOOTSTS --> 1 CTRL1 --> 0 |
Usage: fpgautil -r <filename> -t 1
root@xilinx-zcu102-2018_3:~# fpgautil -r readbackdata -t 1 root@xilinx-zcu102-2018_3:~# hexdump readbackdata.bin > data.txt |
To verify the readback data stream, compare it to the RBD golden readback file and masking readback bits with the MSD file. This approach is simple because there is a 1:1 correspondence between the start of the readback data stream and the start of the RBD and MSD files, making the task of aligning readback, mask, and expected data easier.
The RBD and MSD files contain an ASCII representation of the readback and mask data along with a file header that lists the file name, etc. This header information should be ignored or deleted. The ASCII 1s and 0s in the RBD and MSD files correspond to the binary readback data from the device.
For generating RBD and MSD refer UG908 (Vivado Design Suite User Guide)
By compiling the below utility(verify_readback.c file) user can verify the read back contents.
Source code: verify_readback
Note:
utility compares the readback with the RBD golden file, by masking readback bits with the MSD file. It is expected from user to remove/strip the header in the msd and rdb files and also remove the print statement in the readback contents file(readback.bin).
Example
$./verify_readback -msd design_1_wrapper.msd -rbd design_1_wrapper.rbd -data reabackdata_2.bin ============================================= Starting verification of readback data... >>>> Readback data is correct! <<<< ============================================= |
Linux FPGA Manager driver creates /sys/class/fpga_manager/<fpga> folder, which contains file attributes that provides control and state.
Set flags for Full Bitstream
Load the Bitstream
mkdir -p /lib/firmware
cp /media/design_1_wrapper.bit.bin /lib/firmware/
echo design_1_wrapper.bit.bin > /sys/class/fpga_manager/fpga0/firmware
Set flags for Partial Bitstream
Load the Bitstream Partial Bitstream
mkdir -p /lib/firmware
cp /media/partail_wrapper.bit.bin /lib/firmware/
echo partail_wrapper.bit.bin > /sys/class/fpga_manager/fpga0/firmware
Linux FPGA Manager driver creates /sys/kernel/debug/fpga/ folder, which contains image attributes that provides readback contents.
Set flags for readback type
echo 0 > /sys/module/zynqmp_fpga/parameters/readback_type
Perform Readback operation by reading the image debug attribute
cat /sys/kernel/debug/fpga/fpga0/image
Example:
root@xilinx-zcu102-2018_3:~# echo 0 > /sys/module/zynqmp_fpga/parameters/readback_type root@xilinx-zcu102-2018_3:~# cat /sys/kernel/debug/fpga/fpga0/image zynqMP FPGA Configuration register contents are CRC --> 0 FAR --> 7fc0000 FDRI --> 0 FDRO --> effffffe CMD --> d CTRL0 --> 101 MASK --> 0 STAT --> 16907ffc LOUT --> 0 COR0 --> 38003fe5 MFWR --> 0 CBC --> 0 IDCODE --> 1484a093 AXSS --> 0 COR1 --> 400000 WBSTR --> 0 TIMER --> 0 BOOTSTS --> 1 CTRL1 --> 0 |
Set flags for readback type
echo 1 > /sys/module/zynqmp_fpga/parameters/readback_type
Perform Readback operation by reading the image debug attribute
cat /sys/kernel/debug/fpga/fpga0/image
Example:
root@xilinx-zcu102-2018_3:~# echo 1 > /sys/module/zynqmp_fpga/parameters/readback_type root@xilinx-zcu102-2018_3:~# cat /sys/kernel/debug/fpga/fpga0/image > reabackdata.bin root@xilinx-zcu102-2018_3:~# hexdump readbackdata.bin > data.txt |
The Device Tree Overlay (DTO) is used to reprogram an FPGA while Linux is running. The DTO overlay will add the child node and the fragments from the .dtbo file to the base device tree,, The newly added device node/drivers will be probed after Bitstream programming
DTO contains:
Copy the Full Bitstream (.bin) and pl.dtbo files into firmware folder
mkdir /configfs
mount -t configfs configfs /configfs
cd /configfs/device-tree/overlays/
mkdir full
echo -n "pl.dtbo" > full/path
Devicetree overlay file contents example:
//Device Tree Example: Full Reconfiguration along with PS PL Configuration (Clock, resets and AFI) // HSI Generated overlay/pl.dtsi file. // Enable the axi-gpio interface /dts-v1/; /plugin/; / { fragment@0 { /* Bitstream Fragment */ target = <&fpga_full>; overlay0: __overlay__ { #address-cells = <2>; #size-cells = <2>; firmware-name = "design_1_wrapper.bit.bin"; resets = <&rst 116>; }; }; fragment@1 { /* PS-PL configuration Fragment */ target = <&amba>; overlay1: __overlay__ { afi0: afi0 { compatible = "xlnx,afi-fpga"; config-afi = < 0 0>, <1 0>, <2 0>, <3 0>, <4 0>, <5 0>, <6 0>, <7 0>, <8 0>, <9 0>, <10 0>, <11 0>, <12 0>, <13 0>, <14 0>, <14 0>, <15 0x000>; }; clocking0: clocking0 { #clock-cells = <0>; assigned-clock-rates = <99990005>; assigned-clocks = <&clk 71>; clock-output-names = "fabric_clk"; clocks = <&clk 71>; compatible = "xlnx,fclk"; }; }; }; fragment@2 { /* PL Drivers Fragment */ target = <&amba>; overlay2: __overlay__ { axi_gpio_0: gpio@a0000000 { #gpio-cells = <3>; #interrupt-cells = <2>; clock-names = "s_axi_aclk"; clocks = <&clk 71>; compatible = "xlnx,axi-gpio-2.0", "xlnx,xps-gpio-1.00.a"; gpio-controller ; interrupt-controller ; interrupt-names = "ip2intc_irpt"; interrupt-parent = <&gic>; interrupts = <0 89 4>; reg = <0x0 0xa0000000 0x0 0x1000>; xlnx,all-inputs = <0x1>; xlnx,all-inputs-2 = <0x0>; xlnx,all-outputs = <0x0>; xlnx,all-outputs-2 = <0x1>; xlnx,dout-default = <0x00000000>; xlnx,dout-default-2 = <0xAAAAAAAA>; xlnx,gpio-width = <0x8>; xlnx,gpio2-width = <0x8>; xlnx,interrupt-present = <0x1>; xlnx,is-dual = <0x1>; xlnx,tri-default = <0xFFFFFFFF>; xlnx,tri-default-2 = <0xFFFFFFFF>; }; psu_ctrl_ipi: PERIPHERAL@ff380000 { /* This is a place holder node for a custom IP, user may need to update the entries */ compatible = "xlnx,PERIPHERAL-1.0"; reg = <0x0 0xff380000 0x0 0x80000>; }; psu_message_buffers: PERIPHERAL@ff990000 { /* This is a place holder node for a custom IP, user may need to update the entries */ compatible = "xlnx,PERIPHERAL-1.0"; reg = <0x0 0xff990000 0x0 0x10000>; }; }; }; }; |
Copy the Partial Bitstream (.bin) and rm.dtbo files into lib/firmware folder
mkdir /configfs
mount -t configfs configfs /configfs
cd /configfs/device-tree/overlays/
mkdir partial
echo -n "pl.dtbo" > partial/path
To remove a device tree overlay:
Refer this link: libdfx - Linux User Space Solution for FPGA Programming
The current driver availble in the xilinx linux git is in sync with the open source 4.19 kernel driver except for the following
2021.1
FPGA Manager framework upgraded to 5.10v
2020.2
Summary:
Commits:
2020.1
2019.2
Summary:
Commits:
2019.1
Summary:
Commits:
2018.3
Summary:
Commits:
2018.2
Summary:
Commits:
2018.1
Summary:
Commits:
2017.4
2017.3
Summary:
Commits:
2017.2
2017.1
Summary:
Commits:
2016.4
2016.3
Summary:
Commits:
https://github.com/Xilinx/linux-xlnx/blob/master/drivers/fpga/zynqmp-fpga.c
https://github.com/Xilinx/linux-xlnx/blob/master/Documentation/fpga/fpga-mgr.txt
https://github.com/Xilinx/embeddedsw/tree/master/lib/sw_services/xilfpga/doc