This tutorial shows how to develop a Partial Reconfiguration (PR) design for the Zynq-7000 SoC using the Xilinx Vivado Design Suite, Vivado HLS, Software Development Kit (XSDK), and PetaLinux design tools. It complements application note XAPP1231 which focuses on conceptual aspects of the PR flow and design considerations specific to the Zynq architecture.
The previous versions of this reference design is available at the links below:
ISE 14.4 XAPP1159 - Partial Reconfiguration of a Hardware Accelerator on Zynq-7000 SoC Devices
1.1 Design Overview
The PR reference design is built on top of the ZC702 Base Targeted Reference Design (TRD) , an embedded video processing application that demonstrates how it is best to separate control and data processing functions. In this example a compute-intensive video filtering algorithm is moved from the Processing System (PS) onto a hardware accelerator in Programmable Logic (PL). The image filter IP core demonstrated in the ZC702 Base TRD is a Sobel filter configured with edge detection coefficients which has been generated using Vivado HLS. For this reference design, three image filter IP cores are generated using Vivado HLS: Posterize, Sobel, FAST. The provided reference design demonstrates how to use software-controlled Partial Reconfiguration through the Processor Configuration Access Port (PCAP) to dynamically reconfigure part of the PL with the desired image filter IP core and observe the modified video output on a monitor.
Note: Some tools are optional and the corresponding design flow tutorials can be skipped.
- ZC702 Evaluation Kit
- Monitor with HDMI or DVI port that supports 1080p60 video resolution
- Avnet FMC-IMAGEON module and external video source that provides 1080p60 video input over HDMI (optional)
- USB mouse and keyboard (optional)
- Xilinx Partial Reconfiguration is a product inside the Vivado Design Suite that requires a license
- Xilinx IP evaluation licenses may be provided with the Vivado Design Suite or can be ordered online
- Xylon logiCVC-ML is provided as evaluation IP core that does not require a license. The evaluation IP core has a 1 hour timeout built-in such that the display output freezes after the timer expires. The pre-built bitstreams and boot images are built from a full logiCVC-ML IP core.
1.3 Design Files
Download and unzip the reference design archive file xapp1231-partial-reconfig-hw-accelerator-vivado.zip to a local directory.
The following files are inside the xapp1231-partial-reconfig-hw-accelerator-vivado top-level directory:
- doc -- Readme and License files
- hardware -- Hardware sources
- vivado_hls -- Vivado HLS projects for image filters
- fast_corners -- FAST
- sobel -- Sobel
- simple_posterize -- Posterize
- vivado_pr -- Vivado Partial Reconfiguration project-less design scripts
- vivado_trd -- Vivado IP Integrator project for Zynq Base TRD
- ready_to_test -- Pre-built SD card image
- software -- Software sources
- boot -- Zynq boot image sources
- petalinux -- PetaLinux board support package (BSP)
- pre-built -- Pre-built Linux open-source libraries, binaries and headers
- xsdk -- XSDK projects
- filter_lib -- HLS image filter library
- perfmon_lib -- AXI performance monitor library
- video_cmd -- Command line application
- video_lib -- Xilinx Video library
- video_qt -- GUI application based on Qt framework
The xapp1231-partial-reconfig-hw-accelerator-vivado directory is the starting point for each of the below tutorials.
1.4 Known Issues
See Known Issues section on Zynq Base TRD wiki.
2 Running the Reference Design
This section describes how to set up and run the reference design applications on the ZC702 evaluation board. The Qt application requires a video resolution of 1080p60 whereas the command line application supports arbitrary resolutions. For either application, the video input and video output resolutions need to match.
2.1 Setting up the ZC702 Evaluation Board
Refer to the ZC702 Evaluation Kit Quick Start Guide for kit contents and default jumper and switch settings. Reference design specific settings are described below.
Connect a power supply to connector J60 on the ZC702 board. Connect a 1080p60-capable monitor to the HDMI port on the ZC702 board. Connect a USB mouse and keyboard (optional for Qt application) to the Micro USB port J1 on the ZC702 board. Connect the FMC-IMAGEON daughter card to the FMC2 slot on the ZC702 board and an HDMI cable to the HDMI IN port on the FMC card (optional - only required to enable external video input).
Set the boot mode switch SW16 to SD boot mode.
Connect a Laptop or PC to the Mini USB-UART port J17 on the ZC702 board (optional for Qt application). Use the following serial port settings:
2.2 SD Card Image
The following files need to be copied to a FAT formatted SD card. A pre-built SD card image can be found in the ready_to_test directory:
- BOOT.BIN -- boot image containing FSBL, full Sobel bitstream, and u-boot
- autostart.sh -- if present, this script is run at the end of the boot process; it starts the Qt application
- bin -- binary executables
- run_video.sh -- wrapper script for calling command line or Qt application
- video_cmd -- command line application
- video_qt -- GUI application based on Qt framework
- devicetree.dtb -- Linux device tree binary with FMC card disabled (default)
- devicetree_fmc.dtb -- Linux device tree binary with FMC card enabled - rename to devicetree.dtb to enable
- partial -- partial bitstreams
- fast_corners.bin -- FAST filter
- simple_posterize -- Posterize filter
- sobel.bin -- Sobel filter
- uImage -- Linux kernel
- uramdisk.image.gz -- Linux root file system
2.3 Software Applications
Insert an SD card with the files listed above into the SD card slot on the ZC702 board and power on the board. The Qt Linux application will start automatically after the system is booted.
To start the application, run:
The -p switch enables Partial Reconfiguration.
Use the mouse to navigate the GUI. The following is a list of control elements and monitors as they appear in the GUI from left to right and top to bottom:
|GUI Transparency||Slider to set alpha blending value between 0 and 50|
|Video Controls||Radio buttons to turn on/off video or select auto mode|
|Video Source||Combo box to select TPG or HDMI input|
|Test Pattern||Combo box to select displayed test pattern|
|Filter Type||Combo box to select between fast_corners, simple_posterize, or sobel filter|
|Filter Mode||Radio buttons to turn off or select software or hardware accelerated filter|
|Sobel Controls||Only available when sobel filter and SW or HW filter mode are selected; check box to invert filter colors; slider to set edge sensitivity threshold value between 100 and 255|
|Performance||Textual visualization of CPU0/CPU1 utilization and HP0/HP2 memory throughput|
|CPU Utilization||Graphical visualization of CPU0/CPU1 utilization|
|Memory Throughput HP Ports||Graphical visualization HP0/HP2 memory throughput|
To display more options, run:
Command Line Application
Connect a terminal emulator to the serial port assigned to the ZC702 UART. You can use XSDK's built-in terminal or a standalone application like TeraTerm for example.
To start the application, run:
The -p switch enables Partial Reconfiguration. If the -r switch is omitted, the default resolution is set to 1920x1080.
Use the terminal window to control the command line application. The following is a list of control elements:
|Video Source||Select TPG or HDMI input - the current selection is marked by asterisk|
|Filter Type||Select fast_corners, simple_posterize or sobel filter - the current selection is marked by asterisk|
|Filter Mode||Toggle filter modes off, software, or hardware - the current selection is printed in brackets|
To display more options, run:
3. Vivado HLS
Vivado HLS provides a tool and methodology for migrating algorithms coded in C, C++ or System-C from the Zynq PS onto the PL by generating RTL code. Three different image filters are used in this design and the source files are located in the respective directories listed under hardware/vivado_hls:
In this design a scripted HLS flow is used. For example, to generate the FAST filter, run:
To open the generated HLS project, run:
Below screenshot shows the HLS project inside the Vivado HLS IDE and the generated synthesis report file:
The following table shows the steps run by the scripts and where the corresponding output products are located under prj/solution1/:
Expand the csim/build folder in the Vivado HLS explorer view and open the files test_1080p.bmp and result_1080p.bmp to view the original and processed images after running C Simulation:
Repeat the above steps for all three HLS image filters.
The following files are generated in this tutorial will be used in step 5:
|FAST filter IP core||fast_corners/prj/solution1/impl/ip/xilinx_com_hls_image_filter_1_0.zip|
|Posterize filter IP core||simple_posterize/prj/solution1/impl/ip/xilinx_com_hls_image_filter_1_0.zip|
|Sobel filter IP core||sobel/prj/solution1/impl/ip/xilinx_com_hls_image_filter_1_0.zip|
4. Vivado - Base TRD
In this step we will generate a synthesis design checkpoint for the static logic i.e. the portion of the design that can not be reconfigured. We are using the ZC702 Base TRD Vivado IP Integrator project as a starting point. The Base TRD includes the Sobel HLS image filter IP core generated in section 3 above. We will carve out the Sobel filter declare a black box module instead which will be the Reconfigurable Module (RM) in this design. We will also floorplan the design by assigning a Reconfigurable Partition (RP) to the RM. The RP is the region that can be reconfigured using Partial Reconfiguration (PR).
To generate and open the Vivado IP Integrator project, run:
In the Vivado IDE, right-click on the system_top block diagram source file in the Sources view and select Generate Output Products...:
After the HDL files have been generated, expand the hierarchy tree all the way down to system_top/processing/system_top_image_filter_1_0 as shown in the screenshot below. Double click on the system_top_image_filter_1_0 source file to open it. Since this file is generated by IP Integrator, it is marked as read-only and cannot be modified within the Vivado editor. Copy the full path of the file (see red box at the top) and open the file in an editor outside of Vivado. Insert the BLACK_BOX property on line 57 as shown in the screenshot. Save the file and switch back to the Vivado view and click reload at the top of the editor view to update the changes.
The black box attribute assigned to the Sobel filter wrapper module ensures that the module is not synthesized in this step. To run synthesis, mark the synth_1 run in the Design Runs view at the bottom and click the green Launch symbol as shown in below screenshot:
Once synthesis is finished, Vivado will prompt you for the next step. Select Open Synthesized Design. The next prompt is a critical warning, stating that no synthesized netlist could be found for the black box module system_top_image_filter_1_0 defined previously. This is expected and intentional. Confirm with OK.
Right-click on the system_top_image_filter_1_0 black box module in the Netlist view. Note the black icon compared to the other netlists. Select Floorplanning > Draw Pblock
Assign a Reconfigurable Partition to the black box module by drawing a Pblock in the desired area in the Device View. Zoom into the X1Y0 clock region first, then left-click and drag the mouse to draw a rectangle as shown in the screenshot below. Make sure the resources preview box shows the same numbers.
In the next prompt, make sure that all available resources and the Assign selected cell box are checked. Confirm with OK.
Next we set the RESET_AFTER_RECONFIG on the Pblock. This will ensure that all the resources in the Reconfigurable Partition are automatically reset every time a new module is loaded into this region. Run the following command in the Tcl Console at the bottom:
Finally, we set the HD.RECONFIGURABLE property on the image filter black box module.Run the following command in the Tcl Console:
At this point, the tools will check for a valid license for Partial Reconfiguration (see screenshot below).
Now save the constraints by left clicking the disk symbol at the top of the screen. Note that the icon for the image filter module has changed again after assigning a Pblock to the module.
Save the constraints to a new file to the location constrs/system_top_pblock.xdc. Confirm with OK.
In the Sources view, expand the Constraints hierarchy and double-click on system_top_pblock.xdc to open and view the newly generated constraints file. Constraints are created for the Pblock and the RESET_AFTER_RECONFIG and HD.RECONFIGURABLE properties.
The following files are generated in this tutorial and will be used in step 5:
|Synthesis design checkpoint for static logic||project/zynq_base_trd_2014.4.runs/synth_1/system_top_wrapper.dcp|
|IP Integrator generated HDL wrapper file for HLS image filter IP core||project/zynq_base_trd_2014.4.srcs/sources_1/bd/system_top/ip/system_top_image_filter_1_0/synth/system_top_image_filter_1_0.v|
|XDC constraints file for Partial Reconfiguration||constrs/system_top_pblock.xdc|
5. Vivado - Partial Reconfiguration
In this step we will run a project-less, script based Vivado flow for Partial Reconfiguration. We define Reconfigurable Modules and Configurations in a tcl file. We out-of-context (OOC) synthesize each Reconfigurable Modules. We link the RMs to the already synthesized static logic and implement each Configuration. Finally, we generate full bitstreams for each Configuration and partial bitstreams for each Reconfigurable Module.
Input Source Files
We are using the following source files and output products from steps 4 and 5 as inputs in this step. The following table lists the origin project and the relative location to the vivado_pr directory:
|Vivado HLS||unzipped FAST IP core||srcs/ip/fast_corners|
|Vivado HLS||unzipped Posterize IP core||srcs/ip/simple_posterize|
|Vivado HLS||unzipped Sobel IP core||srcs/ip/sobel|
|Vivado Base TRD||PR constraints||constrs/system_top_pblock.xdc|
|Vivado Base TRD||Pin constraints||constrs/system_top_pin.xdc|
|Vivado Base TRD||Timing constraints||constrs/system_top_timing.xdc|
|Vivado Base TRD||Synthesized design checkpoint||srcs/dcp/system_top_wrapper.dcp|
|Vivado Base TRD||HLS image filter wrapper||srcs/hdl/system_top_image_filter_1_0.v|
RM Synthesis File
For each of the HLS image filter IP cores, we need to create a .prj file listing the HDL sources and the image filter wrapper source file required for OOC synthesis. The files are located at srcs/prj. For example, this is what the simple_posterize.prj file looks like:
PR Design File
The Vivado PR scripts are located at scripts/tcl and are read only. The file scripts/design.tcl is the only file that needs to be modified to set up the sources and parameters for the PR run. We are looking at excerpts of the design.tcl file in the following.
This code snippet sources the Vivado PR tcl scripts and makes the corresponding routines available.
This code snippet sets up the target part, package and speed grade for implementation.
This code snippet sets up some basic variables to control the PR flow and where the input and output directories are located.
This code snippet defines the static logic of the design. Synthesis will not be run on this module since we already have a synthesized design checkpoint available from step 4.
This code snippet defines the three HLS image filters as Reconfigurable Modules of this design. The variant name, an XDC constraints file, and a list of HDL files (.prj file) is provided per RM. The RM instance name in the design hierarchy (black box module) is defined at the bottom. OOC synthesis is run on each of the RMs.
This code snippet defines three Configurations, one for each RM. For each Configuration, the XDC constraints files, the static module, and the RM are defined. Note that the first configuration has the static module set to implement whereas the second and third Configurations have it set to import under the partitions attribute. The static logic only needs to be implemented i.e. placed and routed once. After that it is locked down, saved, and re-used for the remaining partitions which is indicated by the import flag. The PR verify design rule check (DRC) is run after implementation to check whether the implementation results are in fact compatible. Full and partial bitstreams are generated for each Configuration and RM respectively. The partial bitstreams are converted to the binary PCAP format since we are using the Processor Configuration Access Port (PCAP) to load the bitstreams.
Now that all the parameters are set up, the main PR tcl script is sourced which in turn calls all the required sub-routines.
PR Log File
To start the Vivado PR flow, run:
The script will run autonomously and no further user intervention is required. The following are excerpts of the run log generated by the PR scripts.
This snippet shows a list of the modules to be synthesized and their selected synthesis options. Note that this list only includes the HLS image filters i.e. the Reconfigurable Modules and hence Top Level is set to 0. The top level module (system_top_wrapper) i.e. the static logic is listed as module not being synthesized since we're just reading in an already synthesized design checkpoint.
This snippet shows a list of configurations to be implemented. The module associated with each Configuration is listed, as well as additional options in the flow like PR verification or bitstream generation. It is also listed if the static logic is implemented as part of this Configuration or if it is imported from a previous Configuration run.
This snippet shows a summary of additional post-implementation steps like PR verification, bitstream generation, and bitstream formatting for a specific interface like PCAP.
The final output products of this step are full bitstreams for each Configuration and partial bitstreams for each Reconfigurable Module. We use the full Sobel bitstreams as default for the boot image i.e. the PL gets programmed with this bitstream during the boot process. The boot image is generated in step 7. The partial bitstreams are formatted correctly and can be deployed to the SD card image from where they are loaded by the application as discussed in step 6.
PR Device View
After the Vivado PR run has finished, we can inspect report files or view saved design checkpoints for the various implementation stages.
To open a saved implemented design checkpoint, for example for the static logic, run:
The following screenshot is a composition of the device view for the static logic (left) and the three RMs (right): simple_posterize (top), sobel (center), fast_corners (bottom). The static resources are highlighted in orange, the Pblock is a purple box in the bottom-right corner. No resources are placed in this partition since it is reserved for the RMs. The gray boxes inside the Pblock are the ports of the Reconfigurable Modules interfacing with the static logic. The RM resources are highlighted in blue in their respective views. Note the difference in resource utilization between the filter variants.
6. Xilinx SDK
Two Linux applications are available in this reference design:
- a command line based application controlled from a UART terminal
- a GUI based application controlled by mouse from an HDMI monitor
Both applications are extensions of the corresponding Base TRD applications. For a tutorial on how to build the two Linux applications, please refer to the Zynq Base TRD wiki - XSDK. The page also contains information on how to debug and deploy an application using XSDK as well as how to measure CPU and HP/ACP port performance metrics using the integrated System Performance Monitor (SPM).
To enable the Partial Reconfiguration feature in either application, simply add the -p or --partial-reconfig switch when executing the application, for example:
The following code snippets show the two added functions for Partial Reconfiguration.
The first function identifies the partial bitstream by its name and loads the partial bitstreams from a pre-defined location on the SD card into a buffer variable in memory. This is to speed up the bitstream transfer when the PL is reconfigured in the subsequent step.
This function uses the xdevcfg driver to load the partial bitstream into the PL through the PCAP interface. The driver initiates a DMA transaction from memory to the configuration logic. Note that the size of the bitstream, defined by the FILTER_PR_BIN_SIZE macro, is required for this operation. Make sure the is_partial_bitstream flag is set to indicate that the bitstream is a partial bitstream.
PetaLinux is an easy-to-use tool to create a complete boot-able Linux image including:
- boot image
- first stage boot loader (FSBL)
- bitstream (full Sobel)
- u-boot (second stage boot loader)
- device tree
- root file system
For a tutorial on how to build the PetaLinux image, please refer to the Zynq Base TRD wiki - PetaLinux. The page also contains information on PetaLinux installation.
- XAPP1231, Partial Reconfiguration of a Hardware Accelerator with Vivado Design Suite
- UG925, Zynq-7000 SoC ZC702 Base Targeted Reference Design (Vivado Design Suite 2014.4)
- ZC702 Evaluation Kit Quick Start Guide
- Feb 12, 2015