Zynq UltraScale MPSoC Software Acceleration TRD 2018.1

Zynq UltraScale MPSoC Software Acceleration TRD 2018.1

Table of Contents

2 Introduction
This wiki page contains information on how to build various components of the Zynq UltraScale+ MPSoC Software Acceleration Targeted Reference Design (TRD), version 2018.1. The page also has information on how to set up the hardware and software platforms and run the design using the ZCU102 evaluation kit (board revision 1.0).

1 Revision History

This wiki page complements the 2018.1 version of the Software Acceleration TRD. For other versions, refer to the Zynq UltraScale+ MPSoC Software Acceleration TRD overview page.

Change Log:
  • Updated all projects, IPs, and tools versions to 2018.1
  • Use dsa instead of hdf, in Petalinux BSP and r5FFT firmware
  • Use new DRM driver for DisplayPort
  • Various fixes and improvements

2 Introduction

3 About the TRD

The Software Acceleration TRD is an embedded signal processing application designed to showcase various features and capabilities of the Zynq UltraScale+ MPSoC ZU9EG device for the embedded domain. The TRD consists of two elements: The Zynq UltraScale+ MPSoC Processing System (PS) and a signal processing application implemented in Programmable Logic (PL). The MPSoC allows you to implement a signal processing algorithm that performs Fast Fourier Transform (FFT) on samples (coming from Test Pattern Generator (TPG) in Application Processing Unit (APU) or System Monitoring (SYSMON) through an external channel either as a software program running on the Zynq UltraScale+ MPSoC based PS or as a hardware accelerator inside the PL. The design has three accelerator cores generated using SDx for computing 4096, 16384, and 65536 point FFTs. The data transfers of the SDx accelerators is controlled by the APU. There is one accelerator (FFT IP from the Vivado IP catalog) for 4096 point FFT controlled by the Real-Time Processing Unit (RPU). The TRD demonstrates how to seamlessly switch between a software or a hardware implementation and to evaluate the cost and benefit of each implementation. The TRD also demonstrates the value of offloading computation-intensive tasks onto PL, thereby freeing the CPU resources to be available for user-specific applications.
For detailed information on the complete feature set, or hardware and software architecture of the design, please refer to the TRD user guide UG1211.

4 Download the TRD

This TRD has been tested on Rev 1.0 of ZCU102 board with production silicon. The current design doesn't support ES1/ES2 silicon.
The design file, rdf0376-zcu102-swaccel-trd-2018-1.zip, can be downloaded from here.

5 TRD Directory Structure and Package Contents

The Software Acceleration TRD package is released with the source code, SDK projects, and an SD card image that enables you to run the demonstration and software application. It also includes the binaries necessary to configure and boot the ZCU102 board. Prior to running the steps mentioned in this wiki page, download the TRD package and extract its contents to a directory referred to as ‘TRD_HOME' which is the home directory.

The table below describes the content of each directory in detail.

apuContains the software source files
petalinux_bspContains the PetaLinux project's configuration
swaccel_appContains GUI application project
zcu102_swaccel_trdSDx platform folder containing the hardware platform, pfm files and FFT accelerator C sources.

swaccel_r5_firmwareContains SDK project for building RPU firmware

sdcardContains ready to test binaries
BOOT.BINBIN file containing FSBL, PL bitstream, U-boot, ARM trusted firmware and PMU firmware
image.ubKernel image
libfft.soFFT accelerator shared-object
r5FFT.elfR5 FFT computation firmware
README.txtContains design version history, steps to implement the design, and Vivado and PetaLinux versions to be used to build the design.
swaccel_qtQt GUI application.

THIRD_PARTY_NOTICES.zipContains the Copyright text for third-party libraries
IMPORTANT_NOTICE_CONCERNING_THIRD_PARTY-CONTENT.txtContains information about the third party licences

  1. ZCU102 Evaluation Kit (Rev 1.0 board with production silicon)
  2. A Linux development PC with following tools installed:
    • Xilinx Vivado Design Suite 2018.1
    • Xilinx SDx 2018.1
    • Xilinx SDK 2018.1
    • PetaLinux 2018.1
    • Distributed version control system Git installed. For information, refer to the Xilinx Git wiki.
    • GNU make utility version 3.81 or higher.

6 Running the Demo

This section provides step by step instructions on bringing up the ZCU102 board for demonstration of the TRD and running different options from the graphical user interface (GUI). The binaries required to run the design are in the $TRD_HOME/sdcard folder. It also includes the binaries necessary to configure and boot the ZCU102 board.

Before running the demo:
  1. Format the SD-MMC card as FAT32 using a SD-MMC card reader.
  2. Copy the contents of the $TRD_HOME/sdcard onto the primary partition of the SD-MMC card.

7 Hardware Setup Requirements

7.1 Requirements for the TRD demo setup:

  • The ZCU102 Evaluation Kit (board revision 1.0)
  • AC power adapter (12 VDC)
  • Optional: A USB Type-A to USB Micro-B cable (for UART communication) and a Tera Term Pro (or similar) UART terminal program.
  • USB-UART drivers from Silicon Labs
  • USB Micro-B to female Adaptor with USB hub is needed for connecting a mouse.
  • USB mouse
  • 4K monitor with Display Port support
  • Certified Display Port cable (version 1.2); TRD tested with 6 feet long E342987, Cable matters
  • Optional, required only for testing with external audio input:
    • XA3 SYSMON Headphone Adapter card from Faster Technology
    • An audio source like MP3 player
    • An aux cable with 3.5mm male jack on both ends.
  • A SD-MMC flash card containing TRD binaries formatted with FAT32. The SD-MMC should have the required binaries in its primary partition. Copy the binaries from sdcard folder of the TRD zip file. The required binaries include :
    • BOOT.BIN
    • image.ub
    • libfft.so
    • r5FFT.elf
    • swaccel_qt

Note: TRD supports Ultra HD (4K) and Full HD (1080p) resolutions. The binaries provided in the sdcard folder have been tested with ViewSonic (4K), ASUS (4K), Acer (4K) and Dell-P2414Hb (1080p) display monitors. However, the binaries should work well with any Display Port certified monitors supporting 4K/1080p resolution in its EDID database. Please make sure to use a DP certified 1.2 version of the cable for connecting the ZCU102 board to the monitor.

8 Board Setup

Connect various cables to the ZCU102 board as shown in the following steps.

1. Connect a 4K monitor to the DP port on ZCU102 using DP 1.2 cable.
2. Connect an USB mouse to the Micro-B USB connector (Jumper J96 on ZCU102 board).
3. Optional: Connect an USB Micro-B cable into the micro USB port (J83) labeled USB UART on the ZCU102 board and the USB Type-A cable end into an open USB port on the host PC for UART communication.
4. Connect the power supply to the ZCU102 board. Do not switch the power ON.
5. Optional: Plug the XA3 Adapter card into the Sysmon Header on ZCU102 board (J3). Connect Jumpers J5 and J4 on XA3 card as shown in below figure.

6. Optional: Connect the 3.5mm auxiliary cable to XA3 card and audio source. One end connects to audio source and the other end connects to 3.5mm female connector on XA3 card.
7. Insert a SD-MMC memory card, which contains the TRD binaries, into the SD receptacle on the ZCU102 board.
8. Make sure the DIP switches (SW6) are set as shown in figure below, which allows the ZCU102 board to boot from the SD-MMC card.

9. Optional: Open a serial communication terminal software like TeraTerm, and set up a new serial communication as shown in below figure.

10. Click on New Connection and select Interface 0 and click OK (as shown in below figure).

11. Click on Setup -> Serial Port and make sure to setup as shown in below figure.

12. The serial terminal displays:

13. After Linux boot is complete, you see the PetaLinux login prompt, as shown in below figure.

9 Run QT GUI application

A Linux application with QT-based GUI is provided with the package included on the SD-MMC memory card. This application provides options to the user to exercise different modes of the demonstration:
  • User can select Test Pattern Generator (TPG) samples or External audio source (requires the XA3 adapter card, aux cable and audio source for testing).
  • User can select to perform FFT computation on APU (run as software code on the PS) or in PL (run in the FPGA fabric as a hardware IP core).
  • User can also apply various windowing techniques on input samples before performing FFT.

9.1 Powering on the QT-based GUI application demo

  • Make sure the monitor is set for DP Ultra HD (4K) resolution.
  • Turn on power switch (J52)

Note: The Linux image and Qt based GUI application will be loaded from the SD-MMC memory card.
  • The Linux image will load and the frame buffer console is displayed on the monitor.
  • The Qt based GUI will load
  • When the GUI starts up, the demonstration starts with FFT being computed by software running in APU on samples coming from TPG (in Software).

9.2 Running the Qt-based GUI application demo

Exercise different options by pressing the buttons available in the GUI to evaluate the different use cases mentioned below.

9.2.1 Test Start/Pause

Demonstration can be paused at any instant by clicking on Pause button, as shown in figure below.

9.2.2 Input Source

There are two sources of data samples.
Use caseInput source
1Test Pattern Generator (TPG in software). This is the default option.
2External audio input(through XA3 SYSMON Headphone Adapter card)
Note : To test the external audio (assuming that setup is made as per procedure mentioned above), play an audio from the MP3 player/Phone. The peak voltage of the audio source depends on the manufacturer. The voltage levels of the samples depend on the volume. If the output voltage of the audio signal goes beyond 1V, the waveform will be clipped. Adjust the volume on the audio source so that the voltage of the samples lies within 1V peak-to-peak.

9.2.3 FFT Computation Engine

For the two input sources mentioned in above table, user can select one of the following compute engines for FFT computation.
FFT Compute EngineDescription
APU (default)FFT computation is done by software running on APU
NEONFFT computation is done by software running on APU. Neon intrinsic APIs are used for FFT computation to make
sure that the instructions are executed on NEON.
APU controlled PL AcceleratorFFT computation is done by the FFT core in Programmable Logic(PL)
RPU as Co-processorFFT computation is done by software running on RPU. APU is involved in moving samples from TPG in PL to PS DDR.
Samples from PS DDR are copied to OCM by APU software and that information is passed to RPU through OpenAMP channel.
RPU controlled PL AcceleratorFFT computation is done by PL FFT IP. RPU controls the AXI DMA transfers to/from PL FFT core from/to PS DDR.
APU is involved in moving samples from TPG in PL to PS DDR. Samples from PS DDR are copied to OCM by APU
software and that information is passed to RPU through OpenAMP channel. PL FFT core fetches samples from OCM
and computes FFT on the samples and writes samples back to OCM.
AllRuns FFT on all engines one at a time. This mode is useful for comparing computation times for various engines.

9.2.4 FFT Length

FFT length determines the number of samples on which FFT computation is performed. User can run the following FFT sizes.
FFT Size
4096 (default)

9.2.5 FFT Window

You can apply one of the window function on the input samples before FFT computation.
Window function
None (Default, No windowing)
Blackman Harris

9.2.6 Frequency Zoom

User can select the following Frequency Zoom options
FFT Zoom optionDescription
ZOOMSelecting this option fixes the units on frequency axis in the Frequency domain plot to 512.
This enables users to closely observe the values on frequency axis. This is 5X zoom.
NONE (default)This is the default option. None is No Zoom. Selecting this option will plot all points on frequency axis (Number of points equal to half of the FFT size)

9.2.7 FFT Scale

User can select the different scales on Voltage/Amplitude axis. This option is important when using external audio source as input. The voltage of the samples is dependent on the volume of the audio signal. Depending on the amplitude of the audio samples, the scale can be selected. Available options are:
FFT Scale
1V (Default)

9.2.8 Sample Rate

The sampling rate of the SYSMON in PL can be changed on run time. Supported sampling rates are:
Sampling Rate
200 kSPS (default)
100 kSPS
50 kSPS

9.2.9 Time and Frequency domain plots

The time domain plot plots the samples corresponding to data generated by either TPG or by external audio source. The number of points in the plot depends on the FFT size.
The frequency domain plot plots the power spectral density (not in logarithm scale). It is a function of voltage vs frequency bins. The value “Fp” on the extreme right corner of frequency domain plot depicts the frequency bin with highest energy. The number of frequency bins plotted is half of FFT size (half because of symmetry for real valued samples) when “NONE” is selected in Frequency Zoom control and 512 by default (ZOOM enabled).

9.2.10 FFT Computation time plot

The time taken for FFT computation by each engine is plotted on the “FFT computation plot”. The average computation times for 4096 point FFT are captured for reference in below table:
Computation Engine~Average computation time (us)
APU with Neon as Co-processor361
APU controlled PL92
RPU controlled PL157*
  • RPU is running at 500 MHz and APU is running at 1.1G. Also, the OpenAMP communication latency is included which is approximately 100 μs.

9.2.11 CPU Utilization plot
The APU cluster (A53 cores) utilization is plotted in “CPU Utilization Plot”.

9.2.12 PS-PL Interface Performance plot

The bandwidth utilization of Full Power domain and Low power domain high performance ports is plotted by “PS-PL performance plot”. The write and read throughputs are plotted.

9.2.13 PL Die temperature

The PL Die temperature is read from the SYSMON and displayed on the GUI.

9.2.14 Block Diagram view

The top-level block diagram and the blocks involved in data path for each of the modes in Input source and FFT computation engines is displayed in the bottom right corner of the GUI.

10 Building the Software components

The following tutorials assume that the $TRD_HOME environment variable has been set as below:
For some modules, the $PETALINUX environment variables needs to be set as well. This is done automatically when you source the PetaLinux settings.sh script (see PetaLinux installation guide).

10.1 Building RPU firmware using XSDK

  • Source the SDK tool-chain and execute the following commands:
A welcome screen is displayed as shown:

  • Click 'Import Project' from the welcome screen, browse to the current working directory and make sure the r5FFT, r5FFT_bsp and zcu102_swaccel_trd_hw_platform_0 projects are selected. Click Finish.

It builds automatically and fails (failure can be ignored as it will build successfully in the next step).

  • From menu-bar, go to Xilinx - > Repositories, as shown in the below figure.

  • Click on New and specify the path to the repository directory in present working directory. Click Apply and then OK.

  • Right click on r5FFT_bsp, then click on Board Support Package Settings. Board Support Package Settings window is displayed.

  • Navigate to Overview > drivers > psu_cortexr5_0. Then append 'value' field for “extra_compiler_flags" with -mfloat-abi=hard.

  • Click OK. It will regenerate BSP sources and build the firmware.
  • Create “images” directory and copy the generated image.

10.2 PetaLinux BSP

This tutorial shows how to build the Linux image using the PetaLinux build tool.
Copy generated image.ub to $TRD/images.

10.3 PetaLinux SDK

This tutorial shows how to use the PetaLinux build tool to generate and install an SDK for application development in XSDK or SDx.
  • Generate and install an SDK for Linux application development.

10.4 Build BitStream and FFT Shared Object using SDSoC

  • Set the build environment variables. This requires that you have previously completed the PetaLinux SDK installation step.
    Note 1: Make sure you set the env variables in the same shell that is used to launch SDx. Also make sure the env variables are set before starting SDx, otherwise close and re-start SDx.
    Note 2: The below command might ask you execute unset LD_LIBRARY_PATH and then re-execute the command. Go ahead and follow those steps
  • Source the SDx tool-chain and execute the following commands:
A welcome screen is displayed as shown in the below figure.

  • Create a new SDx Project (File > New > SDx Project…). A New SDx Project window is displayed.

  • Choose Application Project as Project Type and Click Next.

  • Enter ' fft ' as project name and click Next.

  • Click 'Add Custom Platform', browse to the $TRD_HOME/apu/zcu102_swaccel_trd directory and click OK. Select the newly added zcu102_swaccel_trd(custom) platform from the list and click 'Next'.

  • Choose the Shared Library as output type and click Next.

  • Select the 'FFT Library' template and click 'Finish'.

  • Change the 'Active build configuration' to Release in the SDx Project Settings window.

  • Right-click the fft project and select 'Build Project'.
  • Copy the content of the generated sd_card folder to the images.

10.5 QT-application:

This tutorial shows how to build Qt application.
  • Set the build environment variables. This requires that you have previously completed the PetaLinux SDK installation step.
    Note 1: Make sure you set the env variables in the same shell that is used to launch SDx. Also make sure the env variables are set before starting SDx, otherwise close and re-start SDx.
    Note 2: The below command might ask you execute unset LD_LIBRARY_PATH and then re-execute the command. Go ahead and follow those steps
  • Generate a Makefile for the Qt project.
  • Create a new SDx workspace.
  • Click on File > Import > General > Existing Projects into Workspace. Browse to the current working directory and make sure the "swaccel_qt" project is selected. Click finish.

  • Right-click the swaccel_qt project and click 'Build Project'.

  • Copy the generated swaccel_qt executable to the images directory.
User can now follow the above Board Setup steps to start the demo.

11 Support

To obtain technical support for this reference design, go to the:

  • Xilinx Answers Database to locate answers to known issues
  • Xilinx Community Forums to ask questions or discuss technical details and issues. Please make sure to browse the existing topics first before filing a new topic. If you do file a new topic, make sure it is filed in the sub-forum that best describes your issue or question e.g. Embedded Linux for any Linux related questions. Please include "ZCU102 Software Acceleration TRD" and the release version in the topic name along with a brief summary of the issue.