This page describes the process for installing, building, and testing support for the ZU+ Video Codec Unit (VCU) under the QNX RTOS. The Zynq UltraScale＋ MPSoC VCU TRD 2019.1 - VCU TRD : Multi Stream Design Module #1 is used as the hardware platform for the examples below.

Table of Contents

Introduction

Blackberry QNX provides support for the Zynq UltraScale+VCU when using their ZCU106 BSP for the QNX Neutrino RTOS. This is support is enabled by way of updates to the QNX Multimedia Suite 2.0 packages as referenced in the Required QNX RTOS Software Packages section below. In addition to the VCU support, there is an addition ADAS 2.0 package which provides MIPI camera support.

In the sections below, the process for building and evaluating the packages to exercise the VCU and MIPI camera input will be described.

Required QNX Software

It is assumed that the user has installed the following tools from QNX:

• QNX Software Center

• QNX Software Development Platform 7.0.4

  • available through QNX Software Center

  • 7.1 Support is Coming Soon

• QNX Momentics IDE

For general information about the ZCU106 BSP that is utilized for this project, you should refer to the ZCU106 BSP User Guide.

Xilinx Boot Files Archive

A set of pre-built boot files are available in the following archive:

• Location TBD

The archive also includes QNX .build files.

Recreating the Boot Files

The bitstream and FSBL are from the Zynq UltraScale＋ MPSoC VCU TRD 2019.1 - VCU TRD : Multi Stream design with one slight change. It is altered to remove the isolation configuration in the PS. Without this change, QNX will not boot.

To reproduce these files, use the Build flow for Design Module #1 (Multi Stream) and follow the steps below:

• Open the project in Vivado

• Disable isolation in the IPI Processor Configure Wizard (PCW).

• Build and export the hardware

• Create a Petalinux project based on BSP in the TRD

• Configure the Petalinux project to use the new .hdf

• Build the petalinux project

• The new boot artifacts will in the <petalinux project>/images/linux directory

Required QNX RTOS Software Packages

The build the QNX reference design for the ZCU106, the following runtime software packages are required from Blackberry QNX. The packages are available through the QNX Software Center application and are available upon request for current QNX customers. Please contact your QNX or Xilinx FAE for assistance.

Limitations

There are a few outstanding issues related to using the 4k display while encoding/decoding with the test applications below. At this time, it is recommended that users limit the display resolution to 1080p for best overall performance.

Building the QNX BSP Project

The QNX BSP Project can be built from the command line or using the QNX Momentics IDE. The Momentics flow running under Windows will be described below.

A “boot files archive” is needed (See “Xilinx Boot Files” section above) with all necessary components to recreate the BOOT.BIN for staring up the system.

Using the Momentics IDE Flow 

Create the ZCU106 BSP Project

1. Launch Momentics and create a new workspace.

2. Create a new project based on the ZCU106 BSP 

   • File->Import

   • Choose QNX->QNX Source Package and BSP. Click Next

   • Browse for the ZCU106 BSP

     • <QNX install path>\qnx700\bsp\BSP_xilinx-zynq-zcu106_br-700_be-700_SVN916472_JBN11.zip

     • Click Next Twice

     • Give the project a suitable name

     • Click Finish

Modify the Project

1. Replace the default zcu106.build file with the one of the ones provided in the boot files archive. There are two options, depending on which set of applications you want to test:
   
   

   • adas/zcu106.build.adas: Use this if you want to capture, display, and record live video from the FMC MIPI camera. You’ll also need to install the ADAS packages mentioned in the table above.

   • multimedia/zcu106.build: Use this to simply exercise the VCU encoder/decoder with the QNX multimedia and Xilinx VCU test apps
     
     In either case, copy the .build file into the QNX BSP project and rename it zcu106.build. In the QNX project, the .build file is found under xilinx-zynq-zcu106->images in the project explorer.
     
     

2. Copy all of the .conf files in the graphics directory of the boot files archive to the following directory (this only needs to be done once and can be used for other projects):
   <QNX install path>/qnx700/target/qnx7/aarch64le/usr/lib/graphics/zcu102
   
   

3. Update the Makefile in the images directory. Modify the QNX-IFS target as shown below so that QNX-IFS gets updated during every build:

Build the Project

1. Right click on the project name and select "Clean Project”

2. Right click on the project name and select "Build Project"

3. Wait for the build to complete. This process will take several minutes.

4. Once complete, the QNX-IFS will be available in the images folder.

Create the Boot Image

1. Copy the QNX Initial Program Loader (IPL) (<qnx proj dir>/prebuilt/aarch64le/boot/sys/ipl-xzynq-ultrascale-zcu10x) to the folder that includes the provided 2019.1 VCU TRD bootfiles. Rename the file to ipl-xzynq-ultrascale-zcu10x.elf.

2. Open an xsct command shell, cd to the boot files directory, and use the Xilinx Bootgen tool to generate BOOT.BIN with the following command:
   
   

   bootgen -image boot_vcu_trd.bif -arch zynqmp -o BOOT.BIN -w

Here is an example boot_vcu_trd.bif:

Preparing the Hardware

By default, this reference design targets the 2019.1 VCU TRD Design Module #1. With modification, other similar hardware configurations can be supported as well - please refer to Appendix A below. From a QNX software perspective, this reference design supports a subset of the VCU TRD interfaces as defined below:

Supported Hardware Interfaces

Sources

• MIPI CSI-2 Rx capture pipeline implemented in the PL

  • Capture Pipeline: MIPI CSI-2 Subsystem → Demosaic → Gamma → VPSS(CSC) → VPS (Scaler) → Framebuffer Write

• File source: SD card

Sinks

• DP Tx display pipeline in the PS.

• File sink: SD card

VCU Codec

• Video Encode/Decode capability using VCU hard block in PL 

  • AVC/HEVC encoding

  • Encoder/decoder parameter configuration.

Video Input Pipeline

MIPI CSI-2 Subsystem → Demosaic → Gamma → VPSS(CSC) → VPS (Scaler) → Framebuffer Write

Setting up the board

To set the board up for testing, please refer to the Board Setup instructions in the VCU TRD: Zynq UltraScale+ MPSoC VCU TRD 2019.1

Booting up the Board

1. Add BOOT.BIN and QNX-IFS to the FAT partition of an SD Card. For best results, use the SD Memory Card Formatter from the SD Association to format the card.

2. Insert the card in ZCU106 and power on the board.

3. When prompted by the QNX IPL, select 'M' to boot from the SD Card.

As part of the default boot process, the boot script will attempt to mount the SD cards at /sdcard.

Setting the MAC Address

To use the Ethernet interface, you’ll need make sure that the MAC address for the board is programmed into the on-board EEPROM. During boot, if you see an error about the MAC address not being set, refer to the ZCU106 BSP User Guide for details regarding programming the MAC address into the EEPROM.

Running the ADAS Test Applications

The camera_example application can be used to exercise the available hardware interfaces. The examples in this section are provided by the QNX® Platform for ADAS 2.0 packages listed above. The available tests are highlighted below. Source code for the camera_example application is available from QNX upon request.

To run using pre-builts, you should use the QNX-IFS file located in the /adas directory of the QNX Pre-built archive. This version of the QNX-IFS is built using adas/zcu106.build.

Camera Viewfinder Test

In the serial console, run the camera_example application and press 1 for “Camera viewfinder”. After that, select all default options:

The camera streaming starts and the viewfinder should show up on the connected DisplayPort Monitor

Camera Recording Test

In the serial console, start the VCU encoder application:

Now run the camera_example application and press 2 for “Record video to a file”. After that, select all default options:

At the final menu, choose “r” to start recording and choose "r" again to stop recording. While recording, you should see the live video on the DisplayPort monitor.

Video File Playback Test

In the serial console, start the VCU decoder application:

Restart the sensor service with a different .conf file to configure it as a “File Camera”. The playback.conf file is defined in zcu106.build.

Now run the camera_example application and press 1 for “Camera Viewfinder”. After that, select all default options:

The video at /sdcard/video.mp4 will begin to play on the monitor. Once the end of the video is reached, it will replay continuously until you press 'x' to exit the test.

Running the Multimedia Test Applications

The examples in this section are provided by the QNX® Platform for Multimedia 2.0 packages listed above. The available tests are highlighted below.

To run using pre-builts, you should use the QNX-IFS file located in the /multimedia directory of the boot files archive. This version of the QNX-IFS is built using multimedia/zcu106.build.

Preparing The System

Once the system is booted, run the following commands:

Xilinx VCU Tests

Included in the multimedia package are four sample applications from Xilinx to exercise the VCU at the Control Software and OpenMax layers. These sample apps operate on raw video files (.hevc, .h264, . yuv, etc. ) instead of containerized formats like .mp4.

VCU Control Software

The VCU control software is a pair of standalone command-line executables that can be used to gain fine-grained control over the operation of the VCU:

• VCU Encode: ctrlsw_encoder

• VCU Decode: ctrlsw_decoder

Command Line Examples

Encode using ctrlsw_encoder - this takes a .cfg file as an input and generates an avc or hevc encoded output file based on the YUV formated input file. The configuration file details can be found in PG252. A sample configuration file can be found in Appendix B below.

For more information on usage of the control software applications, please refer to PG252: VCU Control Software Sample Applications. The source code for the VCU Control Software is available at https://github.com/Xilinx/vcu-ctrl-sw.

VCU OpenMax Sample Applications

The OpenMax Integration Layer (OMXIL) sample applications provide example for utilizing the OpenMax Integration Layer.

• VCU Encode: vcu_omx_encoder

• VCU Decode: vcu_omx_decoder

Command Line Examples

Decode a encoded video to YUV:

Encode a 1080p60 YUV video:

For more information on the usage of the OpenMax sample applications, please refer to PG252: VCU OpenMax Integration Layer. The source code for the OpenMax Integration Layer sample applications is available at https://github.com/Xilinx/vcu-omx-il/tree/master/exe_omx.

QNX Codec Tests

mmrplay

• Used to playback .mp4 videos on the DisplayPort interface
  
  Usage:

Changing the Resolution of the Display

To change the resolution of the DisplayPort interface, you need to restart the screen service and specify a new screen configuration file. Screen configuration files have been provided for the following resolutions:

• 1920x1080 (default)

• 3840x2160

• 1280x720

To restart the screen service, issue the following commands:

Mounting an NFS Partition

For some of the tests above, it’s beneficial to use an NFS mounted filesystem due to the size of the files and the amount of bandwidth necessary. The commands below give an example for connecting to an NFS server. It is assumed that the NFS server is already running and configured correctly.

Other Hardware Configurations

It’s possible to use the QNX software implementation provided with this reference design with other hardware configurations. The sections below describe the modifications need to run this design on the design module #3.

Modifying the QNX BSP Project

Changes are required to the zcu106.build file to support the new hardware configuration.

Adding AXI Interrupt Controller

The ZCU106 BSP supports the use of the AXI Interrupt Controller soft IP in the PL to aggregate interrupts before sending a single interrupt to the GIC through the PL->PS interface. In order to use it, we need to enable it at startup and define the characteristics of the interrupt.

In this design, the AXI INTC is configured as follows:

• Base address = 0xA0055000

• Interrupt output is connected to GIC 142 (ps->pl IRQ ).

• The interrupt output is active high, level triggered

In the .boostrap section of the .build file, add the following command:

Updating the Sensor Service Configuration

By default, several xilinx-specific parameters used by the sensor service configuration are set to default values that match the 2019.1 VCU TRD Design Module #1 hardware settings. In order to configure the sensor service to values that don’t match DM#1, we can override the parameter settings in imx274.conf as shown below.

In zcu106.build, replace the /etc/imx274.conf section with:

Please refer to Appendix A: Hardware Configurations for details on the hardware configurations of DM#1 and DM#3.

Appendix A: Hardware Configurations

The following sections provide details on the PL configuration of various Vivado designs that are compatible with the QNX ADAS software libraries.

2019.1 VCU TRD Design Module #1

Image Sensor Configuration

Sensor Type: IMX274 (LI-IMX274MIPI-FMC)

Sensor I2C Interface

Sensor Video Pipeline

Note: This EMIO for MIPI RX is an active low SPI ChipSelect

Video Codec Unit IP Parameters

AXI Interrupt Controllers (PL Based)

2019.1 VCU TRD Design Module #3

Image Sensor Configuration
Sensor Type: IMX274 (LI-IMX274MIPI-FMC)
Sensor I2C Interface

 Sensor Video Pipeline

Note: This EMIO for MIPI RX is an active low SPI ChipSelect

Video Codec Unit IP Parameters

AXI Interrupt Controllers (PL Based)

Common Platform Notes

• For QNX, MIO is mapped to GPIO 0-77, EMIO is mapped to GPIO 78-173

• pl_ps_irq0 = GIC 121:128, pl_ps_irq1 = GIC 136:143 . QNX uses actual GIC number.

• For Interrupts connected through AXI_INTC, the logical interrupt numbers used by QNX are as follows:

Appendix B: Example ctrlsw_encoder Config File Format

The following example configuration file is used to a convert a 1080p YUV420 file avc/h264. For more info, refer to PG252.

ctrlsw_encoder -cfg avc_1080p60.cfg

Input Summary:

• file: csw_AIR_1080p30_nv12_4B_h264.yuv

• Size: 1920x1080

• Color Space: I420

Output Summary:

• file: csw_AIR_1080p30_nv12_4B_h264_yuv.h264

• profile: AVC_MAIN Level 4

Appendix C: Test Videos

Test .mp4 Videos

Big Buck Bunny: https://download.blender.org/demo/movies/BBB/

Elephant’s Dream: https://archive.org/details/ElephantsDream

Creating avc/hevc Encoded Videos From .mp4

Creating .mp4 From avc/hevc Encoded Videos

Raw Videos

https://media.xiph.org/video/derf/

These .y4m videos can be converted to YUV with the following ffmpeg command: