Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4


Table of Contents

Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#1.1 Board Setup
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#1.2 Run Flow
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#1.2.1 Graphic User Interface
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#1.2.2 GStreamer Application (vcu_gst_app)
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#1.2.3 Multistream
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#1.3 Build Flow
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#1.3.1 Hardware Design
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#1.3.2 VCU PetaLinux BSP
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#1.3.3 VCU APM library
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#1.3.4 VCU Video library
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#1.3.5 VCU GST library
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#1.3.6 VCU GST APP
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#1.3.7 VCU Qt Application
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#2 Other Information
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#2.1 Supported Features
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#3 Appendix A - Input Configuration File (input.cfg)
Zynq UltraScale+ MPSoC VCU TRD 2018.2 - Design Module 4#4 Appendix B
1 Tutorials
The below sections describe the build and run flow tutorials of the DM4 - PL Video Capture and Display design. For the overview, software tools and system requirements, and design files, follow the link below:

1.1 Board Setup

  1. Connect the Micro USB cable into the ZCU106 Board Micro USB port J83, and the other end into an open USB port on the host PC. This cable is used for UART over USB communication.
  2. Insert the SD card with the images copied into the SD card slot J100.
  3. Set the SW6 switches as shown in the below Figure. This configures the boot settings to boot from SD
  4. Connect 12V Power to the ZCU106 6-Pin Molex connector.
  5. Connect one end of Display Port (DP) cable to board’s U129 connector and the other end to DP port of 4K monitor.
  6. Connect one end of HDMI cable to board’s P7 stacked HDMI connector (lower port) and another end to HDMI source.
  7. Connect one end of HDMI cable to board’s P7 stacked HDMI connector (upper port) and another end to HDMI monitor.
  8. For USB storage device, connect USB hub along with the mouse. (Optional)
  9. For SATA storage device, connect SATA data cable to SATA 3.0 port. (Optional).
  10. For MIPI CSI-2, Insert the LI-IMX274MIPI-FMC image sensor daughter card into the FMC0 connector and set VADJ to 1.2V (See FMC VADJ Voltage Settings).
  11. Set up a terminal session between a PC COM port and the serial port on the evaluation board (See the Determine which COM to use to access the USB serial port on the ZCU106 board for more details).
  12. Switch ON SW1 to power the ZCU106 board.

The figure below shows the ZCU106 board connections for HDMI/MIPI design.


It takes about a minute for Linux to boot. The user may miss the boot sequence while the terminal session is being configured.

FMC VADJ Voltage Settings
VADJ voltage needs to be set to 1.2V for LI-IMX274MIPI-FMC image sensor daughter card. Refer to AR# 67308 for setting VADJ value.

Determine which COM to use to access the USB serial port on the ZCU106 board.
Make sure that the ZCU106 board is powered on and a micro USB cable is connected between ZCU106 board and host PC. This ensures that the USB-to-serial bridge is enumerated by the PC host.

Open your computer's Control Panel by clicking on Start > Control Panel.

Note that the Start button is typically located in the lower left corner of the screen. Occasionally, it is in the upper left corner.
  1. Click Device Manager to open the Device Manager window. Note: You may be asked to confirm opening the Device Manager. If so, click YES.
  2. Expand Ports (COM & LPT).
  3. Locate the Silicon Labs Quad CP210x USB to UART Bridge: Interface 0 (COM#).
  4. Note down the COM Port number for further steps.
  5. Close the Device Manager by clicking the red X in the upper right corner of the window.

Launch any Terminal application like Tera term to view the serial messages
  1. Launch Tera Term and open the COM port that is associated to Silicon Labs Quad CP210x USB to UART Bridge: Interface 0 of the USB-to-serial bridge.
  2. Set the COM port to 115200 Baud rate, 8, none, 1 –Set COM port.
  3. Power ON the board which has SD card. It boots Linux on board.

1.2 Run Flow



Please note that Windows formatter can't be used and partition should be FAT32, not FAT.
NOTE: The SD card file system is mounted at /media/card. Optional storage medium SATA and USB are mounted at /media/sata and /media/usb respectively.

1.2.1 Graphic User Interface

A QT based graphical user interface (GUI) provides control and monitoring interface. By default, GUI will run on DP only.

Run the below command to launch GUI on Display port when connected.
% run_vcu.sh
The GUI is not supported for HDMI Tx. In HDMI 4kp60 mode, GUI requires additional ~2GB memory bandwidth. If you invoke the GUI cause memory bandwidth issue. So HDMI Tx, use only command line application (vcu_gst_app). Table below provides the supported display interfaces for both GUI and command line application.
Display InterfaceGUICommand Line Application
DisplayPort
HDMINot supported

The figure below shows the home screen of DP when only DP is connected.


GUI only supports single stream playback. Supported resolutions are 1080p and 4kp. The supported frame rate for 1080p is 60 fps and 30fps for 4kp. For all multi-stream use case and 4kp60 single stream, the user has to run command line applications.

QT GUI has following options for the user to select:

Number of Input
This determines the number of active video sources. Only one input source is supported.

Output
This option allows the user to select sink for the pipeline. Supported output sink types is display port, record, and stream. For the Display Port , the available Codec options are either enc-dec or Pass-through. For Record and Stream Out, the available Codec option is Enc.

Input Settings
The below figure shows the input settings window with configuration options.


Different configuration options available in the QT pop up window are:

Input Source (Input 1)
The following video sources (4K) are available as part of input video selection:

Codec
The following options are available as part of Codec functionality

Preset
There are six predefined presets. If the user edits any control options preset mode switches to “Custom”.

Profile = Main, Rate Control= CBR, Filler Data = true, QP=uniform, L2 Cache = true, Latency mode = Normal, Low Bandwidth = true, Gop Mode = Basic, B-frame = 0, Slice = 4 and GoP Length = 30 common for all above options.

Please refer to the Supported Encoder Format section for the supported encoder format in this release.

Settings
A user can control the encoder, record and stream out configuration from the GUI, Settings option are enabled when the pipeline is in the stop state. User can control the encoder parameters when output option is selected as Display port/Record/Stream.

Encoder
This can be either H264 or H265.

Profile
The standard defines a set of capabilities, which are referred to as profiles, targeting specific classes of applications. These are declared as a profile code (profile_idc) and a set of constraints applied in the encoder. This allows a decoder to recognize the requirements to decode that specific stream.
H264 supports baseline, main and high profile. In H265 only main profile is supported.

QP
Quantization in an encoder is controlled by a quantization parameter. It specifies how to generate the QP per coding unit (CU). Two modes are supported-

Rate Control
Constant bitrate (CBR), variable bitrate (VBR) and low-latency are supported. CBR mode generates a constant bitrate that you can predefine. CBR is recommended for limited bandwidth use cases.

Bitrate
Encoding bit rate. In digital multimedia, bit rate often refers to the number of bits used per unit of playback time to represent a continuous medium such as audio or video after source coding (data compression).

B-frame
Short for bi-directional frame, or bi-directional predictive frame, a video compression method used by the MPEG standard. It ranges from 0 to 4.

Slice
The number of slices produced for each frame. Each slice contains one or more complete macroblock/CTU row(s). Slices are distributed over the frame as regularly as possible. If slice-size is defined as well more slices may be produced to fit the slice-size requirement
Range:
4-22 4kp resolution with HEVC codec
4-32 4kp resolution with AVC codec
4-32 1080p resolution with HEVC codec
4-32 1080p resolution with AVC codec

GoP Length
In video coding, a group of pictures, or GOP structure, specifies the order in which intra and inter-frames are arranged. And GoP-length is a length between two Intra frames. The GOP is a collection of successive pictures within a coded video stream. Each coded video stream consists of successive GOPs, from which the visible frames are generated. Its range is from 1 to 1000. NOTE: the GoP-length must be a multiple of B-Frames+1 AR71253)

Filler Data
Filler Data NAL units for CBR rate control to maintain constant bitrate throughout the playback. It can be enabled or disabled. It is applicable to CBR mode only.

L2 Cache
Enable or Disable L2Cache buffer in encoding process.

Latency Mode
Encoder latency mode. It can be normal or sub_frame mode.

Low Bandwidth
If enabled, decrease the vertical search range used for P-frame motion estimation to reduce the bandwidth.

Gop Mode
Group of Pictures mode. It can be Basic, low_delay_p, or low_delay_b.

Record
When Output option is selected as Record, A tab will be enabled in settings window through which user can record the video into the storage medium.




Storage
This option specifies storage device for the recorded file. The list is dynamically populated based on mounted storage devices. Supported storage devices are SD, USB, and SATA.

Output File Name
Name of the output file. For H264, recorded file get saved as <Source name>_H264_rec_<timestamp>.mp4 and for H265 it will be <Source name>_H265_rec_<timestamp>.mp4

Duration
This option will specify the Recording time duration. It ranges from 1-3 mins.

NOTE: Due to speed and storage constraints it is recommended to use USB/SATA storage devices for the record.

Stream-out
The stream-out panel allows the user to configure streaming parameters.

Stream-out Prerequisites:



Sink
Provides sink option for the stream-out. It is set to PS Ethernet.

Host IP
IP address of a computer or other Zynq® UltraScale+™ MPSoC board in which streaming video will be played on. It is set to 192.168.25.89 by default.

IP
This option will show the IP address of the board if the Ethernet link is up. If no Ethernet link is connected then it will show “Not Connected”.

Port
Port number of the Ethernet link. By default, it is set to 5004.

Steps for Stream Out
Here are the steps to get the ZCU106 TRD to play out to VLC.
v=0 c=IN IP4 192.168.25.90 m=video 5004 RTP/AVP 96 a=rtpmap:96 H265/90000 a=framerate=30
Depending on the network, change IP address and port number. IP address given in .sdp is ZCU106 board's IP. For H264 RTP client, change encoder to H264.

NOTE: In video streaming use case it is recommended to set b-frame=0

1.2.2 GStreamer Application (vcu_gst_app)

The vcu_gst_app is a command line multi-threaded Linux application that uses the vcu_gst_lib interface similar to vcu_qt. The command line application requires an input configuration file (input.cfg) to be provided in the plain text. Refer to the Appendix A - Input Configuration File (input.cfg) for the file format and description.

Before execution of vcu_gst_app, we need to check the HDMI-Rx link status. Refer Appendix B for how to check the HDMI Link Status
% vcu_gst_app < path to *.cfg file>

Note: Required Storage medium SATA and USB are mounted at /media/sata and /media/usb respectively.

configuration files(input.cfg) for various Resolutions are placed in the following directory structure in /media/card/
config/
├── 1-4kp60
│   ├── Display
│   ├── Record
│   └── Stream
├── 2-4kp30
│   ├── Display
│   ├── Record
│   └── Stream
├── 4-1080p60
│   ├── Display
│   ├── Record
│   └── Stream
└── input.cfg

Ex: 4kp60 HEVC_HIGH Display Pipeline execution

% vcu_gst_app /media/card/config/1-4kp60/Display/Single_4kp60_HEVC_HIGH.cfg
% GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency;scheduletime"  ./vcu_gst_app ./input.cfg >&& dump_log.txt

1.2.3 Multistream

When the number of input is more than one in the command line application, it is a multi-stream use case. The command line application supports multi-streaming, multi-recording, or multi-display.
The input source type can be HDMI and MIPI. For multi-streaming/multi-recording, the source type can be HDMI and MIPI.



1.3 Build Flow


The following tutorials assume that the $TRD_HOME environment variable is set as given below.
% export TRD_HOME=</path/to/downloaded/zipfile>/rdf0428-zcu106-vcu-trd-2018-2

NOTE: It is recommended to follow the build steps in sequence.

1.3.1 Hardware Design

Refer to the Vivado Design Suite User Guide: Using the Vivado IDE, UG893, for setting up Vivado environment.

On Linux:
  1. Open a Linux terminal
  2. Change directory to $TRD_HOME/pl
  3. Apply 2018.2 Vivado patch related to HDMI from AR71203
  4. Run the following command in Vivado shell to create the Vivado IPI project and invoke the GUI for VCU TRD hardware design.

% vivado -source scripts/vcu_trd_proj.tcl

After executing the script, the Vivado IPI block design comes up as shown in the below figure.






Note: If the user gets any pop-up with “No implementation Results available”. Click “Yes”. Then, if any pop-up comes up with “Launch runs”, Click "OK”.

The design is implemented and a pop-up window comes up saying “Open Implemented Design”. Click "OK".



After opening implemented design, the window looks as shown in the below figure.
Note: The actual results might graphically look different than the image shown








The HDF is created at $TRD_HOME/pl/vcu_trd_proj/zcu106_vcu_trd.sdk/zcu106_vcu_trd_wrapper.hdf

1.3.2 VCU PetaLinux BSP

This tutorial shows how to build the Linux image and boot image using the PetaLinux build tool.
PetaLinux Installation: Refer to the PetaLinux Tools Documentation (UG1144) for installation.
% source <path/to/petalinux-installer>/Petalinux-v2018.2/petalinux-v2018.2-final/settings.sh
% echo $PETALINUX
Post PetaLinux installation $PETALINUX environment variable should be set.
% cd $TRD_HOME/apu/vcu_petalinux_bsp
% petalinux-create -t project -s xilinx-vcu-trd-zcu106-zu7-v2018.2.bsp
% cd xilinx-vcu-trd-zcu106-zu7-v2018.2
% petalinux-config --get-hw-description=$TRD_HOME/pl/pre-built/vcu_trd/
% petalinux-build
NOTE:
In 2018.1, there is a separate BSP for SDI-RX/SDI-RX-TX and HDMI-Rx/VCU-TRD design. Now it is merged into a single BSP. With this BSP all the VCU TRD design modules(DM1, DM2, DM3, and DM4) can be built

NOTE:
% petalinux-config --get-hw-description=$TRD_HOME/pl/vcu_trd_proj/zcu106_vcu_trd.sdk
vi project-spec/meta-user/recipes-kernel/modules/kernel-module-hdmi_%.bbappend
 
SRC_URI_append = " \
                 file://0001-xilinx_drm_hdmi-Remove-unsupported-format-print-in-d.patch \
                 file://0001-Subject-PATCH-xilinx-hdmi-rx-Add-HPD-hack-for-ZCU106.patch \
"
% petalinux-build --sdk
% petalinux-package --sysroot
% cd images/linux
% petalinux-package --boot --fsbl zynqmp_fsbl.elf --u-boot u-boot.elf --pmufw pmufw.elf --fpga system.bit
cp BOOT.BIN image.ub $TRD_HOME/images/vcu_trd/rev-x
Note: rev-c binaries can be used for rev-d and later board versions.

Custom EDID Support
The TRD design is tested/validated with ABOX 2017 and NVIDIA SHIELD Pro. If you wants to try with any new HDMI source, you need to generate the EDID of the new source and update it in the TRD bsp. Refer to Custom EDID Support for adding the newly generated EDID file.

1.3.3 VCU APM library

This tutorial shows how to build the VCU APM library.
% source <compiled petalinux bsp path>/images/linux/sdk/environment-setup-aarch64-xilinx-linux
% cd $TRD_HOME/apu/apps
% xsdk -workspace . &&






1.3.4 VCU Video library

This tutorial shows how to build the VCU video library.



1.3.5 VCU GST library

This tutorial shows how to build the VCU GST library.




1.3.6 VCU GST APP

This tutorial shows how to build the VCU GST APP.




1.3.7 VCU Qt Application

This tutorial shows how to build the VCU Qt application.
% cd $TRD_HOME/apu/vcu_qt
% qmake vcu_qt.pro -r -spec linux-oe-g++
% make

% cp -f vcu_qt $TRD_HOME/images/vcu_trd/rev-x/bin/


2 Other Information


2.1 Supported Features

The table below provides the supported encoder feature in this release.
Encoder FormatGUICommand Line
Single StreamSingle StreamMulti-stream
HEVC 4kp60XNA
AVC 4kp60XNA
HEVC 4kp30√ (Max 2)
AVC 4kp30√ (Max 2)
HEVC 1080p60X√ (Max 4)
AVC 1080p60X√ (Max 4)

√ - Supported
NA – Not applicable
x – Not supported



3 Appendix A - Input Configuration File (input.cfg)

The example configuration files are stored at /media/card/config/ folder.

Common Configuration:
It is the starting point of common configuration.

Num of Input:
Provide the number of input. It is ranging from 1 to 4.

Output:
Select the video interface.
Options: HDMI or DP

Out Type:
Options: display, record, and stream

Display Rate:
Pipeline frame rate.
Options: 30 FPS or 60 FPS for each stream.

Exit:
It indicates to the application that the configuration is over.

Input Configuration:
It is the starting point of input configuration.

Input Num:
Starting Nth input configuration.
Options: 1-4

Input Type:
Input source type.
Options: TPG, HDMI, HDMI_2, HDMI_3, MIPI, File

Uri:
File path. Applicable for file playback only. File path should be appended by “file:/”
Options: file:/media/usb/abc.mp4

Raw:
To tell pipeline is processed or pass-through.
Options: True, False

Width:
The width of the live source.
Options: 3840, 1920

Height:
The height of the live source.
Options: 2160, 1080

Exit:
It indicates to the application that the configuration is over.

Encoder Configuration:
It is the starting point of encoder configuration.

Encoder Num:
Starting Nth encoder configuration.
Options: 1-4

Encoder Name:
Name of the encoder.
Options: AVC, HEVC

Profile:
Name of the profile.
Options: baseline, main or high for AVC. Main for HEVC.

Rate Control:
Rate control options.
Options: CBR, VBR, and low-latency.

Filler Data:
Filler Data NAL units for CBR rate control.
Options: True, False

QP:
QP control mode used by the VCU encoder.
Options: Uniform, Auto

L2 Cache:
Enable or Disable L2Cache buffer in encoding process.
Options: True, False

Latency Mode:
Encoder latency mode.
Options: normal, sub_frame

Low Bandwidth:
If enabled, decrease the vertical search range used for P-frame motion estimation to reduce the bandwidth.
Options: True, False

Gop Mode:
Group of Pictures mode.
Options: Basic, low_delay_p, low_delay_b

Bitrate:
Target bitrate in Kbps
Options: 1-60000

B Frames:
Number of B-frames between two consecutive P-frames
Options: 0-4

Slice:
The number of slices produced for each frame. Each slice contains one or more complete macroblock/CTU row(s). Slices are distributed over the frame as regularly as possible. If slice-size is defined as well more slices may be produced to fit the slice-size requirement.
Options:

4-22 4kp resolution with HEVC codec
4-32 4kp resolution with AVC codec
4-32 1080p resolution with HEVC codec
4-32 1080p resolution with AVC codec

GoP Length:
The distance between two consecutive I frames
Options: 1-1000

Format:
The format of input data.
Options: NV12

Preset:
Options: HEVC_HIGH, HEVC_MEDIUM, HEVC_LOW, AVC_HIGH, AVC_MEDIUM, AVC_LOW, Custom

Exit
It indicates to the application that the configuration is over.

Record Configuration:
It is the starting point of record configuration.

Record Num:
Starting Nth record configuration.
Options: 1-4.

Out-File Name:
Record file path.
Options: /media/usb/abc.mp4

Duration:
Duration in minutes.
Options: 1-3

Exit
It indicates to the application that the configuration is over.

Streaming Configuration:
It is the starting point of streaming configuration.

Streaming Num:
Starting Nth Streaming configuration.
Options: 1-4.

Host IP:
The host to send the packets to
Options: 192.168.25.89 or Windows PC IP

Port:
The port to send the packets to
Options: 5004,5008,5012 and 5016.

Exit
It indicates to the application that the configuration is over.

Trace Configuration:
It is the starting point of trace configuration.

FPS Info:
To display fps info on the console.
Options: True, False

APM Info:
To display APM counter number on the console.
Options: True, False

Pipeline Info:
To display pipeline info on console.
Options: True, False

Exit
It indicates to the application that the configuration is over.


4 Appendix B

Steps for GStreamer command line execution:

Support for only Single stream playback from the command line is available in this release.
Note: Using below commands, only limited configuration options are supported. Full-fledged command line support is available in next release.

$ killall -9 run_vcu.sh
$ killall -9 vcu_qt
$ killall -9 Xorg



$ modetest -D a0070000.v_mix -s 39:3840x2160-60@AR24 -w 35:"alpha":0