This page provides all the information related to VCU TRD PL DDR HDR10 HDMI design.

Table of Contents

1 Overview

This module supports the reception and insertion of HDR10 static metadata for HDMI. This HDR10 metadata that contains critical information needed to support HDR will be carried throughout the pipeline - from the source to the sink. It enables the capture of HDR10 video from an HDMI-Rx Subsystem implemented in the PL. The video can be displayed through HDR10 compatible HDMI-Tx through the PL and recorded in SD cards or USB/SATA drives. The module can Stream-in or Stream-out HDR10 encoded data through an Ethernet interface. This module supports single-stream for XV20 and XV15 format.

This is the new design approach proposed to use PL_DDR for decoding and PS_DDR for encoding so that DDR bandwidth would be enough to support high bandwidth VCU applications requiring simultaneous encoder and decoder operations and transcoding at 4k@60 FPS. This approach makes the most effective use of limited AXI4 read/write issuance capability in minimizing latency for the decoder. DMA buffer sharing requirements determine how capture, display, and intermediate processing stages should be mapped to the PS or PL DDR.

This design supports the following video interfaces:
Sources:

• HDMI-Rx capture pipeline implemented in the PL.

• File source (SD card, USB storage, SATA hard disk).

• Stream-In from network or internet.

Sinks:

• HDMI-Tx display pipeline implemented in the PL.

VCU Codec:

• Video Encode/Decode capability using VCU hard block in PL 

  • AVC/HEVC encoding

  • Encoder/decoder parameter configuration.

Video format:

• XV20, XV15

Supported Resolution:

The table below provides the supported resolution from the command line app only in this design.

√ - Supported
NA – Not applicable
x – Not supported

The below table gives information about the features supported in this design. 

The below figure shows the PL DDR HDR10 HDMI design hardware block diagram.

Open

The below figure shows the PL DDR HDR10 HDMI design software block diagram.

Open

1.1 Hardware, Software Tools and System Requirements

1.1.1 Hardware Tools

Required :

• ZCU106 evaluation board (rev C/D/E/F/1.0) with power cable

• HDR10 compatible Monitor with HDMI input supporting 3840x2160 resolution

• HDMI cable 2.0 certified

• Class-10 SD card

• HDMI Receiver - NVIDIA SHIELD Pro

• Ethernet cable

Optional :

• USB pen drive formatted with the FAT32 file system and hub

• SATA drive formatted with the FAT32 file system, external power supply, and data cable

1.1.2 Software Tools

Required :

• Linux host machine for all tool flow tutorials (see UG1144 for detailed OS requirements)

• Petalinux Tools version 2020.1 (see UG1144 for installation instructions)

• VIVADO Design suite version 2020.1

• Git a distributed version control system

• Silicon Labs quad CP210x USB-to-UART bridge driver

• Serial terminal emulator e.g. teraterm

Compatibility :
The VCU HDR10 design has been tested successfully with the following user-supplied components.

HDMI Monitor :

HDMI Input Sources:

• NVIDIA SHIELD Pro

Cable:

• HDMI 2.0 compatible cable

1.2 Download, Installation, and Licensing

The Vivado Design Suite User Guide explains how to download and install the Vivado® Design Suite tools, which include the Vivado Integrated Design Environment (IDE), High-Level Synthesis tool, and System Generator for DSP. This guide also provides information about licensing and administering evaluation and full copies of Xilinx design tools and intellectual property (IP) products. The Vivado Design Suite can be downloaded from here.

LogiCORE IP Licensing:

The following IP cores require a license to build the design.

• Video Mixer- Included with Vivado - PG243

• Video PHY Controller - Included with Vivado - PG203

• HDMI-Rx/Tx Subsystem - Purchase license (Hardware evaluation available) - PG235 & PG236

• Video Processing Subsystem (VPSS) - Included with Vivado - PG231

To obtain the LogiCORE IP license, please visit the respective IP product page and get the license.

• AR# 44029 - Licensing - LogiCORE IP Core licensing questions

• Xilinx Licensing FAQ

• LogiCORE IP Project License Terms

1.3 Board Setup

The below section will provide the information on the ZCU106 board setup for running VCU HDR10 design.

1. Connect the Micro USB cable into the ZCU106 Board Micro USB port J83, and the other end into an open USB port on the host PC. This cable is used for UART over USB communication.

2. Insert the SD card with the images copied into the SD card slot J100.

3. Set the SW6 switches as shown in the below Figure. This configures the boot settings to boot from SD.

4. Connect 12V Power to the ZCU106 6-Pin Molex connector.

5. Connect one end of HDMI cable to the board’s P7 stacked HDMI connector (lower port) and another end to HDMI source.

6. Connect one end of HDMI cable to the board’s P7 stacked HDMI connector (upper port) and another end to HDMI monitor.

7. For SATA storage device, connect SATA data cable to SATA 3.0 port. (Optional)

8. Set up a terminal session between a PC COM port and the serial port on the evaluation board (See the Determine which COM to use to access the USB serial port on the ZCU106 board for more details)

9. Copy the VCU HDR10 images into the SD card and insert the SD card on the board.

10. The below images will show how to connect interfaces on the ZCU106 board.

The above  figure shows all the ZCU106  board connections

1.3.1 Determine which COM to use to access the USB serial port on the ZCU106 board

Make sure that the ZCU106 board is powered on and a micro USB cable is connected between the ZCU106 board and host PC. This ensures that the USB-to-serial bridge is enumerated by the PC host.
Open your computer's Control Panel by clicking on Start > Control Panel.
Note that the Start button is typically located in the lower-left corner of the screen. Occasionally, it is in the upper left corner.

1. Click Device Manager to open the Device Manager window. You may be asked to confirm opening the Device Manager. If so, click YES.

2. Expand Ports (COM & LPT).

3. Locate the Silicon Labs Quad CP210x USB to UART Bridge: Interface 0 (COM#).

4. Note down the COM Port number for further steps.
5. Close the Device Manager by clicking the red X in the upper right corner of the window.

Launch any Terminal application like Tera term to view the serial messages

1. Launch Tera Term and open the COM the port that is associated with Silicon Labs Quad CP210x USB to UART Bridge: Interface 0 of the USB-to-serial bridge.

2. Set the COM port to 115200 Baud rate, 8, none, 1 –Set COM port.

3. Power ON the board which has an SD card. Switch ON SW1 to power the ZCU106 board.

4. It boots Linux on board and It takes about a minute for Linux to boot. 

1.4 Download the TRD

• Download the VCU TRD 2020.1 HDR10 release package here.

  • https://www.xilinx.com/member/forms/download/design-license.html?cid=ac612281-451c-4257-b5b7-b1ae8d0f57ed&filename=rdf0428-zcu106-vcu-hdr10-2020-1.zip

1.5 Run Flow

The TRD package is released with the source code, Vivado project, Petalinux BSP, and SD card image that enables the user to run the demonstration. It also includes the binaries necessary to configure and boot the ZCU106 board. Prior to running the steps mentioned in this wiki page, download the TRD package and extract its contents to a directory referred to as TRD_HOME which is the home directory. TRD package contents are placed in the following directory structure.

TRD package contents specific to VCU HDR10 HDMI design are placed in the following directory structure.

Configuration files(input.cfg) for various resolutions are placed in the following directory structure.

1.5.1 Prepare a SD-Card

• The user needs to copy all the files from the $TRD_HOME/images/vcu_hdr10_hdmi/ to FAT32 formatted SD card directory.

• Power on the board; make sure INIT_B, DONE and all power rail LEDs are lit green

• After a successful boot, a login prompt would appear as shown below. Enter username and password as root and root respectively to login to the board file system.

1.5.2 GStreamer Application (vcu_gst_app)

The vcu_gst_app is a command-line multi-threaded Linux application. The command-line application requires an input configuration file (input.cfg) to be provided in plain text.

Run below modetest command to set CRTC configurations for 4kp60:

Run below modetest command to set CRTC configurations for 4kp30:

Execution of the application is shown below:

Examples:

4kp60 XV20 HEVC_HIGH HDR10 Display Pipeline execution

4kp60 XV20 HEVC_HIGH HDR10 Record Pipeline execution

4kp60 XV20 HEVC_HIGH HDR10 Stream-out Pipeline execution

4kp60 XV20 HEVC_HIGH HDR10 Stream-in Pipeline execution

To measure the latency of the pipeline, run the below command. The latency data is huge, so dump it to a file.

1.6 Build Flow

The following tutorials assume that the $TRD_HOME environment variable is set as given below.

1.6.1 Hardware Design

Refer to the Vivado Design Suite User Guide: Using the Vivado IDE, UG893, for setting up the Vivado environment.

On Linux:

1. Open a Linux terminal

2. Change directory to $TRD_HOME/pl

3. Run the following command in Vivado shell to create the Vivado IPI project for VCU HDR10 and invoke the GUI.

After executing the script, the Vivado IPI block design comes up as shown in the below figure.

• Click on “Generate Bitstream”.

The design is implemented and a pop-up window comes up saying “Open Implemented Design”. Click "OK".

After opening the implemented design, the window looks as shown in the below figure.

Go to File > Export > Export Hardware

In the Export Hardware Platform select Fixed in the Platform type and click Next.

In the next window, select Include bitstream and click Next.

The default XSA file name is <hardware design name_wrapper>. Choose the path where the XSA file has to be written.

The XSA is created at $TRD_HOME/pl/build/zcu106_HDR10/zcu106_HDR10_wrapper.xsa for VCU HDR10 hardware design .

Click Finish for the XSA file to be generated.

1.6.2 VCU PetaLinux BSP

This tutorial shows how to build the Linux image and boot image using the PetaLinux build tool.
PetaLinux Installation: Refer to the PetaLinux Tools Documentation UG1144 for installation.

Post PetaLinux installation $PETALINUX environment variable should be set.

• Create a PetaLinux project.

• Configure the PetaLinux project.

• If the Vivado project is modified, the user is expected to configure the bsp with the modified .xsa file and build. e.g.

• Create a soft link of design dtsi file to system-user.dtsi using below command

• Build the PetaLinux project

• Build SDK components to use it as sysroot for application development.

• Create a boot image (BOOT.BIN) including FSBL, ATF, bitstream, and u-boot.

• Copy the generated boot image and Linux image to the SD card directory.

1.6.3 VCU GST APP

vcu_gst_app and supporting library will be built as a vcu-gst-app recipe inside petalinux-project. Refer project-spec/meta-user/recipes-apps/vcu-gst-app directory inside petalinux project for vcu-gst-app recipe. Source of vcu_apm_lib, vcu_video_lib, vcu_gst_lib and vcu_gst_app is provided as zip inside project-spec/meta-user/recipes-apps/vcu-gst-app/files/ directory. vcu_gst_app will be built as part of petalinux project and the executable is placed in /usr/bin/ location of rootfs. User can update the zip file if any source code modifications need to be and run following command to build vcu-gst-app recipe.

2 Other Information

2.1 Known Issues

• For Petalinux related known issues please refer: PetaLinux 2020.1 - Product Update Release Notes and Known Issues

• For VCU related known issues please refer AR# 66763: LogiCORE H.264/H.265 Video Codec Unit (VCU) - Release Notes and Known Issues and Xilinx Zynq UltraScale+ MPSoC Video Codec Unit.

• The BenQ Monitor (Model-No : 9H.LGVLA.TSN) not able to identify the HDR input signal: It is observed that, while playing HDR10 contents with VCU PLDDR HDR10 pipelines on zcu106 with BenQ Monitor, "HDR ON" pop-up does not appears on Monitor. It is expected that “HDR ON“ pop-up should be appear on the mentioned BenQ Monitor - while playing HDR10 contents. → Please refer AR# 75482 : Why do some monitors not show the HDR ON label while playing HDR content? for the fixes of this issue.

2.2 Limitations

• For Petalinux related limitations please refer: PetaLinux 2020.1 - Product Update Release Notes and Known Issues

• For VCU related limitations please refer AR# 66763: LogiCORE H.264/H.265 Video Codec Unit (VCU) - Release Notes and Known Issues, Xilinx Zynq UltraScale+ MPSoC Video Codec Unit and PG252.

2.3 Optimum VCU Encoder parameters for use-cases.

Video streaming:

• Video streaming use-case requires a very stable bitrate graph for all pictures.

• It is good to avoid periodic large Intra pictures during the encoding session

• Low-latency rate control (hardware RC) is the preferred control-rate for video streaming, it tries to maintain equal amount frame sizes for all pictures.

• Good to avoid periodic Intra frames instead use low-delay-p (IPPPPP…)

• VBR is not a preferred mode of streaming.

Performance: AVC Encoder settings:

• It is preferred to use 8 or higher slices for better AVC encoder performance.

• AVC standard does not support Tile mode processing which results in the processing of MB rows sequentially for entropy coding.

Quality: Low bitrate AVC encoding:

• Enable profile=high and use qp-mode=auto for low-bitrate encoding use-cases.

• The high profile enables 8x8 transform which results in better video quality at low bitrates.

3 Appendix A - Input Configuration File (input.cfg)

The example configuration files are stored at /media/card/config/ folder.

Common Configuration:
It is the starting point of common configuration.
 
Num of Input:
1

Output:
Select the video interface.
Options: HDMI

Out Type:
Options: display, record, and stream

Display Rate:
Pipeline frame rate.
Options: 30 FPS or 60 FPS for each stream

Exit:
It indicates to the application that the configuration is over

Input Configuration:
It is the starting point of the input configuration.

Input Num:
Starting N^th input configuration.
Options: 1

Input Type:
Input source type.
Options: HDMI, File, Stream

Uri:
File path or Network URL. Applicable for file playback and stream-in pipeline only. Supported file formats for playback are ts, mp4, and mkv.
Options: file:///media/usb/abc.ts (for file path), udp://192.168.25.89:5004/ (for Network streaming, Here 192.168.25.89 is IP address and 5004 is port no)

Raw:
To tell the pipeline is processed or pass-through.
Options: False

Width:
The width of the live source.
Options: 3840, 1920

Height:
The height of the live source.
Options: 2160, 1080

Format:
The format of input data.
Options: XV20, XV15

HDR Mode:
Specifies High Dynamic Range(HDR) mode
Options: HDR10, NONE

Exit:
It indicates to the application that the configuration is over.

Encoder Configuration:
It is the starting point of encoder configuration.

Encoder Num:
Starting N^th encoder configuration.
Options: 1

Encoder Name:
Name of the encoder.
Options: AVC, HEVC

Profile:
Name of the profile.
Options: high for AVC and main for HEVC.

Rate Control:
Rate control options.
Options: CBR, VBR, and low-latency.

Filler Data:
Filler Data NAL units for CBR rate control.
Options: True, False

QP:
QP control mode used by the VCU encoder.
Options: Uniform, Auto

L2 Cache:
Enable or Disable L2Cache buffer in encoding process.
Options: True, False

Latency Mode:
Encoder latency mode.
Options: normal, sub_frame

Low Bandwidth:
If enabled, decrease the vertical search range used for P-frame motion estimation to reduce the bandwidth.
Options: True, False

Gop Mode:
Group of Pictures mode.
Options: Basic, low_delay_p, low_delay_b

Bitrate:
Target bitrate in Kbps
Options: 1-60000

B Frames:
Number of B-frames between two consecutive P-frames
Options: 0-4

Slice:
The number of slices produced for each frame. Each slice contains one or more complete macroblock/CTU row(s). Slices are distributed over the frame as regularly as possible. If slice-size is defined as well more slices may be produced to fit the slice-size requirement.
Options:
4-22 4kp resolution with HEVC codec
4-32 4kp resolution with AVC codec
4-32 1080p resolution with HEVC codec
4-32 1080p resolution with AVC codec

GoP Length:
The distance between two consecutive I frames
Options: 1-1000

GDR Mode:
It specifies which Gradual Decoder Refresh(GDR) scheme should be used when gop-mode = low_delay_p
Options: Horizontal/Vertical/Disabled

Entropy Mode:
It specifies the entropy mode for H.264 (AVC) encoding process
Options: CAVLC/CABAC/Default

Max Picture Size:
It is used to curtail instantaneous peak in the bit-stream. It works in CBR/VBR rate-control only. When it is enabled, max-picture-size value is calculated and set with 10% of AllowedPeakMargin.
i.e. max-picture-size = (TargetBitrate / FrameRate) * 1.1
Options: TRUE/FALSE

Preset:
Options: HEVC_HIGH, HEVC_MEDIUM, HEVC_LOW, AVC_HIGH, AVC_MEDIUM, AVC_LOW, Custom

Exit
It indicates to the application that the configuration is over.

Record Configuration:
It is the starting point of record configuration.

Record Num:
Starting N^th record configuration.
Options: 1

Out-File Name:
Record file path.
Options: /media/usb/abc.ts

Duration:
Duration in minutes.
Options: 1-3

Exit
It indicates to the application that the configuration is over.

Streaming Configuration:
It is the starting point of streaming configuration.

Streaming Num:
Starting N^th streaming configuration.
Options: 1

Host IP:
The host to send the packets to
Options: 192.168.25.89 or Windows PC IP

Port:
The port to send the packets to
Options: 5004

Exit
It indicates to the application that the configuration is over.

Trace Configuration:
It is the starting point of trace configuration.

FPS Info:
To display fps info on the console.
Options: True, False

APM Info:
To display the APM counter number on the console.
Options: True, False

Pipeline Info:
To display pipeline info on console.
Options: True, False

Exit
It indicates to the application that the configuration is over.

4 Appendix B - HDMI-Rx/Tx Link-up and GStreamer Commands

This section covers configuration of HDMI-Rx using media-ctl utility and HDMI-Tx using modetest utility, along with demonstrating HDMI-Rx/Tx link-up issues and steps to switch HDMI-Rx resolution. It also contains sample GStreamer HDMI video pipelines for Display, Record & Playback, Stream-in and Stream-out use-cases.

• HDMI source can be locked to any resolution. Run the below command for all media nodes to print media device topology. where, mediaX represents different media nodes. In the topology, log look for the v_hdmi_rx_ss string to identify the HDMI input source media node.

• To check the link status, resolution and video node of the HDMI input source, run below media-ctl command, where ,mediaX indicates media node for the HDMI input source.

• When HDMI source is connected to 4KP60 resolution, it shows:

• When the HDMI source is not connected, it shows:

Notes for gst-launch-1.0 commands:

• Video node for HDMI-Rx source can be checked using media-ctl command. Run below media-ctl command to check video node for HDMI-Rx source. where, media0 indicates media node for HDMI input source.

• Make sure the HDMI-Rx media pipeline is configured for 4kp60 resolution and source/sink has the same color format for connected nodes. For XV20 format, run below media-ctl commands to set resolution and format of HDMI scaler node where media0 indicates media node for HDMI input source.

When HDMI Input Source is NVIDIA SHIELD

• Follow the below steps to switch the HDMI-Rx resolution from 1080p60 to 4kp60.

  • Check current HDMI Input Source Resolution (1080p60) by following the steps mentioned earlier to check HDMI-Rx resolution using media-ctl command.

  • Run vcu_gst_app for current HDMI-Rx resolution (1080p60) by executing the following command.

Below configurations needs to be set in input.cfg for HDMI-Rx 1080p60 resolution.

• Change Resolution of HDMI Input Source from 1080p60 to 4kp60 by following the below steps.

  • Set the HDMI source resolution to 4kp60 (Homepage → Settings → Display & Sound → Resolution → change to 4kp60).

  • Save the configuration to take place the change.

  • Verify the desired HDMI Input Source Resolution (4kp60) by following the above-mentioned steps.

• If the HDMI-Tx link-up issue is observed after Linux booting, use the following command to get the blue screen on HDMI-Tx for 4kp60.

• The table below lists the parameters of the pixel format.

• Run the following gst-launch-1.0 command to display the XV20 HDR10 video on HDMI-Tx using the GStreamer pipeline ( capture(HDR10) → encode → decode → display(HDR10) ).

• Run the following gst-launch-1.0 command to record the XV20 HDR10 video using the GStreamer pipeline.

• Run the following gst-launch-1.0 command to play XV20 HDR10 recorded file on HDMI-Tx using the GStreamer pipeline.

• Run the following gst-launch-1.0 command to stream-out the XV20 HDR10 video using the GStreamer pipeline.

• Run the following gst-launch-1.0 command to display XV20 HDR10 stream-in video on HDMI-Tx using the GStreamer pipeline.