Zynq UltraScale+ MPSoC VCU TRD 2021.2 - PL DDR HLG SDI Audio Video Capture and Display

Owned by Neel Gandhi
Last updated: Apr 07, 2022 by Sunil Vaghela
25 min read

This page provides detailed information related to Design Module 2 - HLG SDI Video Capture and Display with PLDDR

Table of Contents

1 Overview

The primary goal of this Design is to demonstrate the capabilities of VCU hard block present in Zynq UltraScale+ EV devices. The TRD will serve as a platform to tune the performance parameters of VCU and arrive at optimal configurations for encoder and decoder blocks. It has also added an initial support of 8-channels audio.

This module enables the capture of the Hybrid Log Gamma(HLG) video from an SDI-Rx subsystem implemented in the PL. The Hybrid Log Gamma(HLG) video can be displayed through the SDI-Tx subsystem implemented in the PL. The module can stream-out and stream-in live captured video frames through an Ethernet interface. This module supports single-stream for XV20 pixel format. In this design, PL_DDR is used for decoding and PS_DDR for encoding so that DDR bandwidth would be enough to support high bandwidth VCU applications requiring simultaneous encoder and decoder operations and transcoding at 4k@60 FPS.

This module supports the Encoding-Decoding and Transmission of Hybrid Log Gamma(HLG) video along with backward compatible Standard Dynamic Range(SDR) for SDI. It provides the ability to encode a wide dynamic range, while still being compatible with the existing transmission standards in the standard dynamic range (SDR) region. This HLG format encodes the HDR and SDR information in single signal enabling HDR-compatible TVs to display an enhanced image. Unlike HDR it does not have any metadata, rather it will use the ATC(Alternative transfer characteristics) SEI(supplemental enhanced information) information in the VUI(video usability information) to add extra encoding details.

From VCU point of view, there are two "types" of HLG, which you can enable:

  1. There is a HLG-SDR Backwards Compatible Mode, which uses the BT2020 value in the SPS VUI parameters instead of the HLG transfer characteristics. Then the VCU encoder will insert a 'Alternative Transfer Characteristics' (ATC) SEI with the HLG value. See below video frame snapshot captured in stream-eye:

Depending on version of stream-eye, you may not see SEI message correctly. But if you look at HEX viewer you will see ATC SEI in bit-stream.

0x93 - Payload Type (147 == ATC)
0x01 - Payload Size (1 byte)
0x12 - 18 (HLG EOTF value)
0x80 - payload bits ending

2. There is a HLG only mode. This directly uses the HLG value in the SPS VUI parameters. See below frame snapshot captured in stream-eye:

This design supports the following video interfaces:
Sources:

  • SDI-Rx capture pipeline implemented in the PL.

  • File source (SD card, USB storage, SATA hard disk).

  • Stream-In from network or internet.

Sinks:

  • SDI-Tx display pipeline implemented in the PL.

VCU Codec:

  • Video Encode/Decode capability using VCU hard block in PL 

    • AVC/HEVC encoding.

    • Encoder/decoder parameter configuration.

Streaming Interfaces:

  • 1G Ethernet PS GEM 

Video format:

  • XV20

Audio Configuration:

  • Codec: Opus

  • Format: S24_32LE

  • Channels: 2, 8

  • Sampling rate: 48 kHz

Supported Resolution

The table below provides the supported resolution from GUI and command-line app in this design.

Resolution

GUI

Command Line

Single Stream

Single Stream

Multi Stream

4Kp60/59.94

X

√

X

4Kp30/29.97

X

√

X

1080p60/59.94

X

√

X

√ - Supported
x – Not supported

The below table gives information about the features supported in this design. 

Pipeline

Input Source

Output Type

ALSA Srivers

Resolution

Audio Codec Type

Audio Configuration

Video Codec
Type

 Deliverables

Pipeline

Input Source

Output Type

ALSA Srivers

Resolution

Audio Codec Type

Audio Configuration

Video Codec
Type

 Deliverables

 

 

Record/Stream-Out pipeline

 

 

 

SDI-Rx

 

 

File Sink/ Stream-Out

 

 

 

SDI-Rx ALSA drivers

 

 


4K/1080p

 

 Opus

2 channel @ 48 kHz

 

 

 

HEVC/AVC

SDI-Rx Audio encode with soft codec and video with VCU and store it in a container format.

 Vorbis

8* channel @ 48 kHz

SDI-Rx Audio encode with soft codec and video with VCU and store it in a container format.

 

 

Playback pipeline

 

File Source/ Stream-In

 

 

SDI-Tx

 

 

SDI-Tx ALSA drivers

 

 

 

4K/1080p

Opus

2 channel @ 48 kHz

 

 

 

HEVC/AVC

Playback of the local-file/stream-in with video decoded using VCU and Audio using GStreamer soft codec.

Vorbis

8* channel @ 48 kHz

Playback of the local-file/stream-in with video decoded using VCU and Audio using GStreamer soft codec.

Capture → Display

SDI-Rx

SDI-Tx

SDI-Rx ALSA drivers and SDI-Tx ALSA drivers

4K/1080p

NA

2 channel @ 48 kHz OR

8* channel @ 48 kHz

HEVC/AVC

SDI-Rx Audio /Video pass to SDI-Tx without VCU/Audio-Codec.

Capture → Encode → Decode → Display

 SDI-Rx

 SDI-Tx

 SDI-Rx ALSA drivers and SDI-Tx ALSA drivers

 4K/1080p

 NA

2 channel @ 48 kHz

OR
8* channel @ 48 kHz

 HEVC/AVC

SDI-Rx raw audio and video with VCU encoder and decode to achieve AV sync.

* The 8-channels audio functionality is validated with Phabrix Qx 12G SDI Analyzer/Generator.

The below figure shows the HLG SDI Video Capture and HLG SDI Display with Audio design hardware block diagram.

The below figure shows the HLG SDI Video Capture and HLG SDI Display with Audio design software block diagram.

1.1 Board Setup

Refer below link for board setup

1.2 Run Flow

The TRD package is released with the source code, Vivado project, Petalinux BSP, and SD card image that enables the user to run the demonstration. It also includes the binaries necessary to configure and boot the ZCU106 board. Prior to running the steps mentioned in this wiki page, download the TRD package and extract its contents to a directory referred to as TRD_HOME which is the home directory.

Refer below link to download all TRD contents.

TRD package contents specific to HLG SDI Video Capture and HLG SDI Display with Audio design are placed in the following directory structure. The user needs to copy all the files from the $TRD_HOME/images/vcu_sdi_xv20 to FAT32 formatted SD card directory.

rdf0428-zcu106-vcu-trd-2021-2/ ├── apu │   └── vcu_petalinux_bsp ├── images │   ├── vcu_audio │   ├── vcu_llp2_hdmi_nv12 │   ├── vcu_llp2_plddr_hdmi │   ├── vcu_llp2_sdi_xv20 │   ├── vcu_multistream_nv12 │   ├── vcu_pcie │   ├── vcu_plddrv1_hdr10_hdmi │   ├── vcu_plddrv2_hdr10_hdmi │   └── vcu_sdi_xv20 ├── pcie_host_package │   ├── COPYING │   ├── include │   ├── LICENSE │   ├── readme.txt │   ├── RELEASE │   ├── tests │   ├── tools │   └── xdma ├── pl │   ├── constrs │   ├── designs │   ├── prebuild │   ├── README.md │   └── srcs └── README.txt └── zcu106_vcu_trd_sources_and_licenses.tar.gz 22 directories, 7 files

TRD package contents specific to HLG SDI Video Capture and HLG SDI Display with Audio are placed in the following directory structure.

rdf0428-zcu106-vcu-trd-2021-2 ├── apu │   └── vcu_petalinux_bsp │   └── xilinx-vcu-zcu106-v2021.2-final.bsp ├─images │   └── vcu_sdi_xv20 │   ├── autostart.sh │   ├── BOOT.BIN │ ├── boot.scr │   ├── config │   ├── Image │   ├── rootfs.cpio.gz.u-boot │   ├── system.dtb │   └── vcu ├── pcie_host_package ├── pl │   ├── constrs │   ├── designs │ │ ├── zcu106_picxo_plddr_sdi_8ch │   ├── prebuild │   │   ├── zcu106_picxo_plddr_sdi_8ch │   ├── README.md │   └── srcs │   ├── hdl │   └── ip └── README.txt └── zcu106_vcu_trd_sources_and_licenses.tar.gz

Configuration files (input.cfg) for various Resolutions are placed in the following directory structure in /media/card.

config ├── 1080p60 │ ├── Display │ ├── Record │ ├── Stream-in │ └── Stream-out ├── 4kp30 │ ├── Display │ ├── Record │ ├── Stream-in │ └── Stream-out └── 4kp60 │ ├── Display │ ├── Record │ ├── Stream-in │ └── Stream-out └── input.cfg

1.2.1 GStreamer Application (vcu_gst_app)

The vcu_gst_app is a command-line multi-threaded Linux application. The command-line application requires an input configuration file (input.cfg) to be provided in the plain text.

Execution of the application is shown below:

Example:

4kp60 HEVC_HIGH Display Pipeline execution

4kp60 HEVC_HIGH Record Pipeline execution

4kp60 HEVC_HIGH Stream-out Pipeline execution

4kp60 HEVC_HIGH Stream-in Pipeline execution

Make sure SDI-Rx should be configured to 4kp60 mode.

To measure the latency of the pipeline, run the below command. The latency data is huge, so dump it to a file.

Refer below link for detailed run flow steps

1.3 Build Flow

Refer below link for build flow

2 Other Information

2.1 Known Issues

2.2 Limitations

2.3 Optimum VCU Encoder parameters for use-cases:

Video streaming:

  • Video streaming use-case requires a very stable bitrate graph for all pictures

  • It is good to avoid periodic large Intra pictures during the encoding session

  • Low-latency rate control (hardware RC) is the preferred control-rate for video streaming, it tries to maintain equal amount frame sizes for all pictures.

  • Good to avoid periodic Intra frames instead use low-delay-p (IPPPPP…)

  • VBR is not a preferred mode of streaming

Performance: AVC Encoder settings:

  • It is preferred to use 8 or higher slices for better AVC encoder performance

  • AVC standard does not support Tile mode processing which results in the processing of MB rows sequentially for entropy coding