Zynq UltraScale+ MPSoC VCU TRD 2021.2 - YUV444 Video Capture and Display
This page provides all the information related to the VCU TRD YUV444 design.
This is a beta release and will be included in the future TRD releases as an additional design module. Also note that this page stands alone and does not rely on the common VCU TRD “Build and Run Flow” page.
Table of Contents
1 Overview
This module enables the capture of YUV444 8-bit and 10-bit video from HDMI-RX . The video can be displayed on HDMI-TX or DisplayPort and recorded on SD cards or USB/SATA drives. The module can stream-in or stream-out encoded data through an Ethernet interface. This module supports up to single-stream 4kp30 YU24 and XV30 format.
Xilinx Zynq UltraScale+ VCU HW does not support YUV444 encode/decode processing. Only YUV 4:0:0, 4:2:0 and 4:2:2 sub-sampling modes are supported. This feature enables encoding yuv444p (single planar) video frames using VCU, reading them as YUV4:0:0 of Width x 3*Height buffer. Similar at decoder side, VCU Decoder output Width x 3*Height YUV4:0:0 raw video frame but display will treat that buffers YUV444 planar buffer. This feature also includes enhancing FB_RD, FB_WR IP/Drivers and V4l2 and DRM frameworks to support YUV444 planar buffer format.
This design supports the following video interfaces:
Sources:
HDMI-RX capture pipeline implemented in the PL.
File source (SD card, USB storage, SATA hard disk).
Stream-In from network or internet.
Sinks:
HDMI-TX display pipeline implemented in the PL.
DP display pipeline implemented in the PL.
Stream-out to network or internet
VCU Codec:
Video Encode/Decode capability using VCU hard block in the PL
AVC/HEVC encoding
Encoder/decoder parameter configuration.
Video format:
YU24
XV30
Supported Resolutions:
The table below provides the supported resolutions from the command line app only in this design.
Resolution | Command Line | |
---|---|---|
Single Stream | Multi-stream | |
4kp60 | x | x |
4kp30 | √ | x |
1080p60 | √ | x |
√ - Supported
x – Not supported
The below table gives information about the features supported in this design.
Pipeline | Input source | Format | Output Type | Resolution | VCU codec |
---|---|---|---|---|---|
Capture → Display (Pass-through) | HDMI-RX | YU24/XV30 | HDMI-TX/DP | 4kp30 | None |
Record/ Stream-Out pipeline | HDMI-RX | YU24/XV30 | File Sink/ Stream-Out | 4kp30 | HEVC/AVC |
File/ Streaming Playback pipeline | File Source/ Stream-In | YU24/XV30 | HDMI-TX/DP | 4kp30 | HEVC/AVC |
Serial pipeline is not supported in this beta release.
The below figure shows the zcu106 YUV444 design hardware block diagram.
The below figure shows the zcu106 YUV444 design software block diagram.
2 Hardware and Software Tools
2.1 Hardware Tools
Required:
ZCU106 evaluation board (rev C/D/E/F/1.0) with power cable
YUV444 10-bit supported Monitor with DisplayPort/HDMI input supporting up to 3840x2160p30 resolution
Display Port cable (DP certified)
HDMI cable 2.0 certified
Class-10 SD card
HDMI Receiver - NVIDIA SHIELD / NVIDIA SHIELD Pro for YUV444 8-bit video capture
HDMI Receiver - MI-BOX for YUV444 10-bit video capture
Ethernet cable for streaming use-cases
Optional:
USB pen drive formatted with the FAT32 file system and hub
SATA drive formatted with the FAT32 file system, external power supply, and data cable
2.2 Software Tools
Required:
Linux host machine for all tool flow tutorials (see UG1144 for detailed OS requirements)
PetaLinux Tools version 2021.2 (see UG1144 for installation instructions)
Vivado Design suite version 2021.2
Git a distributed version control system
Serial terminal emulator, for example Tera Term
2.3 Download, Installation, and Licensing
The Vivado Design Suite User Guide explains how to download and install the Vivado Design Suite tools, which include the Vivado Integrated Design Environment (IDE), High-Level Synthesis tool, and System Generator for DSP. This guide also provides information about licensing and administering evaluation and full copies of Xilinx design tools and intellectual property (IP) products. The Vivado Design Suite can be downloaded from here.
LogiCORE IP Licensing:
The following IP cores require a license to build the design.
Video Timing Controller (VTC) - Included with Vivado - PG016
Video Mixer - Included with Vivado - PG243
Video PHY Controller - Included with Vivado - PG230
HDMI-RX/TX Subsystem - Purchase license (Hardware evaluation available) - PG235 & PG236
Video Processing Subsystem (VPSS) - Included with Vivado - PG231
XDMA - Included with Vivado - PG195
To obtain the LogiCORE IP license, please visit the respective IP product page and get the license.
(Xilinx Answer 44029) - Licensing - LogiCORE IP Core licensing questions
Note: Hardware Evaluation keys allow you to simulate and implement your design, run timing analysis and generate a time-limited bitstream to program a Xilinx FPGA. The core in the programmed device will function in hardware for anywhere from 2 to 8 hours, depending on the core.
3 Board Setup
The below section will provide the information on the ZCU106 board setup for running the TRD.
Connect the Micro USB cable into the ZCU106 Board Micro USB port J83, and the other end into an open USB port on the host PC. This cable is used for UART over USB communication.
Insert the SD card with the images copied into the SD card slot J100. Please find here how to prepare the SD card for a specific design.
Set the SW6 switches as shown in the below Figure. This configures the boot settings to boot from SD.
Connect 12V Power to the ZCU106 6-Pin Molex connector.
Connect one end of the Display Port (DP) cable to the board’s U129 connector and the other end to the DP port of the 4K monitor.
Connect one end of the HDMI cable to the board’s P7 stacked HDMI connector (lower port) and another end to the HDMI source in case of a HDMI design.
Connect one end of the HDMI cable to the board’s P7 stacked HDMI connector (upper port) and another end to the HDMI monitor in case of a HDMI design.
Note: It is highly recommended to disconnect DP or HDMI cable whenever not in used. Using both simultaneously might lead to unexpected behaviors.
For a USB storage device, connect the USB hub along with the mouse. (Optional)
For a SATA storage device, connect the SATA data cable to the SATA 3.0 port. (Optional)
Set up a terminal session between a PC COM port and the serial port on the evaluation board (See Determine which COM to use to access the USB serial port on the ZCU106 board for more details)
Copy YUV444 images into the SD card and insert the SD card on the board.
The below images will show how to connect interfaces on the ZCU106 board.
3.1 Determine which COM to use to access the USB serial port on the ZCU106 board
Make sure that the ZCU106 board is powered on and a micro USB cable is connected between the ZCU106 board and host PC. This ensures that the USB-to-serial bridge is enumerated by the PC host.
Open your computer's Control Panel by clicking on Start > Control Panel.
Note that the Start button is typically located in the lower-left corner of the screen. Occasionally, it is in the upper left corner.
Click Device Manager to open the Device Manager window. Note: You might be asked to confirm opening the Device Manager. If so, click YES.
Expand Ports (COM and LPT).
Locate the Silicon Labs Quad CP210x USB to UART Bridge: Interface 0 (COM#)
Note the COM Port number for further steps.
Close the Device Manager by clicking the red X in the upper right corner of the window.
Launch any Terminal application such as Tera Term to view the serial messages
Launch Tera Term and open the COM the port that is associated with Silicon Labs Quad CP210x USB to UART Bridge: Interface 0 of the USB-to-serial bridge.
Set the COM port to 115200 Baud rate, 8 bit data, none parity, 1 stop bit.
Power ON the board which has an SD card. Switch ON SW1 to power the ZCU106 board.
It boots Linux on board and it will take about a minute for Linux to boot.
4 Download the TRD
Download the VCU TRD 2021.2 YUV444 release package from below link:
5 Run Flow
The TRD package is released with the source code, Vivado project, PetaLinux BSP, and SD card image that enables the user to run the demonstration.
It also includes the binaries necessary to configure and boot the ZCU106 board. Prior to running the steps outlined in this wiki page, download the TRD package and extract its contents to a directory referred to as TRD_HOME
which is the home directory.
TRD package contents are placed in the following directory structure. The user needs to copy all of the files from the $TRD_HOME/images/vcu_yuv444
/ to the FAT32 formatted SD card directory.
└── rdf0428-zcu106-vcu-yuv444-EA-2021-2
├── apu
│ └── vcu_petalinux_bsp
│ └── xilinx-vcu-zcu106-v2021.2-final.bsp
├── images
│ └── vcu_yuv444
│ ├── autostart.sh
│ ├── BOOT.BIN
│ ├── bootfiles
│ │ ├── bl31.elf
│ │ ├── bootgen.bif
│ │ ├── pmufw.elf
│ │ ├── system.bit
│ │ ├── u-boot.elf
│ │ └── zynqmp_fsbl.elf
│ ├── boot.scr
│ ├── config
│ ├── Image
│ ├── rootfs.cpio.gz.u-boot
│ ├── system.dtb
│ └── vcu
├── pl
│ ├── constrs
│ │ ├── hdcp_keymngmt_blk.xdc
│ │ └── vcu_hdr.xdc
│ ├── designs
│ │ └── zcu106_HDR10_DCI4K_YUV444
│ ├── README.md
│ └── srcs
│ ├── hdl
│ └── ip
├── README.txt
└── zcu106_vcu_trd_sources_and_licenses.tar.gz
Scripts to run yuv444 use-cases for various resolutions are placed in following directory structure:
config
├── 1080p60
│ ├── Display
│ ├── Playback
│ ├── Record
│ ├── Stream-in
│ └── Stream-out
└── 4kp30
├── Display
├── Playback
├── Record
├── Stream-in
└── Stream-out
5.1 Prepare an SD Card
There are many options to format the SD Card in the windows tool, but always format with FAT32 option. Use the SD Card Formatter tool to format the SD card, https://www.sdcard.org/downloads/formatter_4/
Please note that the Windows format option cannot be used.
Copy all of the files from the
$TRD_HOME/images/<Path to Design Images>/
to a FAT32 formatted SD card directoryPower on the board. Make sure INIT_B, DONE and all power rail LEDs are lit green
After a successful boot, a shell prompt will appear as shown below.
root@zcu106_vcu_yuv444:~#
The SD card file system is mounted at /media/card
. Optional storage medium SATA and USB are mounted at /media/sata
and /media/usb
respectively.
5.2 GStreamer pipelines
YUV444 8-bit pipelines
4kp30 YU24 HEVC YUV444 8-bit Display HDMI Pass-through pipeline execution
4kp30 YU24 HEVC YUV444 8-bit Display DP Pass-through pipeline execution
4kp30 YU24 HEVC YUV444 8-bit record pipeline execution
4kp30 YU24 HEVC YUV444 8-bit HDMI playback pipeline execution
4kp30 YU24 HEVC YUV444 8-bit DP playback pipeline execution
4kp30 YU24 HEVC YUV444 8-bit stream-out pipeline execution
4kp30 YU24 HEVC YUV444 8-bit stream-in HDMI pipeline execution
4kp30 YU24 HEVC YUV444 8-bit stream-in DP pipeline execution
YUV444 10-bit pipelines
4kp30 XV30 HEVC YUV444 10-bit Display HDMI Pass-through pipeline execution
4kp30 XV30 HEVC YUV444 10-bit Display DP Pass-through pipeline execution
4kp30 XV30 HEVC YUV444 10-bit record pipeline execution
4kp30 XV30 HEVC YUV444 10-bit HDMI playback pipeline execution
4kp30 XV30 HEVC YUV444 10-bit DP playback pipeline execution
4kp30 XV30 HEVC YUV444 10-bit stream-out pipeline execution
4kp30 XV30 HEVC YUV444 10-bit stream-in HDMI pipeline execution
4kp30 XV30 HEVC YUV444 10-bit stream-in DP pipeline execution
5.3 Mount locations
The mount locations for various devices can be found in the below table.
The mount locations can vary. Users can use lsblk
or mount
to find the location of the mounted devices.
Below are some example mount points:
Device | Mount Location |
---|---|
SD Card | /media/card |
Sata Drive | /run/media/sda |
USB Drive | /media/usb |
RAM Disk | /run/media |
5.4 QoS Configuration
Check the Read QoS, Write QoS, Read Commands Issuing Capability and Write Commands Issuing Capability configuration of HP ports that interface the VCU with the PS DDR.
The AXI-QoS{3:0] lines behavior define the following three types of Traffic in Decimal format on the AXI Bus.
Traffic Class | Read QoS Value (default) | Write QoS Value (default) |
Best Effort (BE) | 0-3 | 0-7 |
Video (V) | 4-11 | 8-15 |
Low Latency (LL) | 12-15 | N/A |
Below is the reference QoS configuration script as per the VCU recommendation.
6 Build Flow
The following tutorials assume that the $TRD_HOME
environment variable is set as shown below.
6.1 Hardware Design
Refer to the Vivado Design Suite User Guide: Using the Vivado IDE (UG893), for information on setting up the Vivado environment.
On Linux:
Open a Linux terminal
Change the directory to
$TRD_HOME/pl
Run the following command in theVivado shell to create the Vivado IPI project and invoke the GUI.
After executing the script, the Vivado IPI block design comes up as shown in the below figure.
Click on “Generate Bitstream”.
The design is implemented and a pop-up window comes up saying “Open Implemented Design”. Click "OK".
After opening the implemented design, the window looks as shown in the below figure.
To generate the .xsa file:
Go to File > Export > Export Hardware
In the Export Hardware Platform window click Next.
In the next window, select Include bitstream and click Next.
The default XSA file name is <hardware design name_wrapper>. Choose the path where the XSA file has to be written.
for example, the XSA is created at $TRD_HOME/pl/build/zcu106_trd/zcu106_trd_wrapper.xsa
for a VCU TRD hardware design.
Click Finish for the XSA file to be generated.
6.2 VCU PetaLinux BSP
This tutorial shows how to build the Linux image and boot image using the PetaLinux build tool.
PetaLinux Installation: Refer to the PetaLinux Tools Documentation UG1144 for installation.
Create a PetaLinux project.
Configure the PetaLinux project.
Use the below command to configure the PetaLinux project for the YUV444 design.
If the Vivado project has been modified for the YUV444 design then use the below command to configure the BSP
Create a soft link of the design dtsi file to
system-user.dtsi
using the below command:
Build the PetaLinux project
Build SDK components to use it as sysroot for application development.
Create a boot image (BOOT.BIN) including FSBL, ATF, bitstream, and U-Boot.
Copy the generated boot image and Linux image to the SD card directory.
7 Other Information
7.1 Known Issues
For PetaLinux related known issues please see PetaLinux 2021.2 - Product Update Release Notes and Known Issues
For VCU related known issues please see AR# 76600: LogiCORE H.264/H.265 Video Codec Unit (VCU) - Release Notes and Known Issues and Xilinx Zynq UltraScale+ MPSoC Video Codec Unit.
7.2 Limitations
For PetaLinux related limitations please see: PetaLinux 2021.2 - Product Update Release Notes and Known Issues
For VCU related limitations please see AR# 76600: LogiCORE H.264/H.265 Video Codec Unit (VCU) - Release Notes and Known Issues, Xilinx Zynq UltraScale+ MPSoC Video Codec Unit and PG252.
8 HDMI-RX/TX Link-up and GStreamer Commands
This section covers configuration of HDMI-RX using the media-ctl
utility and HDMI-TX using the modetest
utility, along with demonstrating HDMI-RX/TX link-up issues and steps to switch HDMI-RX resolution. It also contains sample GStreamer HDMI video pipelines for Display, Record & Playback, Stream-in and Stream-out use-cases.
The HDMI source can be locked to any resolution. Run the below command for all media nodes to print the media device topology, where
mediaX
represents different media nodes. In the topology log, look for thev_hdmi_rx_ss
string to identify the HDMI input source media node.
To check the link status, resolution and video node of the HDMI input source, run the below media-ctl command, where ,
mediaX
indicates the media node for the HDMI input source.
When the HDMI source is connected to the 4Kp30 resolution and YUV444 8-bit color-space, it shows the following:
When the HDMI source is not connected, it shows the following:
Notes for gst-launch-1.0 commands:
The Video node for the HDMI-RX source can be checked using the media-ctl command. Run the below media-ctl command to check the video node for the HDMI-RX source, where
media0
indicates the media node for the HDMI input source.
Make sure the HDMI-RX media pipeline is configured for 4kp60 resolution and source/sink has the same color format for connected nodes. For XV20 format, run the below
media-ctl
commands to set the resolution and format of the HDMI scaler node wheremedia0
indicates the media node for HDMI input source.
If the HDMI Input Source is NVIDIA SHIELD and display format for the NVIDIA SHIELD is set to YUV444 8-bit format, run the below commands to set the v_proc_ss input and output format to YUV444 8-bit.
If the HDMI Input Source is NVIDIA SHIELD and display format for the NVIDIA SHIELD is set to YUV422 10-bit format, run the below commands to set the v_proc_ss input to YUV422 10-bit and output format to YUV444 10-bit.
Change the resolution of the HDMI Input Source from 1080p60 to 4kp30 by following the below steps.
Set the HDMI source resolution to 4kp60 (Homepage → Settings → Display & Sound → Resolution → change to 4kp30).
Save the configuration for the change to take effect.
Verify the desired HDMI Input Source Resolution (4kp30) by following the above steps.
To run HDMI use-cases, use below modetest commands:
For YUV444 8-bit
For YUV444 10-bit
To run DP use-cases, use below modetest commands:
For YUV444 8-bit
For YUV444 10-bit
The table below lists the parameters of the pixel format.
| Pixel Format | GStreamer Format | Media Bus Format |
---|---|---|---|
YUV444 8-bit | YU24 | Y444 | VUY8_1X24 |