Zynq UltraScale+ MPSoC ZCU104 VCU HDMI ROI 2020.2
This page provides all the information related to VCU HDMI ROI designs for ZCU104.
This design is based on the ZCU106 VCU HDMI ROI TRD which can be found at the following link
The two designs are functionally equivalent, please refer to the page linked above for details related to the system architecture, gstreamer plugin info, DPU configuration and usage, etc. The sections below provide direct links to the relevant information.
Table of Contents
1 Overview
The primary goal of this VCU ROI design is to demonstrate the use of DPU (Deep learning Processor Unit) block for extracting the ROI (Region of Interest) from input video frames and to use this information to perform ROI based encoding using VCU (Video Codec Unit) encoder hard block present in Zynq UltraScale+ EV devices.
For detailed overview please refer to this link.
1.1 System Architecture
For system architecture please refer to this link. System architecture diagram shown in above refer link is applicable for ZCU104 board.
1.2 Hardware Architecture
This section gives a detailed description of the blocks used in the hardware design.
1.3 VCU ROI Software
1.3.1 GStreamer Pipeline
The GStreamer plugin demonstrates the DPU capabilities with Xilinx VCU encoder’s ROI(Region of Interest) feature. The plugin will detect ROI (i.e. face co-ordinates) from input frames using DPU IP and pass the detected ROI information to the Xilinx VCU encoder.
1.3.2 DPU(Deep Learning Processor Unit)
DPU is a programmable engine optimized for deep neural networks. It is a group of parameterizable IP cores pre-implemented on the hardware with no place and route required. The DPU is released with the Vitis AI specialized instruction set, allowing efficient implementation of many deep learning networks.
1.4 Software Tools and System Requirements
Hardware
Required:
ZCU104 evaluation board rev 1.0 with power cable
Monitor with HDMI input supporting 3840x2160 resolution or 1920x1080 resolution
HDMI Receiver - NVIDIA SHIELD Pro for 4Kp60
Two HDMI cables 2.0 certified
Class-10 Micro-SD card (ZCU104)
Ethernet cable
Optional:
USB pen drive formatted with the FAT32 file system and hub
SATA drive formatted with the FAT32 file system, external power supply, and data cable
Software Tools
Required:
Linux host machine for all tool flow tutorials (see UG1144 for detailed OS requirements)
PetaLinux Tools version 2020.2 (see UG1144 for installation instructions)
Git a distributed version control system
Serial terminal emulator e.g. teraterm
Download, Installation, and Licensing of Vivado Design Suite 2020.2
The Vivado Design Suite User Guide explains how to download and install the Vivado® Design Suite tools, which include the Vivado Integrated Design Environment (IDE), High-Level Synthesis tool, and System Generator for DSP. This guide also provides information about licensing and administering evaluation and full copies of Xilinx design tools and intellectual property (IP) products. The Vivado Design Suite can be downloaded from here.
LogiCORE IP Licensing
The following IP cores require a license to build the design.
Video Mixer- Included with Vivado - PG243
Video PHY Controller - Included with Vivado - PG230
HDMI-Rx/Tx Subsystem - Purchase license (Hardware evaluation available) - PG235 & PG236
Video Processing Subsystem (VPSS) - Included with Vivado - PG231
To obtain the LogiCORE IP license, please visit the respective IP product page and get the license.
AR# 44029 - Licensing - LogiCORE IP Core licensing questions
Compatibility
The reference design has been tested successfully with the following user-supplied components.
HDMI Monitor:
Make/Model | Resolutions |
LG 24UD58 | 3840 x 2160 @ 30Hz |
Samsung LU28E90DS/XL | 3840 x 2160 @ 30Hz |
Cable:
HDMI 2.0 compatible cable
1.5 Board Setup
The below section will provide the information on the ZCU104 boards setup for running ROI design.
ZCU104 - Connect the Micro USB cable into the ZCU104 Board Micro USB port J164, and the other end into an open USB port on the host PC. This cable is used for UART over USB communication.
Insert the SD card with the images copied into the SD card slot J100. Please find here how to prepare the SD card for a specific design.
Set the SW6 switches as shown in the below Figure. This configures the boot settings to boot from SD.
Connect 12V Power to the 6-Pin Molex connector on the board
Connect one end of HDMI cable to the board’s P7 stacked HDMI connector (lower port) and another end to HDMI source
Connect one end of HDMI cable to the board’s P7 stacked HDMI connector (upper port) and another end to the HDMI monitor
For a USB storage device, connect the USB hub along with the mouse. (Optional)
For SATA storage device, connect SATA data cable to SATA 3.0 port. (Optional)
Set up a terminal session between a PC COM port and the serial port on the evaluation board (See the Determine which COM to use to access the USB serial port on the ZCU104 board for more details).
The below images will show how to connect interfaces on the ZCU104 board
ZCU104 Board
1.6 Run Flow
The VCU ROI packages released with the source code, Vivado project, Petalinux BSP, and SD card image that enables the user to run the demonstration. It also includes the binaries necessary to configure and boot the ZCU104 board. Prior to running the steps mentioned in this wiki page, download the desired VCU ROI package and extract its contents to a directory referred to as TRD_HOME
which is the home directory.
For ZCU104 Board:
Refer below link to download the VCU HDMI ROI package
TRD package contents are placed in the following directory structure for ZCU104
rdf0428-zcu104-vcu-hdmi-roi-2020-2
├── apu
│ └── vcu_petalinux_bsp
│ └── xilinx-vcu-roi-zcu104-v2020.2-final.bsp
├── dpu
│ ├── 0001-Added-ZCU104-configuration-to-support-DPU-in-ZCU104.patch
│ ├── dpu_conf.vh
│ └── vitis_platform
│ └── zcu104_dpu
├── image
│ ├── bootfiles
│ │ ├── bl31.elf
│ │ ├── linux.bif
│ │ ├── pmufw.elf
│ │ ├── system.bit
│ │ ├── system.dtb
│ │ ├── u-boot.elf
│ │ └── zynqmp_fsbl.elf
│ ├── README.txt
│ ├── sd_card
│ │ ├── boot
│ │ └── root
│ └── sd_card.img
└── pl
├── constrs
│ ├── roi_async.xdc
│ └── vcu_zcu104_roi.xdc
├── designs
│ └── zcu104_ROI_HDMI
├── prebuild
│ └── zcu104_ROI_HDMI.xsa
├── README.md
└── srcs
├── hdl
└── ip
1.6.1 Preparing the SD card
There are three ways to prepare the SD card for booting. Each method is detailed below.
Using ready to test image
Flash SD Card with
sd_card.img
using Etcher or Win32DiskImagerBoot the board with Flashed SD Card
Using Pre-built images
To Create SD Card with two partitions: Boot(FAT32+Bootable) and Root(EXT4) Refer this Link.
Copy
boot
content fromrdf0428-zcu104-vcu-hdmi-roi-2020-2/image/sd_card/boot
to Boot partition in SD CardExtract
rootfs.ext4
fromrdf0428-zcu104-vcu-hdmi-roi-2020-2/image/sd_card/root
to Root partition in SD Card usingBoot the board with Flashed SD Card
Use the Output of the Build Flow
To Create SD Card with two partitions: Boot(FAT32+Bootable) and Root(EXT4) Refer this Link.
For Build Flow refer this steps and copy mentioned generated dpu build images
bd.hwh BOOT.BIN boot.scr dpu.xclbin Image system.dtb
into BOOT partition of the SD card and extract generatedrootfs.ext4
into ROOT partition of SD CardCopy the mentioned
boot
contentconfig, vitis, start.sh, setup.sh
fromrdf0428-zcu104-vcu-hdmi-roi-2020-2/image/sd_card/boot/
directory to Boot partition in SD CardBoot the board with Flashed SD Card
1.6.2 GStreamer Pipelines using mediasrcbin plugin
This section covers the GSreamer pipelines using mediasrcbin plugin for serial and streaming ROI use-cases. This mediasrcbin plugin is Xilinx specific plugin which is a bin element on top of v4l2src. It parses and configures the media graph of a media device automatically.
Before Running Gstremer pipelines run below modetest commands to set 4Kp30 video mixer configurations for HDMI-Tx.
First modetest command is used to set resolution of CRTC and alpha property for primary plane
Second modetest command is used to get the blue screen once the alpha property for primary plane is set to
modetest -D a00c0000.v_mix -s 40:3840x2160-30@AR24 -w 36:"alpha":0 &
modetest -D a00c0000.v_mix -s 40:3840x2160-30@AR24
Serial: Capture → (ROI) → Encode → Decode → Display
Run the following
gst-launch-1.0
command for serial pipeline (capture → roi_plugin → encode → decode → display)Serial Pipeline
gst-launch-1.0 mediasrcbin media-device=/dev/media0 v4l2src0::io-mode=4 ! video/x-raw, width=3840, height=2160, format=NV12, framerate=30/1 ! xlnxroivideo1detect capture-io-mode=4 output-io-mode=5 relative-qp=-21 ! omxh265enc gop-mode=basic gop-length=60 b-frames=0 target-bitrate=1500 num-slices=8 control-rate=constant prefetch-buffer=true low-bandwidth=false filler-data=true cpb-size=1000 initial-delay=500 qp-mode=roi ! video/x-h265, profile=main, alignment=au ! queue ! omxh265dec internal-entropy-buffers=5 low-latency=0 split-input=true ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false 'video-sink=kmssink bus-id=a00c0000.v_mix hold-extra-sample=1 show-preroll=false sync=true' sync=true -v
Stream-out ( Server ): Capture → (ROI) → Encode → Stream-out
Set IP address for server:
Run the following
gst-launch-1.0
command for stream-out pipelineStream-out Pipeline
Stream-in ( Client ): Stream-in→ Decode → Display
Set IP address for the client:
Run the following
gst-launch-1.0
command for stream-in pipeline where5004
is port numberStream-in Pipeline
1.6.3 GStreamer Pipelines using v4l2src plugin
This section covers the GSreamer pipelines using v4l2src plugin for serial and streaming ROI use-cases. Before Running Gstremer pipelines run modetest command for HDMI-Tx and media-ctl command for HDMI-Rx as mentioned below.
Before Running Gstremer pipelines run below modetest commands to set 4Kp30 video mixer configurations for HDMI-Tx.
First modetest command is used to set resolution of CRTC and alpha property for primary plane
Second modetest command is used to get the blue screen once the alpha property for prima