This page provides all the information related to VCU HDMI ROI designs for ZCU104.

This design is based on the ZCU106 VCU HDMI ROI TRD which can be found at the following link
- Zynq UltraScale+ MPSoC ZCU106 VCU HDMI ROI TRD 2020.2
The two designs are functionally equivalent, please refer to the page linked above for details related to the system architecture, gstreamer plugin info, DPU configuration and usage, etc. The sections below provide direct links to the relevant information.

1 1 Overview
- 1.1 1.1 System Architecture
- 1.2 1.2 Hardware Architecture
- 1.3 1.3 VCU ROI Software
  - 1.3.1 1.3.1 GStreamer Pipeline
  - 1.3.2 1.3.2 DPU(Deep Learning Processor Unit)
- 1.4 1.4 Software Tools and System Requirements
- 1.5 1.5 Board Setup
- 1.6 1.6 Run Flow
- 1.7 1.7 Build Flow
2 3 Other Information
- 2.1 3.1 Known Issues
- 2.2 3.2 Limitations
3 4 Appendix A - HDMI-Rx Link-up

1 Overview

The primary goal of this VCU ROI design is to demonstrate the use of DPU (Deep learning Processor Unit) block for extracting the ROI (Region of Interest) from input video frames and to use this information to perform ROI based encoding using VCU (Video Codec Unit) encoder hard block present in Zynq UltraScale+ EV devices.

For detailed overview please refer to this link.

1.1 System Architecture

For system architecture please refer to this link. System architecture diagram shown in above refer link is applicable for ZCU104 board.

1.2 Hardware Architecture

This section gives a detailed description of the blocks used in the hardware design.

1.3 VCU ROI Software

1.3.1 GStreamer Pipeline

The GStreamer plugin demonstrates the DPU capabilities with Xilinx VCU encoder’s ROI(Region of Interest) feature. The plugin will detect ROI (i.e. face co-ordinates) from input frames using DPU IP and pass the detected ROI information to the Xilinx VCU encoder.

1.3.2 DPU(Deep Learning Processor Unit)

DPU is a programmable engine optimized for deep neural networks. It is a group of parameterizable IP cores pre-implemented on the hardware with no place and route required. The DPU is released with the Vitis AI specialized instruction set, allowing efficient implementation of many deep learning networks.

1.4 Software Tools and System Requirements

Hardware

Required:

ZCU104 evaluation board rev 1.0 with power cable
Monitor with HDMI input supporting 3840x2160 resolution or 1920x1080 resolution
HDMI Receiver - NVIDIA SHIELD Pro for 4Kp60
Two HDMI cables 2.0 certified
Class-10 Micro-SD card (ZCU104)
Ethernet cable

Optional:

USB pen drive formatted with the FAT32 file system and hub
SATA drive formatted with the FAT32 file system, external power supply, and data cable

Software Tools

Required:

Linux host machine for all tool flow tutorials (see UG1144 for detailed OS requirements)
PetaLinux Tools version 2020.2 (see UG1144 for installation instructions)
Vivado Design suite version 2020.2
Vitis Core Development Kit 2020.2
Xilinx Runtime 2020.2
Git a distributed version control system
Serial terminal emulator e.g. teraterm

Download, Installation, and Licensing of Vivado Design Suite 2020.2

The Vivado Design Suite User Guide explains how to download and install the Vivado® Design Suite tools, which include the Vivado Integrated Design Environment (IDE), High-Level Synthesis tool, and System Generator for DSP. This guide also provides information about licensing and administering evaluation and full copies of Xilinx design tools and intellectual property (IP) products. The Vivado Design Suite can be downloaded from here.

LogiCORE IP Licensing

The following IP cores require a license to build the design.

Video Mixer- Included with Vivado - PG243
Video PHY Controller - Included with Vivado - PG230
HDMI-Rx/Tx Subsystem - Purchase license (Hardware evaluation available) - PG235 & PG236
Video Processing Subsystem (VPSS) - Included with Vivado - PG231

To obtain the LogiCORE IP license, please visit the respective IP product page and get the license.

AR# 44029 - Licensing - LogiCORE IP Core licensing questions
Xilinx Licensing FAQ
LogiCORE IP Project License Terms

Compatibility

The reference design has been tested successfully with the following user-supplied components.

HDMI Monitor:

Make/Model	Resolutions
LG 24UD58	3840 x 2160 @ 30Hz
Samsung LU28E90DS/XL	3840 x 2160 @ 30Hz

Cable:

HDMI 2.0 compatible cable

1.5 Board Setup

The below section will provide the information on the ZCU104 boards setup for running ROI design.

ZCU104 - Connect the Micro USB cable into the ZCU104 Board Micro USB port J164, and the other end into an open USB port on the host PC. This cable is used for UART over USB communication.
Insert the SD card with the images copied into the SD card slot J100. Please find here how to prepare the SD card for a specific design.
Set the SW6 switches as shown in the below Figure. This configures the boot settings to boot from SD.
Connect 12V Power to the 6-Pin Molex connector on the board
Connect one end of HDMI cable to the board’s P7 stacked HDMI connector (lower port) and another end to HDMI source
Connect one end of HDMI cable to the board’s P7 stacked HDMI connector (upper port) and another end to the HDMI monitor
For a USB storage device, connect the USB hub along with the mouse. (Optional)
For SATA storage device, connect SATA data cable to SATA 3.0 port. (Optional)
Set up a terminal session between a PC COM port and the serial port on the evaluation board (See the Determine which COM to use to access the USB serial port on the ZCU104 board for more details).
The below images will show how to connect interfaces on the ZCU104 board

ZCU104 Board

1.6 Run Flow

The VCU ROI packages released with the source code, Vivado project, Petalinux BSP, and SD card image that enables the user to run the demonstration. It also includes the binaries necessary to configure and boot the ZCU104 board. Prior to running the steps mentioned in this wiki page, download the desired VCU ROI package and extract its contents to a directory referred to as TRD_HOME which is the home directory.

For ZCU104 Board:

Refer below link to download the VCU HDMI ROI package
- Zynq UltraScale+ MPSoC VCU ROI 2020.2 Download zip

TRD package contents are placed in the following directory structure for ZCU104

rdf0428-zcu104-vcu-hdmi-roi-2020-2
├── apu
│   └── vcu_petalinux_bsp
│       └── xilinx-vcu-roi-zcu104-v2020.2-final.bsp
├── dpu
│   ├── 0001-Added-ZCU104-configuration-to-support-DPU-in-ZCU104.patch
│   ├── dpu_conf.vh
│   └── vitis_platform
│       └── zcu104_dpu
├── image
│   ├── bootfiles
│   │   ├── bl31.elf
│   │   ├── linux.bif
│   │   ├── pmufw.elf
│   │   ├── system.bit
│   │   ├── system.dtb
│   │   ├── u-boot.elf
│   │   └── zynqmp_fsbl.elf
│   ├── README.txt
│   ├── sd_card
│   │   ├── boot
│   │   └── root
│   └── sd_card.img
└── pl
    ├── constrs
    │   ├── roi_async.xdc
    │   └── vcu_zcu104_roi.xdc
    ├── designs
    │   └── zcu104_ROI_HDMI
    ├── prebuild
    │   └── zcu104_ROI_HDMI.xsa
    ├── README.md
    └── srcs
        ├── hdl
        └── ip

1.6.1 Preparing the SD card

There are three ways to prepare the SD card for booting. Each method is detailed below.

Using ready to test image

Flash SD Card with sd_card.img using Etcher or Win32DiskImager
Boot the board with Flashed SD Card

Using Pre-built images

To Create SD Card with two partitions: Boot(FAT32+Bootable) and Root(EXT4) Refer this Link.
Copy boot content fromrdf0428-zcu104-vcu-hdmi-roi-2020-2/image/sd_card/boot to Boot partition in SD Card
Extract rootfs.ext4 from rdf0428-zcu104-vcu-hdmi-roi-2020-2/image/sd_card/rootto Root partition in SD Card using
Boot the board with Flashed SD Card

Use the Output of the Build Flow

To Create SD Card with two partitions: Boot(FAT32+Bootable) and Root(EXT4) Refer this Link.
For Build Flow refer this steps and copy mentioned generated dpu build images bd.hwh BOOT.BIN boot.scr dpu.xclbin Image system.dtb into BOOT partition of the SD card and extract generated rootfs.ext4 into ROOT partition of SD Card
Copy the mentioned boot content config, vitis, start.sh, setup.sh from rdf0428-zcu104-vcu-hdmi-roi-2020-2/image/sd_card/boot/ directory to Boot partition in SD Card
Boot the board with Flashed SD Card

1.6.2 GStreamer Pipelines using mediasrcbin plugin

This section covers the GSreamer pipelines using mediasrcbin plugin for serial and streaming ROI use-cases. This mediasrcbin plugin is Xilinx specific plugin which is a bin element on top of v4l2src. It parses and configures the media graph of a media device automatically.

Before Running Gstremer pipelines run below modetest commands to set 4Kp30 video mixer configurations for HDMI-Tx.

First modetest command is used to set resolution of CRTC and alpha property for primary plane
Second modetest command is used to get the blue screen once the alpha property for primary plane is set to

modetest -D a00c0000.v_mix -s 40:3840x2160-30@AR24 -w 36:"alpha":0 &
modetest -D a00c0000.v_mix -s 40:3840x2160-30@AR24

Serial: Capture → (ROI) → Encode → Decode → Display

Run the following gst-launch-1.0 command for serial pipeline (capture → roi_plugin → encode → decode → display)
- Serial Pipeline
  gst-launch-1.0 mediasrcbin media-device=/dev/media0 v4l2src0::io-mode=4 ! video/x-raw, width=3840, height=2160, format=NV12, framerate=30/1 ! xlnxroivideo1detect capture-io-mode=4 output-io-mode=5 relative-qp=-21 ! omxh265enc gop-mode=basic gop-length=60 b-frames=0 target-bitrate=1500 num-slices=8 control-rate=constant prefetch-buffer=true low-bandwidth=false filler-data=true cpb-size=1000 initial-delay=500 qp-mode=roi ! video/x-h265, profile=main, alignment=au ! queue ! omxh265dec internal-entropy-buffers=5 low-latency=0 split-input=true ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false 'video-sink=kmssink bus-id=a00c0000.v_mix hold-extra-sample=1 show-preroll=false sync=true' sync=true -v

Stream-out ( Server ): Capture → (ROI) → Encode → Stream-out

Set IP address for server:
Run the following gst-launch-1.0 command for stream-out pipeline
- Stream-out Pipeline

Stream-in ( Client ): Stream-in→ Decode → Display

Set IP address for the client:
Run the following gst-launch-1.0 command for stream-in pipeline where 5004 is port number
- Stream-in Pipeline

1.6.3 GStreamer Pipelines using v4l2src plugin

This section covers the GSreamer pipelines using v4l2src plugin for serial and streaming ROI use-cases. Before Running Gstremer pipelines run modetest command for HDMI-Tx and media-ctl command for HDMI-Rx as mentioned below.