This page provides all the information related to VCU HDMI ROI designs for ZCU104.

note
  • This design is based on the ZCU106 VCU HDMI ROI TRD which can be found at the following link

  • The two designs are functionally equivalent, please refer to the page linked above for details related to the system architecture, gstreamer plugin info, DPU configuration and usage, etc.  The sections below provide direct links to the relevant information.

  • This design is based on the ZCU106 VCU HDMI ROI TRD which can be found at the following link

  • The two designs are functionally equivalent, please refer to the page linked above for details related to the system architecture, gstreamer plugin info, DPU configuration and usage, etc.  The sections below provide direct links to the relevant information.

Table of Contents

1 Overview

The primary goal of this VCU ROI design is to demonstrate the use of DPU (Deep learning Processor Unit) block for extracting the ROI (Region of Interest) from input video frames and to use this information to perform ROI based encoding using VCU (Video Codec Unit) encoder hard block present in Zynq UltraScale+ EV devices.

note
  • For detailed overview please refer to this link.

  • For detailed overview please refer to this link.

1.1 System Architecture

note
  • For system architecture please refer to this link. System architecture diagram shown in above refer link is applicable for ZCU104 board.

  • For system architecture please refer to this link. System architecture diagram shown in above refer link is applicable for ZCU104 board.

1.2 Hardware Architecture

This section gives a detailed description of the blocks used in the hardware design. 

note
  • For detailed hardware architecture please refer to this link.

  • For detailed hardware architecture please refer to this link.

1.3 VCU ROI Software

1.3.1 GStreamer Pipeline

The GStreamer plugin demonstrates the DPU capabilities with Xilinx VCU encoder’s ROI(Region of Interest) feature. The plugin will detect ROI (i.e. face co-ordinates) from input frames using DPU IP and pass the detected ROI information to the Xilinx VCU encoder.

note
  • For detailed VCU ROI software please refer to this link.

  • For detailed VCU ROI software please refer to this link.

1.3.2 DPU(Deep Learning Processor Unit)

DPU is a programmable engine optimized for deep neural networks. It is a group of parameterizable IP cores pre-implemented on the hardware with no place and route required. The DPU is released with the Vitis AI specialized instruction set, allowing efficient implementation of many deep learning networks.

note
  • For detailed Deep Learning Processor Unit please refer to this link.

  • For detailed Deep Learning Processor Unit please refer to this link.

1.4 Software Tools and System Requirements

Hardware

Required:

Optional:

Software Tools

Required:

Download, Installation, and Licensing of Vivado Design Suite 2020.2

The Vivado Design Suite User Guide explains how to download and install the Vivado® Design Suite tools, which include the Vivado Integrated Design Environment (IDE), High-Level Synthesis tool, and System Generator for DSP. This guide also provides information about licensing and administering evaluation and full copies of Xilinx design tools and intellectual property (IP) products. The Vivado Design Suite can be downloaded from here.

LogiCORE IP Licensing 

The following IP cores require a license to build the design.

To obtain the LogiCORE IP license, please visit the respective IP product page and get the license.

Compatibility

The reference design has been tested successfully with the following user-supplied components.

HDMI Monitor:

Make/Model

Resolutions

LG 24UD58

3840 x 2160 @ 30Hz

Samsung LU28E90DS/XL

3840 x 2160 @ 30Hz

Cable:

1.5 Board Setup

The below section will provide the information on the ZCU104 boards setup for running ROI design.

  1. ZCU104 - Connect the Micro USB cable into the ZCU104 Board Micro USB port J164, and the other end into an open USB port on the host PC. This cable is used for UART over USB communication.

  2. Insert the SD card with the images copied into the SD card slot J100. Please find here how to prepare the SD card for a specific design.

  3. Set the SW6 switches as shown in the below Figure. This configures the boot settings to boot from SD.

  4. Connect 12V Power to the 6-Pin Molex connector on the board

  5. Connect one end of HDMI cable to the board’s P7 stacked HDMI connector (lower port) and another end to HDMI source

  6. Connect one end of HDMI cable to the board’s P7 stacked HDMI connector (upper port) and another end to the HDMI monitor

  7. For a USB storage device, connect the USB hub along with the mouse. (Optional)

  8. For SATA storage device, connect SATA data cable to SATA 3.0 port. (Optional)

  9. Set up a terminal session between a PC COM port and the serial port on the evaluation board (See the Determine which COM to use to access the USB serial port on the ZCU104 board for more details).

  10. The below images will show how to connect interfaces on the ZCU104 board

ZCU104 Board

1.6 Run Flow

The VCU ROI packages released with the source code, Vivado project, Petalinux BSP, and SD card image that enables the user to run the demonstration. It also includes the binaries necessary to configure and boot the ZCU104 board. Prior to running the steps mentioned in this wiki page, download the desired VCU ROI package and extract its contents to a directory referred to as TRD_HOME which is the home directory.

For ZCU104 Board:

TRD package contents are placed in the following directory structure for ZCU104

rdf0428-zcu104-vcu-hdmi-roi-2020-2
├── apu
│   └── vcu_petalinux_bsp
│       └── xilinx-vcu-roi-zcu104-v2020.2-final.bsp
├── dpu
│   ├── 0001-Added-ZCU104-configuration-to-support-DPU-in-ZCU104.patch
│   ├── dpu_conf.vh
│   └── vitis_platform
│       └── zcu104_dpu
├── image
│   ├── bootfiles
│   │   ├── bl31.elf
│   │   ├── linux.bif
│   │   ├── pmufw.elf
│   │   ├── system.bit
│   │   ├── system.dtb
│   │   ├── u-boot.elf
│   │   └── zynqmp_fsbl.elf
│   ├── README.txt
│   ├── sd_card
│   │   ├── boot
│   │   └── root
│   └── sd_card.img
└── pl
    ├── constrs
    │   ├── roi_async.xdc
    │   └── vcu_zcu104_roi.xdc
    ├── designs
    │   └── zcu104_ROI_HDMI
    ├── prebuild
    │   └── zcu104_ROI_HDMI.xsa
    ├── README.md
    └── srcs
        ├── hdl
        └── ip

1.6.1 Preparing the SD card

There are three ways to prepare the SD card for booting. Each method is detailed below. 

Using ready to test image

note
  • sd_card.img is available at rdf0428-zcu104-vcu-hdmi-roi-2020-2/image/sd_card.img

  • All the required Vitis packages are already installed in ready to test rdf0428-zcu104-vcu-hdmi-roi-2020-2/image/sd_card.img

  • sd_card.img is available at rdf0428-zcu104-vcu-hdmi-roi-2020-2/image/sd_card.img

  • All the required Vitis packages are already installed in ready to test rdf0428-zcu104-vcu-hdmi-roi-2020-2/image/sd_card.img

Using Pre-built images

Use the Output of the Build Flow

note

All the required Vitis AI packages are already available in rdf0428-zcu104-vcu-hdmi-roi-2020-2/image/sd_card/boot/vitis directory which needs to be installed using start.sh script after boot-up. Please wait till target setup complete and Vitis AI packages are installed.

All the required Vitis AI packages are already available in rdf0428-zcu104-vcu-hdmi-roi-2020-2/image/sd_card/boot/vitis directory which needs to be installed using start.sh script after boot-up. Please wait till target setup complete and Vitis AI packages are installed.

1.6.2 GStreamer Pipelines using mediasrcbin plugin

This section covers the GSreamer pipelines using mediasrcbin plugin for serial and streaming ROI use-cases. This mediasrcbin plugin is Xilinx specific plugin which is a bin element on top of v4l2src. It parses and configures the media graph of a media device automatically.

Before Running Gstremer pipelines run below modetest commands to set 4Kp30 video mixer configurations for HDMI-Tx. 

modetest -D a00c0000.v_mix -s 40:3840x2160-30@AR24 -w 36:"alpha":0 &
modetest -D a00c0000.v_mix -s 40:3840x2160-30@AR24

Serial: Capture → (ROI) → Encode → Decode → Display

Stream-out ( Server ): Capture → (ROI) → Encode → Stream-out

note
  • Here 192.168.25.89 is host/client IP address and 5004 is port no.

  • Make sure HDMI-Rx source should be configured to 4Kp30 mode

  • Here 192.168.25.89 is host/client IP address and 5004 is port no.

  • Make sure HDMI-Rx source should be configured to 4Kp30 mode

Stream-in ( Client ): Stream-in→ Decode → Display

note

For all 1080p30 commands replace width from 3840 to 1920 and height from 2160 to 1080

For all 1080p30 commands replace width from 3840 to 1920 and height from 2160 to 1080

1.6.3 GStreamer Pipelines using v4l2src plugin

This section covers the GSreamer pipelines using v4l2src plugin for serial and streaming ROI use-cases. Before Running Gstremer pipelines run modetest command for HDMI-Tx and media-ctl command for HDMI-Rx as mentioned below.

Before Running Gstremer pipelines run below modetest commands to set 4Kp30 video mixer configurations for HDMI-Tx. 

modetest -D a00c0000.v_mix -s 40:3840x2160-30@AR24 -w 36:"alpha":0 &
modetest -D a00c0000.v_mix -s 40:3840x2160-30@AR24

Run below media-ctl commands to set scaler configuration when HDMI Input Source is configured for 4Kp30 resolution and RBG888_1x24 color format.

media-ctl -d /dev/media0 -V "\"a0040000.v_proc_ss\":0 [fmt:RBG888_1X24/3840x2160 field:none]"
media-ctl -d /dev/media0 -V "\"a0040000.v_proc_ss\":1 [fmt:VYYUYY8_1X24/3840x2160 field:none]"
note
  • Make sure HDMI-Rx is configured for 4Kp resolution and RBG888_1x24 color format

  • v_proc_ss:0 should match the HDMI-Rx input source resolution and color format, and that the v_proc_ss:1 should match the Gstremer Pipeline resolution and color format

  • Make sure HDMI-Rx is configured for 4Kp resolution and RBG888_1x24 color format

  • v_proc_ss:0 should match the HDMI-Rx input source resolution and color format, and that the v_proc_ss:1 should match the Gstremer Pipeline resolution and color format

Serial: Capture → (ROI) → Encode → Decode → Display

Stream-out ( Server ): Capture → (ROI) → Encode → Stream-out

note
  • Here 192.168.25.89 is host/client IP address and 5004 is port no.

  • Make sure HDMI-Rx source should be configured to 4Kp30 mode

  • Here 192.168.25.89 is host/client IP address and 5004 is port no.

  • Make sure HDMI-Rx source should be configured to 4Kp30 mode

Stream-in ( Client ): Stream-in→ Decode → Display

note

For all 1080p30 commands replace width from 3840 to 1920 and height from 2160 to 1080

For all 1080p30 commands replace width from 3840 to 1920 and height from 2160 to 1080

1.7 Build Flow

note
  • For detailed Build flow please refer to this link.

  • For ZCU104 Board, replace zcu106 with zcu104 in above refer link.

  • For detailed Build flow please refer to this link.

  • For ZCU104 Board, replace zcu106 with zcu104 in above refer link.

3 Other Information

3.1 Known Issues

3.2 Limitations

4 Appendix A - HDMI-Rx Link-up

note
  • For Appendix A please refer this link

  • For Appendix A please refer this link