This page provides all the information related to VCU Multi-Stream ROI TRD using Avnet Quad Sensor design for ZCU106.

Table of Contents

1 Overview

The primary goal of this VCU Multi-Stream ROI TRD using Avnet Quad Sensor design is to demonstrate the use of Deep learning Processor Unit (DPU) block for extracting the Region of Interest (ROI) from input video frames and to use this information to perform ROI based encoding using Video Codec Unit (VCU) encoder hard block present in Zynq UltraScale+ EV devices. Video captures from the quad sensor connected through MIPI CSI-2 Rx which is implemented in the PL. The Avnet Multi-Camera FMC module is used to capture four video streams through a MIPI CSI-2 interface.

The design will serve as a platform to accelerate Deep Neural Network inference algorithms using DPU and demonstrate the ROI feature of VCU encoder. The design uses a Deep Convolutional Neural Network (CNN) named Densebox, running on DPU to extract ROI Information (e.g. ‘face’ in this case). The design will also use Vitis Video Analytics SDK (VVAS) framework to take leverage of its rich set of highly optimized and ready to use - Kernels and GStreamer plugins.

The Design will use Vivado IPI flow for building the hardware platform and Xilinx Yocto Petalinux flow for software design. It will use Xilinx IP and Software driver to demonstrate the capabilities of different components.

The Vitis platform will be created from the Vivado/PetaLinux build artifacts, and then using the Vitis acceleration flow will be used to insert the DPU into the platform to create the final bitstream.

The following figure shows streaming pipeline use-case with enhanced ROI + face detection model on ZCU106. For a detailed view of VVAS block, please refer to Section 1.3.3 - GStreamer Pipeline Flow.

Open

1.1 System Architecture

The following figure shows the block diagram of the ROI design

Open