Running the VCU ROI Demo on Certified Ubuntu 20.04 LTS for Xilinx Devices

Owned by Terry O'Neal
Last updated: Sept 03, 2021
5 min read

The pages provides detailed instructions for running the VCU Region of Intrest (ROI) Demo on the ZCU104 and ZCU106 boards running Certified Ubuntu 20.04 LTS for Xilinx Devices.

Table of Contents

Introduction

The primary goal of the VCU ROI design is to demonstrate the use of the Xilinx Deep learning Processor Unit (DPU) block for extracting the Region of Interest (ROI) data from input video frames.  This information is then used to perform ROI-based encoding using the Video Codec Unit (VCU) encoder hard block present in Zynq UltraScale+ EV devices.

To run the VCU ROI TRD demo you'll need to connect an HDMI input source to the  bottom HDMI connector on the board, and an HDMI Monitor to the top HDMI connector.  The 2020.2 VCU HDMI ROI TRD wiki page has more information about the VCU ROI TRD and setting up the board. 

The following figure outlines the “Serial Pipeline” usage model with face detection based on ROI detection on ZCU106. This use case is the foundation of this section. 

 

Running the Demo

There are two ways to run  the VCU ROI TRD Demo:

  • Using discrete commands

  • Using the xlnx-vcu-roi-trd snap from the Xilinx Snap Store on Snapcraft.io. (coming soon!)

Using Discrete Commands

This demonstration assumes that both the HDMI source and HDMI monitor run at 1080p30 (1920x1080 @ 30fps).

This demo can also run at up to 4Kp30 on the ZCU106. 

This demo only supports 1080p30 on the ZCU104.

Before getting started, be sure to do the following:

  1. Set the HDMI output resolution

  2. Launch the gstreamer pipeline

Depending on whether or not you have a USB camera plugged into your system, /dev/media0 in the following steps may need to change to /dev/media1. Use the command media-ctl -p -d /dev/media[X] to determine which device to use for HDMI Rx. 

Setting the HDMI Output Resolution

It is important to set up the video pipeline with a resolution and refresh rate that matches something your monitor is able to support. In order to find out which resolutions your monitor can support, use the modetest command with the syntax below. It is important to use the device a00c0000.v_mix as this corresponds to the video mixer IP.

sudo modetest -cD a00c0000.v_mix | head -n 32

Sample output from this command appears below. Note that in Connector #0, the supported resolution is 1920x1080@60.00Hz. The output of your monitor’s connector section will likely be different.

From the command line, issue the following commands to set the HDMI Monitor to 1080p30 in the AR24 color space. The AR24 color space is a good default to assume. Other common alternatives are BG24 and NV12.

sudo modetest -D a00c0000.v_mix -s 40:1920x1080-30@AR24

The -s flag actively sets the mode with the syntax -s <connector_id>[,<connector_id>][@<crtc_id>]:[#<mode index>]<mode>[-<vrefresh>][@<format>]. In our example, 40: is the connector ID, 1920x1080-30 is the resolution and refresh rate, and @AR24 is the color space mode. For more details, consult the output of modetest --help.

If you are having trouble finding a valid resolution and color space, re-run the modetest command without piping the output to the head command in order to get the full output, including a list of supported color spaces.

Once you have found a valid resolution and color space, you should see the default video test pattern displayed on the screen after issuing the modetest command.

Next, set the display to initialize the display and wait for further input:

sudo modetest -D a00c0000.v_mix -s 40:1920x1080-30@AR24 -w 36:"alpha":0 &

The -w 36:”alpha”:0 set the alpha layer to 0 on object ID 36. This can be seen in the full output of the modetest command:

The ampersand & is a standard Linux operator which makes the modetest command run in the background without exiting (eg, it waits).

Set up the HDMI Rx Pipeline

Configure the VPSS IP color space converter and scaler to support a 1080p input. 

This step is not necessary if using the mediasrcbin plugin below, it is only added for reference. 

Run gstreamer

mediasrcbin - The following gstreamer command line uses the mediasrcbin plugin.  This mediasrcbin plugin is Xilinx-specific plugin which is an element on top of v4l2src. It parses and configures the media graph of a media device automatically.

v4l2src -  The following gstreamer command line uses the v4l2src plugin. It requires manual commands to setup the VPSS as shown above. 

Troubleshooting

Error

Reason

Error

Reason

ERROR: from element /GstPipeline:pipeline0/GstMediaSrcBin:mediasrcbin0/GstV4l2Src:v4l2src0: Internal data stream error.

You may be using the wrong /dev/mediaX interface.  Use media-ctl -p -d /dev/mediaX to determine which one maps to the video pipeline you want to use. 

[2021-06-09 19:16:20.768828622] [omx_core.cpp:181] [OMX_GetHandle] Couldnt allocate dma allocator (tried using /dev/allegroDecodeIP)

You need to run the gstreamer command with sudo.

GET_DMA_FD: Cannot allocate memory
[2021-06-09 19:14:17.541401720] [module_dec.cpp:467] [AllocateDMA] No more memory
[2021-06-09 19:14:17.541861042] [omx_component_dec.cpp:261] [AllocateBuffer] OMX_ErrorInsufficientResources

This is usually due to running the ZCU104 VCU in 4K mode, (gstreamer command line has width=3840,height=1080). This mode is not supported on the ZCU104.

Using the xlnx-vcu-roi-trd Snap

Coming Soon!

Demo Operation

Once the gstreamer command or snap starts successfully the output of the HDMI input source is shown on the HDMI monitor. Any faces detected in the input stream will be marked by a red bounding box. If you look closely, you will see that the area inside the bounding box is clearer and includes less compression artifacts than the rest of the image. 

image credit: public domain from RODNAE Productions via http://pexels.com

 

© Copyright 2019 - 2022 Xilinx Inc. Privacy Policy