Zynq UltraScale+ MPSoC VCU TRD 2021.1 - Multi Stream Audio Video Capture and Display

This page provides all the information related to Design Module 3 - VCU TRD Multi stream Audio-Video design.

Table of Contents

1 Overview

The primary goal of this Design is to demonstrate the capabilities of VCU hard block present in Zynq UltraScale+ EV devices with soft audio codec. The TRD will serve as a platform to tune the performance parameters of VCU and arrive at optimal configurations for encoder and decoder blocks with audio-video synchronization.

This design supports the following video interfaces:

Sources:

  • HDMI-Rx capture pipeline implemented in the PL

  • MIPI CSI-2 Rx capture pipeline implemented in the PL

  • File source (SD card, USB storage, SATA hard disk)

  • Stream-In from network or internet

Sinks:

  • DP-Tx display pipeline in the PS

  • HDMI-Tx display pipeline implemented in the PL

VCU Codec:

  • Video Encode/Decode capability using VCU hard block in PL 

    • AVC/HEVC encoding

    • Encoder/decoder parameter configuration

Streaming Interfaces:

  • 1G Ethernet PS GEM 

Video format:

  • NV12

Audio Configuration:

  • Codec: Opus

  • Format: S24_32LE

  • Channel: 2

  • Sampling rate: 48 kHz

  • Source: HDMI-Rx/ I2S-Rx

  • Render: HDMI-Tx/ I2S-Tx/ DP

Audio Deliverables:

Pipeline

Video-Input Source 

Audio Input Source

Video Output Type

Audio Output Type

ALSA Drivers

Resolution

Audio Codec Type

Audio Configuration

Video Codec Type

Deliverables

Record / Stream-Out pipeline

  1. HDMI-Rx

  2. MIPI-Rx

  1. HDMI-Rx

  2. I2S-Rx

File-Sink
Stream-Out

File-Sink
Stream-Out

HDMI-Rx ALSA drivers

4K / 1080p

Opus

2 channel @ 48 kHz

HEVC / AVC

HDMI-Rx Audio encode with soft codec and video with VCU and store it in a container format.

Playback pipeline

File Source/ Stream-In

File Source/ Stream-In

DP
HDMI –Tx

  1. HDMI-Tx

  2. I2S-Tx

  3. DP

HDMI-Tx ALSA drivers

4K / 1080p

Opus

2 channel @ 48 kHz

HEVC / AVC

Playback of the local-file / stream-in with video decoded using VCU and Audio using GStreamer soft codec.

Capture → Display

  1. HDMI-Rx

  2. MIPI-Rx

  1. HDMI-Rx

  2. I2S-Rx

DP
HDMI -Tx

  1. HDMI-Tx

  2. I2S-Tx

  3. DP

HDMI-Rx/Tx ALSA drivers

4K / 1080p

NA

2 channel @ 48 kHz

HEVC / AVC

HDMI-Rx Audio / Video pass to HDMI-Tx without VCU/Audio-Codec.

Capture → Encod → Decode → Display

  1. HDMI-Rx

  2. MIPI-Rx

  1. HDMI-Rx

  2. I2S-Rx

DP
HDMI -Tx

  1. HDMI-Tx

  2. I2S-Tx

  3. DP

HDMI-Rx/Tx ALSA drivers

4K / 1080p

NA

2 channel @ 48 kHz

HEVC / AVC

HDMI-Rx raw audio and video with VCU encoder and decode to achieve AV sync.

  • Supports 1-4Kp60 Single Stream with either HDMI-Rx or I2S-Rx as input Audio source + HDMI-Rx / MIPI Rx as input Video source; and HDMI-Tx / I2S-Tx as Output Audio Sink + HDMI-Tx / DP as Output Video sink pipeline

  • Supports 1-4Kp30 Single Stream with either HDMI-Rx or I2S-Rx as input Audio source + HDMI-Rx / MIPI Rx as input Video source; and HDMI-Tx / I2S-Tx as Output Audio Sink + HDMI-Tx / DP as Output Video sink pipeline

  • Supports 1-1080p60 Single Stream with either HDMI-Rx or I2S-Rx as input Audio source + HDMI-Rx / MIPI Rx as input Video source; and HDMI-Tx / I2S-Tx as Output Audio Sink + HDMI-Tx / DP as Output Video sink pipeline

  • Supports 2-4Kp30 multi-stream feature with HDMI-Rx and I2S-Rx as input Audio sources, with HDMI-Rx and MIPI Rx as an input Video sources;  and with HDMI-Tx and I2S-Tx as Output Audio Sink + HDMI-Tx as Output Video sink pipeline

  • Supports 2-1080p60 multi-stream feature with HDMI-Rx and I2S-Rx as input Audio sources with HDMI-Rx and MIPI Rx as an input Video sources; and with HDMI-Tx and I2S-Tx as Output Audio Sink + HDMI-Tx as Output Video sink pipeline

Other features:

  • This design supports single-channel Stream-Based SCD(Scene Change Detection) IP, only for the HDMI input source. SCD must be enabled for the HDMI input source through configuration.

Supported Resolution:

The table below provides the supported resolution from GUI and command-line app in this design.

Resolution

GUI

Command Line

Single Stream

Single Stream

Multi-stream

4Kp60

X

NA

4Kp30

√ (Max 2)

1080p60

√ (Max 2)

√ - Supported
x – Not supported
NA – Not applicable

The below sections describe the HDMI / MIPI Video Capture and HDMI Display with the Audio from HDMI / I2S sources. It is VCU TRD design supporting HDMI-Rx audio/video + HDMI-Tx with Audio/video and MIPI-Rx video + I2S-Rx audio with HDMI-Tx video + I2S-Tx audio.

For the overview, software tools, system requirements, and design files follow the link below:

The below figure shows the HDMI, MIPI Video Capture along with HDMI, I2S Audio Capture and HDMI Display with Audio design hardware block diagram.

                                                                   

The below figure shows the HDMI, MIPI Video Capture  along with HDMI, I2S Audio Capture and HDMI Display with Audio design software block diagram.

                                                       

1.1 Board Setup

Refer below link for Board Setup

  • Zynq UltraScale+ MPSoC VCU TRD 2021.1 Board Setup

  • I2S Audio. signals from MPSoC PL fabric are connected to PMOD0 GPIO Header (J55 - right angle female connector )

  • PMOD I2S2 Add on card connects to J55 connector and its Master/Slave select jumper (JP1) should be placed into the Slave (SLV) position

1.2 Run Flow

The TRD package is released with the source code, Vivado project, Petalinux BSP, and SD card image that enables the user to run the demonstration. It also includes the binaries necessary to configure and boot the ZCU106 board. Prior to running the steps mentioned in this wiki page, download the TRD package and extract its contents to a directory referred to as TRD_HOME which is the home directory.

Refer below link to download all TRD contents.

  • Refer