Versions Compared

Old Version 11

changes.mady.by.user Ronak Shah

Saved on

compared with

New Version Current

changes.mady.by.user Daksha Rohit

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Table of Contents
excludeZynq UltraScale＋ MPSoC VCU TRD 2018.3 - 10G HDMI Video Capture and Display

1 Overview

The primary goal of this Design is to demonstrate the capabilities of VCU hard block present in Zynq UltraScale+ EV devices. The TRD will serve as a platform to tune the performance parameters of VCU and arrive at optimal configurations for encoder and decoder blocks with the streaming use case where bandwidth plays a vital role. 10G will give sufficient bandwidth for the streaming protocol to play video pipeline smoothly.

This design supports the following video interfaces:

Sources:

  • HDMI-Rx capture pipeline implemented in the PL.
  • File source (SD card, USB storage, SATA hard disk).
  • Stream-In from network or internet.

Sinks:

  • HDMI-Tx display pipeline implemented in the PL.

VCU Codec:

  • Video Encode/Decode capability using VCU hard block in PL 
    • AVC/HEVC encoding.
    • Encoder/decoder parameter configuration.

Streaming Interfaces:

  • 10G Ethernet on PL 
  • 1G Ethernet on PS

Video format:

  • NV12


10G Deliverables:

Pipeline

Input source

Output Type

Resolution

Video encode/Decoder type

Deliverables

Record/Stream-Out pipeline

HDMI-Rx

File Sink/ Stream-Out

4K/1080p

HEVC/AVC

HDMI Rx Video encodes with VCU and stores it in a container format.

Playback pipeline

File Source/ Stream-In

HDMI –Tx

4K/1080p

HEVC/AVC

Playback of the local-file/stream-in with video decoded using VCU and display on  HDMI-Tx.

Capture--> Display

HDMI-Rx

HDMI -Tx

4K/1080p

HEVC/AVC

HDMI Rx Video passes to HDMI-Tx without VCU.

Capture--> Encode--> Decode--> Display

HDMI-Rx

HDMI -Tx

4K/1080p

HEVC/AVC

HDMI Rx raw video passes through  VCU elements encoder and decoder and finally displays on HDMI-Tx.


Supported Resolution:

The table below provides the supported resolution in this design.

Resolution
Command Line
Single StreamMulti-stream
4kp60√NA
4kp30√NA
1080p60√NA


√ - Supported
NA – Not applicable
x – Not supported


The below figure shows the TRD block diagram.

The below sections describe the 10G HDMI Video Capture and HDMI Display design. It is VCU TRD design supporting 10G HDMI-Rx and HDMI-Tx. For the overview, software tools, system requirements and design files follow the link below:

The below figure shows the 10G HDMI Video Capture and HDMI Display design hardware block diagram.

The below figure shows the 10G HDMI Video Capture and HDMI Display design software block diagram.

1.1 Board Setup

Refer below link for Board Setup

...

4 Appendix B

  • To check the link status, resolution and video node of HDMI input source, run below xmedia-ctl command.
Code Block
themeMidnight
$ xmedia-ctl -p -d /dev/media0

When HDMI source is connected to 4KP60 resolution, it shows:

Code Block
themeMidnight
root@zcu106_vcu_trd:/media/card# xmedia-ctl -p -d /dev/media0
Media controller API version 4.19.0

Media device information
------------------------
driver          xilinx-video
model           Xilinx Video Composite Device
serial          
bus info        
hw revision     0x0
driver version  4.19.0

Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0 -----> Video node for HDMI Rx source
        pad0: Sink
                <- "a0080000.v_proc_ss":1 [ENABLED]

- entity 5: a0080000.v_proc_ss (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        pad0: Sink
                [fmt:VYYUYY8_1X24/3840x2160 field:none]
                <- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
        pad1: Source
                [fmt:VYYUYY8_1X24/3840x2160 field:none]
                -> "vcap_hdmi output 0":0 [ENABLED]

- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Source
                [fmt:VYYUYY8_1X24/3840x2160 field:none colorspace:rec709]
                [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custo]
                [dv.detect:BT.656/1120 3840x2160p60 (4400x2250) stds:CEA-861 flags:CE-video] -----> Resolution and Frame-rate of HDMI Rx source 
                -> "a0080000.v_proc_ss":0 [ENABLED]

NOTE: Check resolution and frame-rate of "dv.detect" under "v_hdmi_rx_ss" node.

When the HDMI source is not connected, it shows:

Code Block
themeMidnight
root@zcu106_vcu_trd:/media/card# xmedia-ctl -p -d /dev/media0
Media controller API version 4.19.0

Media device information
------------------------
driver          xilinx-video
model           Xilinx Video Composite Device
serial          
bus info        
hw revision     0x0
driver version  4.19.0

Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0 -----> Video node for HDMI Rx source
        pad0: Sink
                <- "a0080000.v_proc_ss":1 [ENABLED]

- entity 5: a0080000.v_proc_ss (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        pad0: Sink
                [fmt:VYYUYY8_1X24/3840x2160 field:none]
                <- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
        pad1: Source
                [fmt:VYYUYY8_1X24/3840x2160 field:none]
                -> "vcap_hdmi output 0":0 [ENABLED]

- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Source
                [fmt:VYYUYY8_1X24/3840x2160 field:none colorspace:rec709]
                [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custo]
                [dv.query:no-link] -----> HDMI Rx Link Status
                -> "a0080000.v_proc_ss":0 [ENABLED]

NOTE: Here "dv.query:no-link" under "v_hdmi_rx_ss" node shows HDMI-Rx source is not connected or HDMI-Rx source is not active(Try waking up the device by pressing the key on remote).


Notes for gst-launch-1.

Follow the below steps to switch

0 commands:

Code Block
themeMidnight
Common Configuration : START Num Of Input : 1 Output : HDMI Out Type : Display Frame Rate
  • Make sure the HDMI-Rx
    • resolution from 1080p60 to 4kp60.
    • Check current HDMI Input Source Resolution (1080p60) by following the above-mentioned steps.
    • Run vcu_gst_app for current HDMI resolution (1080p60) by executing the following command.
Code Block
themeMidnight
$ vcu_gst_app /media/card/config/input.cfg

Below configurations needs to be set in input.cfg for HDMI-1080p60.

  • media pipeline is configured for 4kp60 resolution and source/sink has the same color format. Run below xmedia-ctl commands to set the resolution and format of the HDMI scaler node.

When HDMI Input Source is NVIDIA SHIELD

Code Block
themeMidnight
$ xmedia-ctl -d /dev/media0 -V "\"a0080000.v_proc_ss\":0  [fmt:RBG888_1X24/3840x2160 field:none]"
$ xmedia-ctl -d /dev/media0 -V "\"a0080000.v_proc_ss\":1  [fmt:VYYUYY8_1X24/3840x2160 field:none]"

NOTE: Make sure NVIDIA SHIELD is configured for 4kp resolution and RGB888 color format.

When HDMI Input Source is ABOX

Code Block
themeMidnight
$ xmedia-ctl -d /dev/media0 -V "\"a0080000.v_proc_ss\":0  [fmt:VYYUYY8_1X24/3840x2160 field:none]"
$ xmedia-ctl -d /dev/media0 -V "\"a0080000.v_proc_ss\":1  [fmt:VYYUYY8_1X24/3840x2160 field:none]"

NOTE: Make sure ABOX is configured for 4kp resolution and VYYUYY8 color format.


  • Follow the below steps to switch the HDMI-Rx resolution from 1080p60 to 4kp60.
    • Check current HDMI Input Source Resolution (1080p60) by following the above-mentioned steps.
    • Run vcu_gst_app for current HDMI resolution (1080p60) by executing the following command.
Code Block
themeMidnight
$ vcu_gst_app /media/card/config/input.cfg

Below configurations needs to be set in input.cfg for HDMI-1080p60.

Code Block
themeMidnight
Common Configuration    : START
Num Of Input            : 1
Output                  : HDMI
Out Type                : Display
Frame Rate              : 60
Exit

Input Configuration     : START
Input Num               : 1
Input Type              : hdmi
Raw                     : TRUE
Width                   : 1920
Height                  : 1080
Exit
    • Change Resolution of HDMI Input Source from 1080p60 to 4kp60 by following the below steps.
      • Set the HDMI source resolution to 4kp60 (Home page → settings → display & Sound → Resolution → change to 4kp60).
      • Save the configuration to take place the change.
    • Verify the desired HDMI Input Source Resolution (4kp60) by following the above-mentioned steps.
  • If HDMI Tx link-up issue is observed after Linux booting, use the following command:
Code Block
themeMidnight
$ modetest -D a0070000.v_mix -s 40:3840x2160-60@BG24 -w 36:"alpha":0
  • Run the following gst-launch-1.0 command to display raw HDMI video using the GStreamer pipeline.
$ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=4 ! video/x-raw, width=3840, height=2160, format=NV12, framerate=60/1 ! queue ! kmssink bus-id="a0070000.v_mix"
Code Block
themeMidnight
themeMidnight
$ modetest -D a0070000.v_mix -s 40:3840x2160-60@BG24 -w 36:"alpha":0
  • Run the following gst-launch-1.0 command to display processed(capture → encode → decode → display) command to display raw HDMI video using the GStreamer pipeline.
Code Block
themeMidnight
$ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=4 ! video/x-raw, width=3840, height=2160, format=NV12, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=basic gop-length=60 b-frames=0 target-bitrate=60000 num-slices=8 control-rate=constant prefetch-buffer=true low-bandwidth=false filler-data=true cpb-size=1000 initial-delay=500queue ! kmssink bus-id="a0070000.v_mix"
  • Run the following gst-launch-1.0 command to display processed(capture → encode → decode → display) HDMI video using the GStreamer pipeline.
Code Block
themeMidnight
$ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=4 ! video/x-h265raw, profilewidth=main3840, alignmentheight=au ! h265parse2160, format=NV12, framerate=60/1 ! omxh265decomxh265enc internal-entropy-buffers=5 low-latency=0 ! queue max-size-bytes=0 ! fpsdisplaysink text-overlay=false video-sink="kmssink bus-id="a0070000.v_mix" sync=true" sync=true
  • Run the following gst-launch-1.0 command to record HDMI video using the GStreamer pipeline.
Code Block
themeMidnight
$ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=4qp-mode=auto gop-mode=basic gop-length=60 b-frames=0 target-bitrate=60000 num-slices=8 control-rate=constant prefetch-buffer=true low-bandwidth=false filler-data=true cpb-size=1000 initial-delay=500 ! video/x-rawh265, formatprofile=NV12main,width=3840,height=2160,framerate=60/1 alignment=au ! h265parse ! omxh265encomxh265dec qpinternal-entropy-modebuffers=auto5 goplow-mode=basic gop-length=60 b-frames=0 target-bitrate=60000 num-slices=8 control-rate=constant prefetch-buffer=true low-bandwidth=false filler-data=true cpb-size=1000 initial-delay=500latency=0 ! queue max-size-bytes=0 ! fpsdisplaysink text-overlay=false video-sink="kmssink bus-id="a0070000.v_mix" sync=true" sync=true
  • Run the following gst-launch-1.0 command to record HDMI video using the GStreamer pipeline.
Code Block
themeMidnight
$ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=4 ! video/x-h265, profile=main, alignment=auraw, format=NV12,width=3840,height=2160,framerate=60/1 ! h265parse ! queue ! mpegtsmux alignment=7 name=mux ! filesink location="/run/test.ts"

NOTE: File location should be USB-3.0 to avoid the read-write bandwidth issue.

  • Run the following gst-launch-1.0 command to play the recorded file using the GStreamer pipeline.
Code Block
themeMidnight
$ gst-launch-1.0 uridecodebin uri="file:///run/test.ts" ! queue max-size-bytes=0 ! fpsdisplaysink text-overlay=false video-sink="kmssink bus-id="a0070000.v_mix""omxh265enc qp-mode=auto gop-mode=basic gop-length=60 b-frames=0 target-bitrate=60000 num-slices=8 control-rate=constant prefetch-buffer=true low-bandwidth=false filler-data=true cpb-size=1000 initial-delay=500 ! video/x-h265, profile=main, alignment=au ! h265parse ! queue ! mpegtsmux alignment=7 name=mux ! filesink location="/run/test.ts"

NOTE: File location should be USB-3.0 to avoid the read-write bandwidth issue.

  • Run the following gst-launch-1.0 command to stream-out HDMI video play the recorded file using the GStreamer pipeline.
Code Block
themeMidnight
$ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=4 ! video/x-raw, format=NV12, width=3840, height=2160, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=basic gop-length=60 b-frames=0 target-bitrate=60000 num-slices=8 control-rate=constant prefetch-buffer=true low-bandwidth=false filler-data=true cpb-size=1000 initial-delay=500 periodicity-idr=60 gst-launch-1.0 uridecodebin uri="file:///run/test.ts" ! queue max-size-bytes=0 ! fpsdisplaysink text-overlay=false video-sink="kmssink bus-id="a0070000.v_mix""

NOTE: File location should be USB-3.0 to avoid the read-write bandwidth issue.

  • Run the following gst-launch-1.0 command to stream-out HDMI video using the GStreamer pipeline.
Code Block
themeMidnight
$ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=4 ! video/x-h265raw, profile=main-422-10format=NV12, alignmentwidth=au ! h265parse ! queue ! mpegtsmux alignment=7 name=mux ! rtpmp2tpay ! udpsink host=192.168.25.89 port=5004

NOTE: Here 192.168.25.89 is host/client IP address and 5004 is port no.

  • Run the following gst-launch-1.0 command to display stream-in video using the Gstreamer pipeline where 5004 is port no.
Code Block
themeMidnight
$ gst-launch-1.0 udpsrc port=5004 buffer-size=60000000 caps="application/x-rtp, clock-rate=90000" ! rtpjitterbuffer latency=1000 ! rtpmp2tdepay ! tsparse ! video/mpegts ! tsdemux name=demux ! queue ! h265parse ! omxh265dec internal-entropy-buffers=5 low-latency=0 ! queue max-size-bytes=0 ! fpsdisplaysink text-overlay=false video-sink="kmssink bus-id="a00c0000.v_mix" " sync=true
Note: Low latency stream pipeline cannot be launched with vcu-gst-app. Hence use below gst-launch pipelines
3840, height=2160, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=basic gop-length=60 b-frames=0 target-bitrate=60000 num-slices=8 control-rate=constant prefetch-buffer=true low-bandwidth=false filler-data=true cpb-size=1000 initial-delay=500 periodicity-idr=60 ! video/x-h265, profile=main-422-10, alignment=au ! h265parse ! queue ! mpegtsmux alignment=7 name=mux ! rtpmp2tpay ! udpsink host=192.168.25.89 port=5004

NOTE: Here 192.168.25.89 is host/client IP address and 5004 is port no.

  • Run the following gst-launch-1.0 command for low-latency stream-out pipeline. Where "videoX" indicates a video node for the input sourceto display stream-in video using the Gstreamer pipeline where 5004 is port no.
Code Block
themeMidnight
$ gst-launch-1.0 v4l2src device=/dev/videoX io-mode=4 ! video/x-raw, format=NV12, width=3840, height=2160, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=basic gop-length=60 b-frames=0 target-bitrate=60000 num-slices=8 control-rate=low-latency prefetch-buffer=true low-bandwidth=false filler-data=true cpb-size=1000 initial-delay=500 periodicity-idr=60 ! video/x-h265, profile=main, alignment=nal ! queue ! mpegtsmux alignment=7 name=mux ! rtpmp2tpay ! udpsink host=192.168.26.89 port=5004
NOTE:Here 192.168.26.89 is host/client IP address and 5004 is port no
0 udpsrc port=5004 buffer-size=60000000 caps="application/x-rtp, clock-rate=90000" ! rtpjitterbuffer latency=1000 ! rtpmp2tdepay ! tsparse ! video/mpegts ! tsdemux name=demux ! queue ! h265parse ! omxh265dec internal-entropy-buffers=5 low-latency=0 ! queue max-size-bytes=0 ! fpsdisplaysink text-overlay=false video-sink="kmssink bus-id="a00c0000.v_mix" " sync=true

Note: Low latency stream pipeline cannot be launched with vcu-gst-app. Hence use below gst-launch pipelines.

  • Run the following gst-launch-1.0 command to display for low-latency stream-in on HDMI-Tx video using the Gstreamer pipeline where 5004 is port no.out pipeline. Where "videoX" indicates a video node for the input source
Code Block
themeMidnight
$ gst-launch-1.0 udpsrc port=5004 buffer-size=60000000 caps="application/x-rtp, clock-rate=90000" ! rtpjitterbuffer latency=1000 ! rtpmp2tdepay ! tsparse ! video/mpegts ! tsdemux name=demux ! queue ! h265parse ! video/x-h265, profile=main, alignment=nal ! omxh265dec internal-entropy-buffers=5 low-latency=1 ! queue max-size-bytes=0 ! kmssink bus-id="a0070000.v_mix"

Notes for gst-launch-1.0 commands:

  • Make sure the HDMI-Rx media pipeline is configured for 4kp60 resolution and source/sink has the same color format. Run below xmedia-ctl commands to set the resolution and format of the HDMI scaler node.

When HDMI Input Source is NVIDIA SHIELD

Code Block
themeMidnight
$ xmedia-ctl -d /dev/media0 -V "\"a0080000.v_proc_ss\":0  [fmt:RBG888_1X24/3840x2160 field:none]"
$ xmedia-ctl -d /dev/media0 -V "\"a0080000.v_proc_ss\":1  [fmt:VYYUYY8_1X24/3840x2160 field:none]"

NOTE: Make sure NVIDIA SHIELD is configured for 4kp resolution and RGB888 color format.

When HDMI Input Source is ABOX

Code Block
themeMidnight
$ xmedia-ctl -d /dev/media0 -V "\"a0080000.v_proc_ss\":0  [fmt:VYYUYY8_1X24/3840x2160 field:none]"
$ xmedia-ctl -d /dev/media0 -V "\"a0080000.v_proc_ss\":1  [fmt:VYYUYY8_1X24/3840x2160 field:none]"
NOTE: Make sure ABOX is configured for 4kp resolution and VYYUYY8 color format.
$ gst-launch-1.0 v4l2src device=/dev/videoX io-mode=4 ! video/x-raw, format=NV12, width=3840, height=2160, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=basic gop-length=60 b-frames=0 target-bitrate=60000 num-slices=8 control-rate=low-latency prefetch-buffer=true low-bandwidth=false filler-data=true cpb-size=1000 initial-delay=500 periodicity-idr=60 ! video/x-h265, profile=main, alignment=nal ! queue ! mpegtsmux alignment=7 name=mux ! rtpmp2tpay ! udpsink host=192.168.26.89 port=5004

NOTE:Here 192.168.26.89 is host/client IP address and 5004 is port no.

  • Run the following gst-launch-1.0 command to display low-latency stream-in on HDMI-Tx video using the Gstreamer pipeline where 5004 is port no.
Code Block
themeMidnight
$ gst-launch-1.0 udpsrc port=5004 buffer-size=60000000 caps="application/x-rtp, clock-rate=90000" ! rtpjitterbuffer latency=1000 ! rtpmp2tdepay ! tsparse ! video/mpegts ! tsdemux name=demux ! queue ! h265parse ! video/x-h265, profile=main, alignment=nal ! omxh265dec internal-entropy-buffers=5 low-latency=1 ! queue max-size-bytes=0 ! kmssink bus-id="a0070000.v_mix"