Zynq UltraScale＋ MPSoC VCU TRD 2019.2 - PL DDR HDMI Video Capture and Display

...

root@zcu106_vcu_trd:~# xmedia-ctl -p -d /dev/media3 -----> media node for HDMI input source
Media controller API version 4.19.0

Media device information
------------------------
driver          xilinx-video
model           Xilinx Video Composite Device
serial          
bus info        
hw revision     0x0
driver version  4.19.0

Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0  -----> Video node for HDMI Rx source
        pad0: Sink
                <- "a0040000.v_proc_ss":1 [ENABLED]

- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev3
        pad0: Sink
                [fmt:VYYUYY10_4X20/1280x720 field:none colorspace:srgb]
                <- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
        pad1: Source
                [fmt:VYYUYY10_4X20/1920x1080 field:none colorspace:srgb]
                -> "vcap_hdmi output 0":0 [ENABLED]

- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev4
        pad0: Source
                [fmt:RBG888_1X24/3840x2160 field:none colorspace:srgb]
                [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custo
m]
                [dv.query:no-link] -----> HDMI Rx Link Status
                -> "a0040000.v_proc_ss":0 [ENABLED]

Notes for gst-launch-1.0 commands:

• Video node for HDMI Rx source can be checked using xmedia-ctl command. Run below xmedia-ctl command to check video node for HDMI Rx source where "media3" indicates media node for HDMI input source.

$

...

Below configurations needs to be set in input.cfg for HDMI-1080p60.

...

 xmedia-ctl -p -d /dev/media3

• Make sure the HDMI-Rx media pipeline is configured for 4kp60 resolution and source/sink has the same color format for connected nodes. For XV20 format, run below xmedia-ctl commands to set resolution and format of HDMI scaler node where "media3" indicates media node for HDMI input source.

When HDMI Input Source is NVIDIA SHIELD

...

$ 

...

xmedia-ctl -d /dev/media3 -V "\"a0040000.v_proc_ss\":

...

0 [fmt:RBG888_1X24/3840x2160 field:none]"
$ xmedia-ctl -d /dev/media3 -V "\"a0040000.v_proc_ss\":1  [fmt:UYVY10_1X20/3840x2160 field:none]"

NOTE: Make sure NVIDIA SHIELD is configured for 4kp resolution and RBG888_1X24 format.

$ vcu_gst_app /media/card/config/input.cfg

Common Configuration    : START
Num Of Input            : 1
Output                  : HDMI
Out Type                : Display
Frame Rate              : 60
Exit

Input Configuration     : START
Input Num               : 1
Input Type              : hdmi
Raw                     : FALSE
Width                   : 1920
Height                  : 1080
Format                  : XV20
Exit

Encoder Configuration   : START
Encoder Num             : 1
Preset                  : HEVC_HIGH
Exit

% modetest -D a00c0000.v_mix -s 37:3840x2160-60@BG24

$ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=4 ! video/x-raw, width=3840, height=2160, format=NV16_10LE32, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=basic gop-length=60 b-frames=0 target-bitrate=60000 num-slices=8 control-rate=constant prefetch-buffer=true low-bandwidth=false filler-data=true cpb-size=1000 initial-delay=500 ! video/x-h265, profile=main-422-10, alignment=au ! queue ! omxh265dec internal-entropy-buffers=5 low-latency=0 ! queue max-size-bytes=0 ! fpsdisplaysink text-overlay=false video-sink="kmssink bus-id="a00c0000.v_mix" plane-id=30 sync=true" sync=true

$ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=4 num-buffers=3600 ! video/x-raw, width=3840, height=2160, format=NV16_10LE32, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=basic gop-length=60 b-frames=0 target-bitrate=60000 num-slices=8 control-rate=constant prefetch-buffer=true low-bandwidth=false filler-data=true cpb-size=1000 initial-delay=500 ! video/x-h265, profile=main-422-10, alignment=au ! h265parse ! queue ! mpegtsmux alignment=7 name=mux ! filesink location="/run/test.ts"

NOTE: File location should be USB-3.0/SATA/RAMFS to avoid the read-write bandwidth issue.

• Run the following gst-launch-1.0 command to play XV20 recorded file on HDMI-Tx using the GStreamer pipeline.

$ gst-launch-1.0 uridecodebin uri="file:///run/test.ts" ! queue max-size-bytes=0 ! fpsdisplaysink text-overlay=false video-sink="kmssink bus-id="a00c0000.v_mix" plane-id=30=true cpb-size=1000 initial-delay=500 ! video/x-h265, profile=main-422-10, alignment=au ! h265parse ! queue ! mpegtsmux alignment=7 name=mux ! filesink location="/run/test.ts"

NOTE: File location should be USB-3.0/SATA/RAMFS to avoid the read-write bandwidth issue.

• Run the following gst-launch-1.0 command to stream-out the XV20 video play XV20 recorded file on HDMI-Tx using the GStreamer pipeline. Where "video0" indicates a video node for the input source.

$ gst-launch-1.0 v4l2srcuridecodebin deviceuri="file:/dev/video0 io-mode=4 ! video/x-raw, width=3840, height=2160, format=NV16_10LE32, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=basic gop-length=60 b-frames=0 target-bitrate=60000 num-slices=8 control-rate=constant prefetch-buffer=true low-bandwidth=false filler-data=true cpb-size=1000 initial-delay=500 periodicity-idr=60 ! video/x-h265, profile=main-422-10, alignment=au ! h265parse ! queue ! mpegtsmux alignment=7 name=mux ! rtpmp2tpay ! udpsink host=192.168.25.89 port=5004 buffer-size=60000000 max-bitrate=120000000 max-lateness=-1 qos-dscp=60 async=false

NOTE: Here 192.168.25.89 is host/client IP address and 5004 is port no.

• Run the following gst-launch-1.0 command to display XV20 stream-in video on HDMI-Tx using the GStreamer pipeline where 5004 is port no.

$ gst-launch-1.0 udpsrc port=5004 buffer-size=60000000 caps="application/x-rtp, clock-rate=90000" ! rtpjitterbuffer latency=1000 ! rtpmp2tdepay ! tsparse ! video/mpegts ! tsdemux name=demux ! queue ! h265parse/run/test.ts" ! queue max-size-bytes=0 ! fpsdisplaysink text-overlay=false video-sink="kmssink bus-id="a00c0000.v_mix" plane-id=30"

NOTE: File location should be USB-3.0/SATA/RAMFS to avoid the read-write bandwidth issue.

• Run the following gst-launch-1.0 command to stream-out the XV20 video using the GStreamer pipeline. Where "video0" indicates a video node for the input source.

$ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=4 ! video/x-raw, width=3840, height=2160, format=NV16_10LE32, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=basic gop-length=60 b-frames=0 target-bitrate=60000 num-slices=8 control-rate=constant prefetch-buffer=true low-bandwidth=false filler-data=true cpb-size=1000 initial-delay=500 periodicity-idr=60 ! video/x-h265, profile=main-422-10, alignment=au ! omxh265dec internal-entropy-buffers=5 low-latency=0 h265parse ! queue max-size-bytes=0 ! fpsdisplaysinkmpegtsmux text-overlayalignment=false7 video-sink="kmssink bus-id="a00c0000.v_mix" plane-id=30" sync=true

• Run the following gst-launch-1.0 command to stream-out XV20 video using a low-latency(LLP1) GStreamer pipeline. Where "video0" indicates a video node for the input source.

$ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=4 ! video/x-raw, width=3840, height=2160, format=NV16_10LE32, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=low-delay-p gop-length=60 periodicity-idr=60 b-frames=0 target-bitrate=25000 num-slices=8 control-rate=low-latency prefetch-buffer=true low-bandwidth=false filler-data=0 cpb-size=1000 initial-delay=500name=mux ! rtpmp2tpay ! udpsink host=192.168.25.89 port=5004 buffer-size=60000000 max-bitrate=120000000 max-lateness=-1 qos-dscp=60 async=false

NOTE: Here 192.168.25.89 is host/client IP address and 5004 is port no.

• Run the following gst-launch-1.0 command to display XV20 stream-in video on HDMI-Tx using the GStreamer pipeline where 5004 is port no.

$ gst-launch-1.0 udpsrc port=5004 buffer-size=60000000 caps="application/x-rtp, clock-rate=90000" ! rtpjitterbuffer latency=1000 ! rtpmp2tdepay ! tsparse ! video/mpegts ! tsdemux name=demux ! queue ! h265parse ! video/x-h265, profile=main-422-10, alignment=nalau ! queueomxh265dec max-size-bytesinternal-entropy-buffers=5 low-latency=0 ! rtph265pay ! udpsink host=192.168.25.89 port=5004 buffer-size=60000000 max-bitrate=120000000 max-lateness=-1 qos-dscp=60 async=false

...

 queue max-size-bytes=0 ! fpsdisplaysink text-overlay=false video-sink="kmssink bus-id="a00c0000.v_mix" plane-id=30" sync=true

• Run the following gst-launch-1.0 command to display XV20 stream-in video on HDMI-Tx out XV20 video using a low-latency(LLP1) GStreamer pipeline where 5004 is port no) GStreamer pipeline. Where "video0" indicates a video node for the input source.

$ gst-launch-1.0 udpsrcv4l2src port=5004 buffer-size=60000000 caps="applicationdevice=/dev/video0 io-mode=4 ! video/x-rtpraw, mediawidth=video3840, clock-rateheight=900002160, payloadformat=96NV16_10LE32, encoding-name=H265"framerate=60/1 ! rtpjitterbufferomxh265enc latencyqp-mode=5 ! rtph265depay ! h265parse ! video/x-h265, alignment=nal ! omxh265dec low-latency=1 ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false video-sink="kmssink bus-id=a00c0000.v_mix plane-id=30" sync=true

NOTE: The low latency(LLP1) stream-out pipeline is supported with tsmux in vcu_gst_app and low latency(LLP1) stream-in pipeline is not supported in vcu_gst_app.

Notes for gst-launch-1.0 commands:

• Video node for HDMI Rx source can be checked using xmedia-ctl command. Run below xmedia-ctl command to check video node for HDMI Rx source where "media3" indicates media node for HDMI input source.

$ xmedia-ctl -p -d /dev/media3

• Make sure the HDMI-Rx media pipeline is configured for 4kp60 resolution and source/sink has the same color format for connected nodes. For XV20 format, run below xmedia-ctl commands to set resolution and format of HDMI scaler node where "media3" indicates media node for HDMI input source.

When HDMI Input Source is NVIDIA SHIELD

$ xmedia-ctl -d /dev/media3 -V "\"a0040000.v_proc_ss\":0 [fmt:RBG888_1X24/3840x2160 field:none]"
$ xmedia-ctl -d /dev/media3 -V "\"a0040000.v_proc_ss\":1  [fmt:UYVY10_1X20/3840x2160 field:none]"

...

auto gop-mode=low-delay-p gop-length=60 periodicity-idr=60 b-frames=0 target-bitrate=25000 num-slices=8 control-rate=low-latency prefetch-buffer=true low-bandwidth=false filler-data=0 cpb-size=1000 initial-delay=500 ! video/x-h265, profile=main-422-10, alignment=nal ! queue max-size-bytes=0 ! rtph265pay ! udpsink host=192.168.25.89 port=5004 buffer-size=60000000 max-bitrate=120000000 max-lateness=-1 qos-dscp=60 async=false

NOTE: Here 192.168.25.89 is host/client IP address and 5004 is port no.

• Run the following gst-launch-1.0 command to display XV20 stream-in video on HDMI-Tx using a low-latency(LLP1) GStreamer pipeline where 5004 is port no.

$ gst-launch-1.0 udpsrc port=5004 buffer-size=60000000 caps="application/x-rtp, media=video, clock-rate=90000, payload=96, encoding-name=H265" ! rtpjitterbuffer latency=5 ! rtph265depay ! h265parse ! video/x-h265, alignment=nal ! omxh265dec low-latency=1 ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false video-sink="kmssink bus-id=a00c0000.v_mix plane-id=30" sync=true

NOTE: The low latency(LLP1) stream-out pipeline is supported with tsmux in vcu_gst_app and low latency(LLP1) stream-in pipeline is not supported in vcu_gst_app.