Zynq UltraScale+MPSoC VCU TRD 2019.2 - Xilinx Low Latency PL DDR NV16 HDMI Video Capture and Display
...
Code Block |
---|
|
root@zcu106_vcu_trd:~# xmedia-ctl -p -d /dev/media1 -----> media node for HDMI input source
Media controller API version 4.19.0
Media device information
------------------------
driver xilinx-video
model Xilinx Video Composite Device
serial
bus info
hw revision 0x0
driver version 4.19.0
Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0 -----> Video node for HDMI Rx source
pad0: Sink
<- "a0040000.v_proc_ss":1 [ENABLED]
- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:VYYUYY10_4X20/1280x720 field:none colorspace:srgb]
<- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
pad1: Source
[fmt:VYYUYY10_4X20/1920x1080 field:none colorspace:srgb]
-> "vcap_hdmi output 0":0 [ENABLED]
- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:RBG888_1X24/1280x720 field:none colorspace:srgb]
[dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom
]
[dv.query:no-link] -----> HDMI Rx Link Status
-> "a0040000.v_proc_ss":0 [ENABLED] |
NOTE: Here "dv.query:no-link" under "v_hdmi_rx_ss" node shows HDMI-Rx source is not connected or HDMI-Rx source is not active(Try waking up the device by pressing a key on remote).
Follow the below steps to switch the HDMI-Rx resolution from 1080p60 to 4kp60.Check current HDMI Input Source Resolution (1080p60) by following the above-mentioned steps.Run vcu_gst_app for current HDMI resolution (1080p60) by executing the following command
Notes for gst-launch-1.0 commands:
- Video node for HDMI Rx source can be checked using xmedia-ctl command. Run below xmedia-ctl command to check video node for HDMI Rx source where "media1" indicates media node for HDMI input source.
...
xmedia-ctl -p -d /dev/media1 |
- Make sure HDMI-Rx media pipeline is configured for 4kp60 resolution and source/sink have the same colour format for connected nodes. For NV16 format, run below xmedia-ctl commands to set resolution and format of HDMI scaler node where "media1" indicates media node for HDMI input source.
When HDMI Input Source is NVIDIA SHIELD
...
...
xmedia-ctl -d /dev/media1 -V "\"a0040000.v_proc_ss\": |
...
0 [fmt:RBG888_1X24/3840x2160 field:none]"
$ xmedia-ctl -d /dev/media1 -V "\"a0040000.v_proc_ss\":1 [fmt:UYVY8_1X16/3840x2160 field:none]" |
NOTE: Make sure NVIDIA SHIELD is configured for 4kp resolution and RBG888_1X24 format.
- Follow the below steps to switch the HDMI-Rx resolution from 1080p60 to 4kp60.
- Check current HDMI Input Source Resolution (1080p60) by following the above-mentioned steps.
- Run vcu_gst_app for current HDMI resolution (1080p60) by executing the following command.
Code Block |
---|
|
$ vcu_gst_app /media/card/config/input.cfg |
Below configurations needs to be set in input.cfg for non-LLP2 HDMI-1080p60 use-case.
Code Block |
---|
|
Common Configuration : START
Num Of Input : 1
InputOutput Type : hdmi_1 Raw : HDMI
Out Type : FALSEDisplay
WidthFrame Rate : 60
Exit
:Input 1920Configuration Height : START
Input Num : 1080 Format : 1
Input Type : NV16hdmi_1
EnableRaw LLP2 : FALSE Exit Encoder Configuration : STARTFALSE
EncoderWidth Num : 1 Encoder Name : 1920
Height : HEVC Profile : 1080
Format : Main Rate Control : Low_LatencyNV16
FillerEnable DataLLP2 : FalseFALSE
QPExit
Encoder Configuration : START
Encoder Num : Auto1
L2Encoder CacheName : HEVC
Profile : TRUE Latency Mode : Main
Sub_FrameRate LowControl Bandwidth : FALSELow_Latency
GopFiller ModeData : False
QP : Basic Bitrate : 25000Auto
BL2 FramesCache : 0
Slice TRUE
Latency Mode : Sub_Frame
Low Bandwidth : 8FALSE
Gop Mode GoP Length : 60
Preset Basic
Bitrate : 25000
B Frames : Custom
Exit |
Change Resolution of HDMI Input Source from 0
Slice : 8
GoP Length : 60
Preset : Custom
Exit |
- Change Resolution of HDMI Input Source from 1080p60 to 4kp60 by following below steps.
- Set the HDMI source resolution to 4kp60 (Homepage → Settings → Display & Sound → Resolution → change to 4kp60).
- Save the configuration to take place the change.
- Verify the desired HDMI Input Source Resolution (4kp60) by following the above-mentioned steps.
- If HDMI Tx link-up issue is observed after Linux booting, use the following command to get the blue screen on HDMI-Tx for 4kp60.
Code Block |
---|
|
% modetest -D a00c0000.v_mix -s 35:3840x2160-60@BG24 |
- The table below lists the parameters of the pixel format.
Pixel Format | GStreamer Format | Media Bus Format | GStreamer HEVC Profile | GStreamer AVC Profile | Kmssink Plane-id |
---|
NV16 | NV16 | UYVY8_1X16 | main-422 | high-4:2:2 | 30 and 31 |
- Run the following gst-launch-1.0 command to display NV16 video on HDMI-Tx using ultra low-latency(LLP2) GStreamer pipeline (capture → encode → decode → display). Where "video0" indicates a video node for the input source.
Code Block |
---|
|
$ gst-launch-1.0 v4l2src io-mode=4 device=/dev/video0 ! video/x-raw\(memory:XLNXLL\), width=3840, height=2160, format=NV16, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=basic gop-length=60 b-frames=0 target-bitrate=25000 num-slices=8 control-rate=low-latency prefetch-buffer=TRUE low-bandwidth=false filler-data=0 cpb-size=1000 initial-delay=500 ! video/x-h265, alignment=nal ! queue max-size-buffers=0 ! omxh265dec low-latency=1 ! video/x-raw\(memory:XLNXLL\) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false video-sink="kmssink bus-id=a00c0000.v_mix plane-id=30 max-lateness=5000000 show-preroll-frame=false sync=true" sync=true Code Block |
---|
|
% modetest -D a00c0000.v_mix -s 35:3840x2160-60@BG24 |
- The table below lists the parameters of the pixel format.
Pixel Format | GStreamer Format | Media Bus Format | GStreamer HEVC Profile | GStreamer AVC Profile | Kmssink Plane-id |
---|
NV16 | NV16 | UYVY8_1X16 | main-422 | high-4:2:2 | 30 and 31 |
- Run the following gst-launch-1.0 command to stream-out display NV16 video on HDMI-Tx using ultra low-latency(LLP2) GStreamer pipeline (capture → encode → decode → display). Where "video0" indicates a video node for the input source.
Code Block |
---|
|
$ gst-launch-1.0 v4l2src io-mode=4 device=/dev/video0 ! video/x-raw\(memory:XLNXLL\), width=3840, height=2160, format=NV16, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode= |
...
...
-length=60 b-frames=0 target-bitrate=25000 num-slices=8 control-rate=low-latency prefetch-buffer=TRUE low-bandwidth=false filler-data=0 cpb-size=1000 initial-delay=500 ! video/x-h265, alignment=nal ! queue max-size-buffers=0 ! omxh265dec low-latency=1 ! |
...
NOTE: Here 192.168.25.89 is host/client IP address and 5004 is port no.
video/x-raw\(memory:XLNXLL\) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false video-sink="kmssink bus-id=a00c0000.v_mix plane-id=30 max-lateness=5000000 show-preroll-frame=false sync=true" sync=true |
- Run the following gst-launch-1.0 command to
...
...
- out NV16 video using ultra low-latency(LLP2) GStreamer pipeline
...
- . Where "video0" indicates a video node for the input source.
Code Block |
---|
|
$ gst-launch-1.0 udpsrc port=5004 buffer-size=60000000 caps="application/x-rtp, media=video, clock-rate=90000, payload=96, encoding-name=H265" ! rtpjitterbuffer latency=5 ! rtph265depay ! h265parse gst-launch-1.0 v4l2src io-mode=4 device=/dev/video0 ! video/x-h265raw\(memory:XLNXLL\), alignmentwidth=nal ! omxh265dec low-latency=3840, height=2160, format=NV16, framerate=60/1 ! omxh265enc video/x-raw\(memory:XLNXLL\) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false video-sink="kmssink bus-id=a00c0000.v_mix plane-id=30" sync=true |
NOTE: Low latency(LLP1/LLP2) stream-in pipeline is not supported in vcu_gst_app.
Notes for gst-launch-1.0 commands:
- Video node for HDMI Rx source can be checked using xmedia-ctl command. Run below xmedia-ctl command to check video node for HDMI Rx source where "media1" indicates media node for HDMI input source.
Code Block |
---|
|
$ xmedia-ctl -p -d /dev/media1 |
- Make sure HDMI-Rx media pipeline is configured for 4kp60 resolution and source/sink have the same colour format for connected nodes. For NV16 format, run below xmedia-ctl commands to set resolution and format of HDMI scaler node where "media1" indicates media node for HDMI input source.
When HDMI Input Source is NVIDIA SHIELD
Code Block |
---|
|
$ xmedia-ctl -d /dev/media1 -V "\"a0040000.v_proc_ss\":0 [fmt:RBG888_1X24/3840x2160 field:none]"
$ xmedia-ctl -d /dev/media1 -V "\"a0040000.v_proc_ss\":1 [fmt:UYVY8_1X16/3840x2160 field:none]" |
...
qp-mode=auto gop-mode=low-delay-p gop-length=60 periodicity-idr=60 b-frames=0 target-bitrate=25000 num-slices=8 control-rate=low-latency prefetch-buffer=TRUE low-bandwidth=false filler-data=0 cpb-size=1000 initial-delay=500 ! video/x-h265, alignment=nal ! queue max-size-buffers=0 ! rtph265pay ! udpsink host=192.168.25.89 port=5004 buffer-size=60000000 max-bitrate=120000000 max-lateness=-1 qos-dscp=60 async=false |
NOTE: Here 192.168.25.89 is host/client IP address and 5004 is port no.
- Run the following gst-launch-1.0 command to display NV16 stream-in video on HDMI-Tx using ultra low-latency(LLP2) GStreamer pipeline where 5004 is port no.
Code Block |
---|
|
$ gst-launch-1.0 udpsrc port=5004 buffer-size=60000000 caps="application/x-rtp, media=video, clock-rate=90000, payload=96, encoding-name=H265" ! rtpjitterbuffer latency=5 ! rtph265depay ! h265parse ! video/x-h265, alignment=nal ! omxh265dec low-latency=1 ! video/x-raw\(memory:XLNXLL\) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false video-sink="kmssink bus-id=a00c0000.v_mix plane-id=30" sync=true |
NOTE: Low latency(LLP1/LLP2) stream-in pipeline is not supported in vcu_gst_app.