This page provides all the information related to Design Module 11 - VCU TRD Xilinx low latency(LLP2) PL DDR NV12 HDMI design.
...
Make sure HDMI-Rx should be configured to 4kp60 mode, while running below example pipelines.
Low latency(LLP1/LLP2) video and audio+video stream-in pipelines are not supported in vcu_gst_app.
The vcu_gst_app uses rtp RTP+RTCP streaming and opus encoder for LLP1/LLP2 audio+video stream-out use-cases.
All single-stream serial/streaming pipelines have audio configuration ON by default. To execute only display pipeline, change the
Audio Enableproperty toFALSEin the configuration file.
...
4kp60 NV12 HEVC ultra low-latency(LLP2) audio+video stream-in pipeline execution. where, 192.168.25.90 is server’s IP address.
| Code Block |
|---|
$ gst-launch-1.0 -v rtpbin latency=407 name=rtpbin udpsrc name=udpsrc_video buffer-size=60000000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H265" port=5004 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtph265depay ! h265parse ! video/x-h265, alignment=nal ! omxh265dec low-latency=1 ! video/x-raw\(memory:XLNXLL\) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false 'video-sink=kmssink bus-id=a0070000.v_mix plane-id=33 hold-extra-sample=1 show-preroll=false sync=true ' sync=true -v udpsrc port=5005 ! rtpbin.recv_rtcp_sink_0 rtpbin.send_rtcp_src_0 ! udpsink host=192.168.25.90 port=5006 sync=false async=false udpsrc name=udpsrc_audio port=5008 caps="application/x-rtp, media=(string)audio, clock-rate=(int)48000, encoding-name=(string)OPUS" ! rtpbin.recv_rtp_sink_1 rtpbin. ! rtpopusdepay ! opusdec ! audioconvert ! alsasink device="hw:1,0" latency-time=100005000 buffer-time=3000010000 udpsrc port=5009 ! rtpbin.recv_rtcp_sink_1 rtpbin.send_rtcp_src_1 ! udpsink host=192.168.25.90 port=5010 sync=false async=false |
Above rtp RTP+RTCP audio+video stream-in pipeline is applicable for streaming port = 5004 only. If the streaming port has changed, the respective port changes should be made in the pipeline to work. Refer to port description in streaming configuration parameters to understand how video and audio rtp/rtcp port numbers are assigned in LLL1/LLP2 audio+video stream-out pipelines.
For LLP1/LLP2 Multistream Multi-stream HEVC serial and stream-out use-cases (2-4kp30, 2-1080p60, 4-1080p60), use ENC_EXTRA_OP_BUFFERS=10 variable before vcu_gst_app command. The sample pipeline is given below:
...
Pixel Format | GStreamer Format | Media Bus Format | GStreamer HEVC Profile | GStreamer AVC Profile | Kmssink Plane-id |
|---|---|---|---|---|---|
NV12 | NV12 | VYYUYY8_1X24 | main | high | 33 and 34 |
Video0in the each gst-launch pipelines indicates a video node for the input source.Make sure HDMI-Rx should be configured to 4kp60 mode, while running below example pipelines.
LLP1/LLP2 video / audio+video stream-in pipelines are not supported using vcu_gst_app.
For LLP1/LLP2 Multi-stream HEVC serial and stream-out use-cases (2-4kp30, 2-1080p60, 4-1080p60), use
ENC_EXTRA_OP_BUFFERS=10variable before gst-launch-1.0 command.For LLP1/LLP2 Multi-stream serial and stream-in use-cases (2-4kp30, 2-1080p60, 4-1080p60), use
internal-entropy-buffers=3property in decoder.
...
latency-timeandbuffer-timein Audio+Video serial and streaming pipelines are very aggressive here compared to upstream GStreamer default values to get optimize latency for LLP2 Audio+Video pipeline. You can tune and experiment these parameters in case of any issue with audio/video sync, audio/video packet drop and audio distortion for your use-case.
Run the following gst-launch-1.0 command to display NV12 video on HDMI-Tx using low-latency(LLP1) GStreamer pipeline.
| Code Block |
|---|
$ gst-launch-1.0 -v v4l2src io-mode=dmabuf device=/dev/video0 ! video/x-raw, width=3840, height=2160, format=NV12, framerate=60/1 ! omxh265enc num-slices=8 control-rate=low-latency gop-mode=low-delay-p target-bitrate=25000 cpb-size=500 gdr-mode=horizontal initial-delay=250 periodicity-idr=240 filler-data=0 prefetch-buffer=true ! video/x-h265, alignment=nal ! queue max-size-buffers=0 ! omxh265dec low-latency=1 internal-entropy-buffers=5 ! video/x-raw ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false 'video-sink=kmssink bus-id=a0070000.v_mix hold-extra-sample=1 show-preroll=false sync=true' sync=true -v |
...
| Code Block |
|---|
$ gst-launch-1.0 -v v4l2src io-mode=dmabuf device=/dev/video0 ! video/x-raw\(memory:XLNXLL\), width=3840, height=2160, format=NV12, framerate=60/1 ! omxh265enc numgop-sliceslength=8 control-rate=low-latency 60 gop-mode=low-delay-p low-bandwidth=FALSE target-bitrate=25000 cpbb-sizeframes=5000 gdrnum-modeslices=horizontal8 initialcontrol-delayrate=250 periodicity-idr=240 filler-data=0low-latency qp-mode=auto prefetch-buffer=true cpb-size=500 initial-delay=250 gdr-mode=horizontal periodicity-idr=240 ! video/x-h265, alignment=nal, profile=main ! queue max-size-buffers=0 ! omxh265dec low-latency=1 internal-entropy-buffers=5 ! video/x-raw\(memory:XLNXLL\) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false 'video-sink=kmssink bus-id=a0070000.v_mix plane-id=33 hold-extra-sample=1 show-preroll=false sync=true ' sync=true -v alsasrc device="hw:1,1" ! queuelatency-time=5000 buffer-time=10000 ! audio/x-raw, channels=2, rate=48000, format=S24_32LE ! queue max-size-bytes=0 ! alsasink device="hw:1,0" latency-time=100005000 buffer-time=30000 10000 |
Run the following gst-launch-1.0 command to stream-out NV12 video using low-latency(LLP1) GStreamer pipeline. where, 192.168.25.89 is host/client IP address and 5004 is a port number.
...
Run the following gst-launch-1.0 command to capture, encode and stream-out Xilinx's ultra low-latency(LLP2) HDMI NV12 video and opus encoded HDMI audio using the Gstreamer pipeline.
...
GStreamer pipeline. It enables transmission and reception of both RTP and RTCP packets. RTP packets are sent on ports 5004(for video) and 5008(for audio). sender RTCP packets are sent on ports 5005(for video) and 5009(for audio). The receiver RTCP packets are received on ports 5006(for video) and 5010(for audio). Here, 192.168.25.89 is client’s IP address.
| Code Block |
|---|
$ gst-launch-1.0 rtpbin name=rtpbin v4l2src device=/dev/video0 io-mode=4 ! video/x-raw\(memory:XLNXLL\), format=NV12, width=3840, height=2160, framerate=60/1 ! omxh265enc =2160, framerate=60/1 ! omxh265enc gop-length=60 gop-mode=low-delay-p low-bandwidth=FALSE target-bitrate=25000 b-frames=0 num-slices=8 control-rate=low-latency qp-mode=auto prefetch-buffer=true cpb-size=500 initial-delay=250 gdr-mode=horizontal periodicity-idr=240 ! video/x-h265, alignment=nal, profile=main ! queue ! rtph265pay ! rtpbin.send_rtp_sink_0 rtpbin.send_rtp_src_0 ! udpsink port=5004 buffer-size=60000000 host=192.168.25.89 qos-dscp=60 max-bitrate=120000000 async=false max-lateness=-1 rtpbin.send_rtcp_src_0 ! udpsink host=192.168.25.89 port=5005 sync=false async=false udpsrc port=5006 ! rtpbin.recv_rtcp_sink_0 alsasrc device="hw:1,1" latency-time=5000 buffer-time=10000 ! audioconvert ! audio/x-raw, channels=2, rate=48000, format=S16LE ! opusenc ! rtpopuspay pt=97 ! rtpbin.send_rtp_sink_1 rtpbin.send_rtp_src_1 ! udpsink port=5008 host=192.168.25.89 sync=false async=false rtpbin.send_rtcp_src_1 ! udpsink host=192.168.25.89 port=5009 sync=false async=false udpsrc port=5010 ! rtpbin.recv_rtcp_sink_1 |
Run the following gst-launch-1.0 command to stream-in, decode and play Xilinx's ultra low-latency(LLP2) HDMI NV12 video and opus encoded HDMI audio using the GStreamer pipeline. It enables transmission and reception of both RTP and RTCP packets. RTP packets are received on ports 5004(for video) and 5008(for audio). sender RTCP packets are received on ports 5005(for video) and 5009(for audio). The receiver RTCP packets are sent on ports 5006(for video) and 5010(for audio). Here, 192.168.25.90 is server’s IP address.
| Code Block |
|---|
$ gst-launch-1.0 -v rtpbin latency=7 name=rtpbin udpsrc name=udpsrc_video buffer-size=60000000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H265" port=5004 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtph265depay ! h265parse ! video/x-h265, alignment=nal ! omxh265dec low-latency=1 ! video/x-raw\(memory:XLNXLL\) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false 'video-sink=kmssink bus-id=a0070000.v_mix plane-id=33 hold-extra-sample=1 show-preroll=false sync=true ' sync=true -v udpsrc port=5005 ! rtpbin.recv_rtcp_sink_0 rtpbin.send_rtcp_src_0 ! udpsink host=192.168.25.90 port=5006 sync=false async=false udpsrc name=udpsrc_audio port=5008 caps="application/x-rtp, media=(string)audio, clock-rate=(int)48000, encoding-name=(string)OPUS" ! rtpbin.recv_rtp_sink_1 rtpbin. ! rtpopusdepay ! opusdec ! audioconvert ! alsasink device="hw:1,0" latency-time=5000 buffer-time=10000 udpsrc port=5009 ! rtpbin.recv_rtcp_sink_1 rtpbin.send_rtcp_src_1 ! udpsink host=192.168.25.90 port=5010 sync=false async=false |
Run the following gst-launch-1.0 command to capture, encode and stream-out Xilinx's ultra low-latency(LLP2) HDMI NV12 video and opus encoded HDMI audio using the GStreamer pipeline. It enables transmission of only RTP packets. RTP packets are sent on ports 5004(for video) and 5005(for audio).
| Code Block |
|---|
$ gst-launch-1.0 rtpbin name=rtpbin v4l2src device=/dev/video0 io-mode=4 ! video/x-raw\(memory:XLNXLL\), format=NV12, width=3840, height=2160, framerate=60/1 ! omxh265enc gop-length=60 gop-mode=low-delay-p low-bandwidth=FALSE target-bitrate=25000 b-frames=0 num-slices=8 control-rate=low-latency gopqp-mode=low-delay-p target-bitrate=25000auto prefetch-buffer=true cpb-size=500 initial-delay=250 gdr-mode=horizontal initial-delay=250 periodicity-idr=240 filler-data=0 prefetch-buffer=true ! video/x-h265, alignment=nal, profile=main ! queue ! rtph265pay pt=96 ! rtpbin.send_rtp_sink_0 rtpbin.send_rtp_src_0 ! udpsink port=5004 buffer-size=60000000 host=192.168.25.89 qos-dscp=60 max-bitrate=120000000 rtpbin.send_rtcp_src_0 ! udpsink port=5005 sync=false async=false udpsrc port=5006 ! rtpbin.recv_rtcp_sink_0max-lateness=-1 alsasrc device="hw:1,1" latency-time=5000 buffer-time=10000 ! audioconvert ! audio/x-raw, channels=2, rate=48000, format=S16LE ! opusenc ! rtpopuspay pt=97 ! rtpbin.send_rtp_sink_1 rtpbin.send_rtp_src_1 ! udpsink port=50085005 host=192.168.25.89 sync=false rtpbin.send_rtcp_src_1 ! udpsink port=5009 sync=false async=false udpsrc port=5010 ! rtpbin.recv_rtcp_sink_1 |
Run the following gst-launch-1.0 command to stream-in, decode and play Xilinx's ultra low-latency(LLP2) HDMI NV12 video and opus encoded HDMI audio using the Gstreamer GStreamer pipeline. It enables reception of only RTP packets. RTP packets are received on ports 5004(for video) and 5005(for audio).
| Code Block |
|---|
$ gst-launch-1.0 -v rtpbin latency=407 name=rtpbin udpsrc name=udpsrc_video buffer-size=60000000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H265" port=5004 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtph265depay ! h265parse ! video/x-h265, alignment=nal ! omxh265dec low-latency=1 internal-entropy-buffers=5-latency=1 ! video/x-raw\(memory:XLNXLL\) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false 'video-sink=kmssink bus-id=a0070000.v_mix plane-id=33 hold-extra-sample=1 show-preroll=false sync=true ' sync=true -v udpsrc port=5005 ! rtpbin.recv_rtcp_sink_0 rtpbin.send_rtcp_src_0 ! udpsink port=5006 sync=false async=false udpsrc name=udpsrc_audio port=50085005 caps="application/x-rtp, media=(string)audio, clock-rate=(int)48000, encoding-name=(string)OPUS" ! rtpbin.recv_rtp_sink_1 rtpbin. ! rtpopusdepay ! opusdec ! audioconvert ! alsasink device="hw:1,0" latency-time=100005000 buffer-time=30000 udpsrc port=5009 ! rtpbin.recv_rtcp_sink_1 rtpbin.send_rtcp_src_1 ! udpsink port=5010 sync=false async=false10000 |