/
Zynq UltraScale+ MPSoC VCU TRD - Debugging - HDMI Rx Capture Pipeline

Zynq UltraScale+ MPSoC VCU TRD - Debugging - HDMI Rx Capture Pipeline

Sept 17, 2019

The VCU TRD has a HDMI Rx capture pipeline. The following are some tips on how to debug the HDMI Rx capture pipeline configuration.  They walk through how to find the settings, adjust them and then send the output to the HDMI Tx Display Pipeline, or capture and dump out some frames to disk, using YAVTA.

This was tested on the Zynq UltraScale+ MPSoC VCU TRD 2018.3, but the same steps should apply other versions of the Zynq UltraScale+ MPSoC VCU TRD, as well as to other platforms which have a similar pipeline, like the Zynq UltraScale+ MPSoC Base TRD 2018.3.

 

Table of Contents

How to find the HDMI Rx Capture Pipeline

Find if the driver was loaded:

Look at the DMESG output for the Xilinx HDMI Rx Subsystem to find out if the driver was loaded and the associated address.

$ dmesg | grep -i "xilinx-hdmi-rx" [ 11.832125] xilinx-hdmi-rx a0000000.v_hdmi_rx_ss: probed [ 11.876814] xilinx-hdmi-rx a0000000.v_hdmi_rx_ss: Using 2 EDID blocks (256 bytes) from 'xilinx/xilinx-hdmi-rx-edid.bin'. [ 11.909455] xilinx-hdmi-rx a0000000.v_hdmi_rx_ss: hdmi rx audio initialized [ 11.916483] xilinx-hdmi-rx a0000000.v_hdmi_rx_ss: probe successful

Find Device ID:

List the devices and find that the HDMI Rx capture is /dev/video0.
NOTE:  The other HDMI captures are all created from the broadcaster as duplicates of the original HDMI Rx to do multi-stream testing.

$ v4l2-ctl --list-devices vcap_csi output 0 (platform:vcap_csi:0): /dev/video1 vcap_hdmi output 0 (platform:vcap_hdmi:0): /dev/video0 vcap_hdmi_2 output 0 (platform:vcap_hdmi_2:0): /dev/video3 vcap_hdmi_3 output 0 (platform:vcap_hdmi_3:0): /dev/video4 vcap_hdmi_4 output 0 (platform:vcap_hdmi_4:0): /dev/video5 vcap_hdmi_5 output 0 (platform:vcap_hdmi_5:0): /dev/video6 vcap_hdmi_6 output 0 (platform:vcap_hdmi_6:0): /dev/video7 vcap_hdmi_7 output 0 (platform:vcap_hdmi_7:0): /dev/video8 vcap_tpg output 0 (platform:vcap_tpg:0): /dev/video2

Find Media Pipeline

To find the associated Media device you can use either of the following methods.
NOTE: This is important as the media device # will change depending on if the LI-IMX274MIPI-FMC card is connected or not.  (In this test case the LI-IMX274MIPI-FMC card is not connected.)

  • Look at the devfs

  • Look through the media devices and to find which media device contains the HDMI Rx (/dev/video0)

$ ls /sys/class/video4linux/video0/device/ | grep -i media media7 $ for m in /dev/media* ; do echo $m ; media-ctl -p -d $m | grep -B 3 /dev/video0 ; echo '-----------' ; done /dev/media0 ----------- /dev/media1 ----------- /dev/media2 ----------- /dev/media3 ----------- /dev/media4 ----------- /dev/media5 ----------- /dev/media6 ----------- /dev/media7 Device topology - entity 1: vcap_hdmi output 0 (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video0 -----------

Display Media Pipeline

You can use media-ctl to list out the structure of the pipeline and you will find that it looks like this:

  • HDMI Rx -> VPSS Scaler -> Video Frame Buffer Write IP

$ media-ctl -d /dev/media7 -p Media controller API version 4.14.0 Media device information ------------------------ driver xilinx-video model Xilinx Video Composite Device serial bus info hw revision 0x0 driver version 4.14.0 Device topology - entity 1: vcap_hdmi output 0 (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video0 pad0: Sink <- "a0080000.v_proc_ss":1 [ENABLED] - entity 5: a0080000.v_proc_ss (2 pads, 2 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev7 pad0: Sink [fmt:Y8_1X8/1280x720 field:none colorspace:srgb] <- "a0000000.v_hdmi_rx_ss":0 [ENABLED] pad1: Source [fmt:VYYUYY8_1X24/1280x720 field:none] -> "vcap_hdmi output 0":0 [ENABLED] - entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev8 pad0: Source [fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb] [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom] [dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video] -> "a0080000.v_proc_ss":0 [ENABLED]

How to Configure the HDMI Rx Pipeline

Update the HDMI Rx Pipeline Formats

Use media-ctl to configure the HDMI Rx Pipeline color formats

#VPSS: Scaler Only with CSC $ media-ctl -d /dev/media7 -V '"a0080000.v_proc_ss":0 [fmt:RBG888_1X24/1920x1080 field:none]' $ media-ctl -d /dev/media7 -V '"a0080000.v_proc_ss":1 [fmt:VYYUYY8_1X24/1920x1080 field:none]'

Use media-ctl to check if the settings were updated.

$ media-ctl -d /dev/media7 -p Media controller API version 4.14.0 Media device information ------------------------ driver xilinx-video model Xilinx Video Composite Device serial bus info hw revision 0x0 driver version 4.14.0 Device topology - entity 1: vcap_hdmi output 0 (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video0 pad0: Sink <- "a0080000.v_proc_ss":1 [ENABLED] - entity 5: a0080000.v_proc_ss (2 pads, 2 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev7 pad0: Sink [fmt:RBG888_1X24/1920x1080 field:none] <- "a0000000.v_hdmi_rx_ss":0 [ENABLED] pad1: Source [fmt:VYYUYY8_1X24/1920x1080 field:none] -> "vcap_hdmi output 0":0 [ENABLED] - entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev8 pad0: Source [fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb] [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom] [dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video] -> "a0080000.v_proc_ss":0 [ENABLED]

Verify that HDMI Rx Pipeline is working:

HDMI Rx to HDMI Tx using GStreamer

Use GStreamer to send the output to HDIM Tx Subsystem for display on HDMI Monitor.

#Make sure modetest is no longer running $ killall modetest #Use GStreamer to pipe the output from the HDMI Rx to the HDMI Tx $ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=4 '!' video/x-raw, width=1920, height=1080, framerate=60/1, format=NV12 '!' queue '!' kmssink bus-id=a0070000.v_mix fullscreen-overlay=1

Dump Frames using Yavta

Use Yavta list the supported formats before capturing:

$ yavta --enum-formats /dev/video0 Device /dev/video0 opened. Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes) device. - Available formats: Format 0: NV12M (32314d4e) Type: Video capture (1) Name: Y/CbCr 4:2:0 (N-C) Format 1: NV12 (3231564e) Type: Video capture (1) Name: Y/CbCr 4:2:0 Format 0: NV12M (32314d4e) Type: Video capture mplanes (9) Name: Y/CbCr 4:2:0 (N-C) Format 1: NV12 (3231564e) Type: Video capture mplanes (9) Name: Y/CbCr 4:2:0 Video format: NV12 (3231564e) 1920x1080 field none, 1 planes: * Stride 1920, buffer size 3110400

Use Yavta to dump frames.

You will receive 3 output frames that can be viewed using your faforite YUV Viewer.

$ yavta -n 3 -c10 -f NV12 -s 1920x1080 --skip 7 -F /dev/video0 Device /dev/video0 opened. Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes) device. Video format set: NV12 (3231564e) 1920x1080 field none, 1 planes: * Stride 1920, buffer size 3110400 Video format: NV12 (3231564e) 1920x1080 field none, 1 planes: * Stride 1920, buffer size 3110400 3 buffers requested. length: 1 offset: 3521476496 timestamp type/source: mono/EoF Buffer 0/0 mapped at address 0x7fa6487000. length: 1 offset: 3521476496 timestamp type/source: mono/EoF Buffer 1/0 mapped at address 0x7fa618f000. length: 1 offset: 3521476496 timestamp type/source: mono/EoF Buffer 2/0 mapped at address 0x7fa5e97000. 0 (0) [-] none 0 0 B 1044.863621 1044.863635 26.603 fps ts mono/EoF 1 (1) [-] none 1 0 B 1044.880305 1044.880315 59.938 fps ts mono/EoF 2 (2) [-] none 2 0 B 1044.897026 1044.897045 59.805 fps ts mono/EoF 3 (0) [-] none 3 0 B 1044.913673 1044.913683 60.071 fps ts mono/EoF 4 (1) [-] none 4 0 B 1044.930356 1044.930366 59.941 fps ts mono/EoF 5 (2) [-] none 5 0 B 1044.947040 1044.947090 59.938 fps ts mono/EoF 6 (0) [-] none 6 0 B 1044.963724 1044.963774 59.938 fps ts mono/EoF 7 (1) [-] none 7 0 B 1044.980409 1044.980458 59.934 fps ts mono/EoF 8 (2) [-] none 8 0 B 1044.997091 1045.017878 59.945 fps ts mono/EoF 9 (0) [-] none 9 0 B 1045.013774 1045.054662 59.941 fps ts mono/EoF Captured 10 frames in 0.228630 seconds (43.738613 fps, 0.000000 B/s). 3 buffers released.

Modify the Pipeline to support RGB

After you are sure that the pipeline is working you can then change the output to RGB.

Use media-ctl to change the output type of the Video Scaler with Color Space Conversion.

#VPSS: Scaler Only with CSC $ media-ctl -d /dev/media7 -V '"a0080000.v_proc_ss":1 [fmt:RBG888_1X24/1920x1080 field:none]'

Use media-ctl to check if the settings were updated.

$ media-ctl -d /dev/media7 -p Media controller API version 4.14.0 Media device information ------------------------ driver xilinx-video model Xilinx Video Composite Device serial bus info hw revision 0x0 driver version 4.14.0 Device topology - entity 1: vcap_hdmi output 0 (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video0 pad0: Sink <- "a0080000.v_proc_ss":1 [ENABLED] - entity 5: a0080000.v_proc_ss (2 pads, 2 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev7 pad0: Sink [fmt:RBG888_1X24/1920x1080 field:none] <- "a0000000.v_hdmi_rx_ss":0 [ENABLED] pad1: Source [fmt:RBG888_1X24/1920x1080 field:none] -> "vcap_hdmi output 0":0 [ENABLED] - entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev8 pad0: Source [fmt:RBG888_1X24/1920x1080 field:none] [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom] [dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video] -> "a0080000.v_proc_ss":0 [ENABLED]

HDMI Rx to HDMI Tx using GStreamer

Because the output pipeline does not support RGB input, you can not use GStreamer to output to HDIM Tx Subsystem for display on HDMI Monitor.  You would need to modify the Video Mixer in the PL design to accept RGB input.

Dump Frames using Yavta

Use Yavta list the supported formats before capturing:

$ yavta --enum-formats /dev/video1 Device /dev/video1 opened. Device `vcap_csi output 0' on `platform:vcap_csi:0' is a video output (without mplanes) device. - Available formats: Format 0: XBGR (52474258) Type: Video capture (1) Name: 32-bit XBGR 8-8-8-8 Format 1: XR24 (34325258) Type: Video capture (1) Name: 32-bit BGRX 8-8-8-8 Format 2: RGB24 (33424752) Type: Video capture (1) Name: 24-bit RGB 8-8-8 Format 0: XBGR (52474258) Type: Video capture mplanes (9) Name: 32-bit XBGR 8-8-8-8 Format 1: XR24 (34325258) Type: Video capture mplanes (9) Name: 32-bit BGRX 8-8-8-8 Format 2: RGB24 (33424752) Type: Video capture mplanes (9) Name: 24-bit RGB 8-8-8 Video format: NV12 (3231564e) 1920x1080 field none, 1 planes: * Stride 1920, buffer size 3110400

Use Yavta to dump frames.

You will receive 3 output frames that can be viewed using your favorite YUV Viewer.

$ yavta -n 3 -c10 -f RGB24 -s 1920x1080 --skip 7 -F /dev/video0 Device /dev/video0 opened. Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes) device. Video format set: RGB24 (33424752) 1920x1080 field none, 1 planes: * Stride 5760, buffer size 6220800 Video format: RGB24 (33424752) 1920x1080 field none, 1 planes: * Stride 5760, buffer size 6220800 3 buffers requested. length: 1 offset: 3309268624 timestamp type/source: mono/EoF Buffer 0/0 mapped at address 0x7f82928000. length: 1 offset: 3309268624 timestamp type/source: mono/EoF Buffer 1/0 mapped at address 0x7f82339000. length: 1 offset: 3309268624 timestamp type/source: mono/EoF Buffer 2/0 mapped at address 0x7f81d4a000. 0 (0) [-] none 0 0 B 1373.336794 1373.336810 26.164 fps ts mono/EoF 1 (1) [-] none 1 0 B 1373.353529 1373.353586 59.755 fps ts mono/EoF 2 (2) [-] none 2 0 B 1373.370162 1373.370172 60.121 fps ts mono/EoF 3 (0) [-] none 3 0 B 1373.386846 1373.386856 59.938 fps ts mono/EoF 4 (1) [-] none 4 0 B 1373.403529 1373.403538 59.941 fps ts mono/EoF 5 (2) [-] none 5 0 B 1373.420213 1373.420222 59.938 fps ts mono/EoF 6 (0) [-] none 6 0 B 1373.436898 1373.436907 59.934 fps ts mono/EoF 7 (1) [-] none 7 0 B 1373.453581 1373.453590 59.941 fps ts mono/EoF 8 (2) [-] none 8 0 B 1373.470265 1373.531092 59.938 fps ts mono/EoF 9 (0) [-] none 9 0 B 1373.486948 1373.608656 59.941 fps ts mono/EoF Captured 10 frames in 0.310082 seconds (32.249497 fps, 0.000000 B/s). 3 buffers released.

 

Debug:

Check if HDMI Input has properly locked:

#Debug HDMI Rx hdmi_addr=$(dmesg | grep -i 'xilinx-hdmi-rx' | grep -i 'probed' | awk '/xilinx-hdmi-rx/ { print $4 }' | sed 's/://g') cat /sys/devices/platform/amba_pl@0/${hdmi_addr}/hdmi_info cat /sys/devices/platform/amba_pl@0/${hdmi_addr}/hdmi_log cat /sys/devices/platform/amba_pl@0/${hdmi_addr}/vphy_info cat /sys/devices/platform/amba_pl@0/${hdmi_addr}/vphy_log

 

Appendix:

List Supported Formats:

You can also use v4l2-ctl to list the supported formats.  These will change when the IP is reconfigured.

When configured for VYYUYY8 Pixel Format:

$ v4l2-ctl -d /dev/video0 -D --list-formats-ext Driver Info (not using libv4l2): Driver name : xilinx-vipp Card type : vcap_hdmi output 0 Bus info : platform:vcap_hdmi:0 Driver version: 4.14.0 Capabilities : 0x84201000 Video Capture Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04201000 Video Capture Multiplanar Streaming Extended Pix Format ioctl: VIDIOC_ENUM_FMT Index : 0 Type : Video Capture Multiplanar Pixel Format: 'NM12' Name : Y/CbCr 4:2:0 (N-C) Index : 1 Type : Video Capture Multiplanar Pixel Format: 'NV12' Name : Y/CbCr 4:2:0

When configured for RBG888 format:

$ v4l2-ctl -d /dev/video0 -D --list-formats-ext Driver Info (not using libv4l2): Driver name : xilinx-vipp Card type : vcap_hdmi output 0 Bus info : platform:vcap_hdmi:0 Driver version: 4.14.0 Capabilities : 0x84201000 Video Capture Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04201000 Video Capture Multiplanar Streaming Extended Pix Format ioctl: VIDIOC_ENUM_FMT Index : 0 Type : Video Capture Multiplanar Pixel Format: 'XBGR' Name : 32-bit XBGR 8-8-8-8 Index : 1 Type : Video Capture Multiplanar Pixel Format: 'XR24' Name : 32-bit BGRX 8-8-8-8 Index : 2 Type : Video Capture Multiplanar Pixel Format: 'RGB3' Name : 24-bit RGB 8-8-8

Related Links: