Zynq UltraScale+ MPSoC VCU TRD - Debugging - HDMI Rx Capture Pipeline

Owned by Chris Arndt
Last updated: Sept 17, 2019 by Forrest Pickett (Unlicensed)
15 min read

The VCU TRD has a HDMI Rx capture pipeline. The following are some tips on how to debug the HDMI Rx capture pipeline configuration.  They walk through how to find the settings, adjust them and then send the output to the HDMI Tx Display Pipeline, or capture and dump out some frames to disk, using YAVTA.

This was tested on the Zynq UltraScale+ MPSoC VCU TRD 2018.3, but the same steps should apply other versions of the Zynq UltraScale+ MPSoC VCU TRD, as well as to other platforms which have a similar pipeline, like the Zynq UltraScale+ MPSoC Base TRD 2018.3.


Table of Contents

How to find the HDMI Rx Capture Pipeline

Find if the driver was loaded:

Look at the DMESG output for the Xilinx HDMI Rx Subsystem to find out if the driver was loaded and the associated address.

$ dmesg | grep -i "xilinx-hdmi-rx"
[   11.832125] xilinx-hdmi-rx a0000000.v_hdmi_rx_ss: probed
[   11.876814] xilinx-hdmi-rx a0000000.v_hdmi_rx_ss: Using 2 EDID blocks (256 bytes) from 'xilinx/xilinx-hdmi-rx-edid.bin'.
[   11.909455] xilinx-hdmi-rx a0000000.v_hdmi_rx_ss: hdmi rx audio initialized
[   11.916483] xilinx-hdmi-rx a0000000.v_hdmi_rx_ss: probe successful

Find Device ID:

List the devices and find that the HDMI Rx capture is /dev/video0.
NOTE:  The other HDMI captures are all created from the broadcaster as duplicates of the original HDMI Rx to do multi-stream testing.

$ v4l2-ctl --list-devices
vcap_csi output 0 (platform:vcap_csi:0):
        /dev/video1

vcap_hdmi output 0 (platform:vcap_hdmi:0):
        /dev/video0

vcap_hdmi_2 output 0 (platform:vcap_hdmi_2:0):
        /dev/video3

vcap_hdmi_3 output 0 (platform:vcap_hdmi_3:0):
        /dev/video4

vcap_hdmi_4 output 0 (platform:vcap_hdmi_4:0):
        /dev/video5

vcap_hdmi_5 output 0 (platform:vcap_hdmi_5:0):
        /dev/video6

vcap_hdmi_6 output 0 (platform:vcap_hdmi_6:0):
        /dev/video7

vcap_hdmi_7 output 0 (platform:vcap_hdmi_7:0):
        /dev/video8

vcap_tpg output 0 (platform:vcap_tpg:0):
        /dev/video2

Find Media Pipeline

To find the associated Media device you can use either of the following methods.
NOTE: This is important as the media device # will change depending on if the LI-IMX274MIPI-FMC card is connected or not.  (In this test case the LI-IMX274MIPI-FMC card is not connected.)

  • Look at the devfs
  • Look through the media devices and to find which media device contains the HDMI Rx (/dev/video0)
$ ls /sys/class/video4linux/video0/device/ | grep -i media
media7

$ for m in /dev/media* ; do echo $m ; media-ctl -p -d $m | grep -B 3 /dev/video0 ; echo   '-----------' ; done
/dev/media0
-----------
/dev/media1
-----------
/dev/media2
-----------
/dev/media3
-----------
/dev/media4
-----------
/dev/media5
-----------
/dev/media6
-----------
/dev/media7
Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
-----------

Display Media Pipeline

You can use media-ctl to list out the structure of the pipeline and you will find that it looks like this:

  • HDMI Rx -> VPSS Scaler -> Video Frame Buffer Write IP
$ media-ctl -d /dev/media7 -p
Media controller API version 4.14.0

Media device information
------------------------
driver          xilinx-video
model           Xilinx Video Composite Device
serial          
bus info        
hw revision     0x0
driver version  4.14.0

Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "a0080000.v_proc_ss":1 [ENABLED]

- entity 5: a0080000.v_proc_ss (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev7
        pad0: Sink
                [fmt:Y8_1X8/1280x720 field:none colorspace:srgb]
                <- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
        pad1: Source
                [fmt:VYYUYY8_1X24/1280x720 field:none]
                -> "vcap_hdmi output 0":0 [ENABLED]

- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev8
        pad0: Source
                [fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]
                [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
                [dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
                -> "a0080000.v_proc_ss":0 [ENABLED]

How to Configure the HDMI Rx Pipeline

Update the HDMI Rx Pipeline Formats

Use media-ctl to configure the HDMI Rx Pipeline color formats

#VPSS: Scaler Only with CSC
$ media-ctl -d /dev/media7 -V '"a0080000.v_proc_ss":0  [fmt:RBG888_1X24/1920x1080 field:none]'
$ media-ctl -d /dev/media7 -V '"a0080000.v_proc_ss":1  [fmt:VYYUYY8_1X24/1920x1080 field:none]'

Use media-ctl to check if the settings were updated.

$ media-ctl -d /dev/media7 -p
Media controller API version 4.14.0

Media device information
------------------------
driver          xilinx-video
model           Xilinx Video Composite Device
serial          
bus info        
hw revision     0x0
driver version  4.14.0

Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "a0080000.v_proc_ss":1 [ENABLED]

- entity 5: a0080000.v_proc_ss (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev7
        pad0: Sink
                [fmt:RBG888_1X24/1920x1080 field:none]
                <- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
        pad1: Source
                [fmt:VYYUYY8_1X24/1920x1080 field:none]
                -> "vcap_hdmi output 0":0 [ENABLED]

- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev8
        pad0: Source
                [fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]
                [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
                [dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
                -> "a0080000.v_proc_ss":0 [ENABLED]

Verify that HDMI Rx Pipeline is working:

HDMI Rx to HDMI Tx using GStreamer

Use GStreamer to send the output to HDIM Tx Subsystem for display on HDMI Monitor.

#Make sure modetest is no longer running
$ killall modetest

#Use GStreamer to pipe the output from the HDMI Rx to the HDMI Tx
$ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=4 '!' video/x-raw, width=1920, height=1080, framerate=60/1, format=NV12 '!' queue '!' kmssink bus-id=a0070000.v_mix fullscreen-overlay=1

Dump Frames using Yavta

Use Yavta list the supported formats before capturing:

$ yavta --enum-formats /dev/video0
Device /dev/video0 opened.
Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes) device.
- Available formats:
        Format 0: NV12M (32314d4e)
        Type: Video capture (1)
        Name: Y/CbCr 4:2:0 (N-C)

        Format 1: NV12 (3231564e)
        Type: Video capture (1)
        Name: Y/CbCr 4:2:0

        Format 0: NV12M (32314d4e)
        Type: Video capture mplanes (9)
        Name: Y/CbCr 4:2:0 (N-C)

        Format 1: NV12 (3231564e)
        Type: Video capture mplanes (9)
        Name: Y/CbCr 4:2:0

Video format: NV12 (3231564e) 1920x1080 field none, 1 planes: 
 * Stride 1920, buffer size 3110400

Use Yavta to dump frames.

You will receive 3 output frames that can be viewed using your faforite YUV Viewer.

$ yavta -n 3 -c10 -f NV12 -s 1920x1080 --skip 7 -F /dev/video0
Device /dev/video0 opened.
Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes) device.
Video format set: NV12 (3231564e) 1920x1080 field none, 1 planes: 
 * Stride 1920, buffer size 3110400
Video format: NV12 (3231564e) 1920x1080 field none, 1 planes: 
 * Stride 1920, buffer size 3110400
3 buffers requested.
length: 1 offset: 3521476496 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x7fa6487000.
length: 1 offset: 3521476496 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x7fa618f000.
length: 1 offset: 3521476496 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x7fa5e97000.
0 (0) [-] none 0 0 B 1044.863621 1044.863635 26.603 fps ts mono/EoF
1 (1) [-] none 1 0 B 1044.880305 1044.880315 59.938 fps ts mono/EoF
2 (2) [-] none 2 0 B 1044.897026 1044.897045 59.805 fps ts mono/EoF
3 (0) [-] none 3 0 B 1044.913673 1044.913683 60.071 fps ts mono/EoF
4 (1) [-] none 4 0 B 1044.930356 1044.930366 59.941 fps ts mono/EoF
5 (2) [-] none 5 0 B 1044.947040 1044.947090 59.938 fps ts mono/EoF
6 (0) [-] none 6 0 B 1044.963724 1044.963774 59.938 fps ts mono/EoF
7 (1) [-] none 7 0 B 1044.980409 1044.980458 59.934 fps ts mono/EoF
8 (2) [-] none 8 0 B 1044.997091 1045.017878 59.945 fps ts mono/EoF
9 (0) [-] none 9 0 B 1045.013774 1045.054662 59.941 fps ts mono/EoF
Captured 10 frames in 0.228630 seconds (43.738613 fps, 0.000000 B/s).
3 buffers released.

Modify the Pipeline to support RGB

After you are sure that the pipeline is working you can then change the output to RGB.

Use media-ctl to change the output type of the Video Scaler with Color Space Conversion.

#VPSS: Scaler Only with CSC
$ media-ctl -d /dev/media7 -V '"a0080000.v_proc_ss":1  [fmt:RBG888_1X24/1920x1080 field:none]'

Use media-ctl to check if the settings were updated.

$ media-ctl -d /dev/media7 -p
Media controller API version 4.14.0

Media device information
------------------------
driver          xilinx-video
model           Xilinx Video Composite Device
serial          
bus info        
hw revision     0x0
driver version  4.14.0

Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "a0080000.v_proc_ss":1 [ENABLED]

- entity 5: a0080000.v_proc_ss (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev7
        pad0: Sink
                [fmt:RBG888_1X24/1920x1080 field:none]
                <- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
        pad1: Source
                [fmt:RBG888_1X24/1920x1080 field:none]
                -> "vcap_hdmi output 0":0 [ENABLED]

- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev8
        pad0: Source
                [fmt:RBG888_1X24/1920x1080 field:none]
                [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
                [dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
                -> "a0080000.v_proc_ss":0 [ENABLED]

HDMI Rx to HDMI Tx using GStreamer

Because the output pipeline does not support RGB input, you can not use GStreamer to output to HDIM Tx Subsystem for display on HDMI Monitor.  You would need to modify the Video Mixer in the PL design to accept RGB input.

Dump Frames using Yavta

Use Yavta list the supported formats before capturing:

$ yavta --enum-formats /dev/video1
Device /dev/video1 opened.
Device `vcap_csi output 0' on `platform:vcap_csi:0' is a video output (without mplanes) device.
- Available formats:
        Format 0: XBGR (52474258)
        Type: Video capture (1)
        Name: 32-bit XBGR 8-8-8-8

        Format 1: XR24 (34325258)
        Type: Video capture (1)
        Name: 32-bit BGRX 8-8-8-8

        Format 2: RGB24 (33424752)
        Type: Video capture (1)
        Name: 24-bit RGB 8-8-8

        Format 0: XBGR (52474258)
        Type: Video capture mplanes (9)
        Name: 32-bit XBGR 8-8-8-8

        Format 1: XR24 (34325258)
        Type: Video capture mplanes (9)
        Name: 32-bit BGRX 8-8-8-8

        Format 2: RGB24 (33424752)
        Type: Video capture mplanes (9)
        Name: 24-bit RGB 8-8-8

Video format: NV12 (3231564e) 1920x1080 field none, 1 planes: 
 * Stride 1920, buffer size 3110400

Use Yavta to dump frames.

You will receive 3 output frames that can be viewed using your favorite YUV Viewer.

$ yavta -n 3 -c10 -f RGB24 -s 1920x1080 --skip 7 -F /dev/video0
Device /dev/video0 opened.
Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes) device.
Video format set: RGB24 (33424752) 1920x1080 field none, 1 planes: 
 * Stride 5760, buffer size 6220800
Video format: RGB24 (33424752) 1920x1080 field none, 1 planes: 
 * Stride 5760, buffer size 6220800
3 buffers requested.
length: 1 offset: 3309268624 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x7f82928000.
length: 1 offset: 3309268624 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x7f82339000.
length: 1 offset: 3309268624 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x7f81d4a000.
0 (0) [-] none 0 0 B 1373.336794 1373.336810 26.164 fps ts mono/EoF
1 (1) [-] none 1 0 B 1373.353529 1373.353586 59.755 fps ts mono/EoF
2 (2) [-] none 2 0 B 1373.370162 1373.370172 60.121 fps ts mono/EoF
3 (0) [-] none 3 0 B 1373.386846 1373.386856 59.938 fps ts mono/EoF
4 (1) [-] none 4 0 B 1373.403529 1373.403538 59.941 fps ts mono/EoF
5 (2) [-] none 5 0 B 1373.420213 1373.420222 59.938 fps ts mono/EoF
6 (0) [-] none 6 0 B 1373.436898 1373.436907 59.934 fps ts mono/EoF
7 (1) [-] none 7 0 B 1373.453581 1373.453590 59.941 fps ts mono/EoF
8 (2) [-] none 8 0 B 1373.470265 1373.531092 59.938 fps ts mono/EoF
9 (0) [-] none 9 0 B 1373.486948 1373.608656 59.941 fps ts mono/EoF
Captured 10 frames in 0.310082 seconds (32.249497 fps, 0.000000 B/s).
3 buffers released.


Debug:

Check if HDMI Input has properly locked:

#Debug HDMI Rx                                                                            
hdmi_addr=$(dmesg | grep -i 'xilinx-hdmi-rx' | grep -i 'probed' | awk '/xilinx-hdmi-rx/ { print $4 }' | sed 's/://g')                                                               
                                                                                          
cat /sys/devices/platform/amba_pl@0/${hdmi_addr}/hdmi_info
cat /sys/devices/platform/amba_pl@0/${hdmi_addr}/hdmi_log
cat /sys/devices/platform/amba_pl@0/${hdmi_addr}/vphy_info
cat /sys/devices/platform/amba_pl@0/${hdmi_addr}/vphy_log


Appendix:

List Supported Formats:

You can also use v4l2-ctl to list the supported formats.  These will change when the IP is reconfigured.

When configured for VYYUYY8 Pixel Format:

$ v4l2-ctl -d /dev/video0 -D --list-formats-ext
Driver Info (not using libv4l2):
        Driver name   : xilinx-vipp
        Card type     : vcap_hdmi output 0
        Bus info      : platform:vcap_hdmi:0
        Driver version: 4.14.0
        Capabilities  : 0x84201000
                Video Capture Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04201000
                Video Capture Multiplanar
                Streaming
                Extended Pix Format
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture Multiplanar
        Pixel Format: 'NM12'
        Name        : Y/CbCr 4:2:0 (N-C)

        Index       : 1
        Type        : Video Capture Multiplanar
        Pixel Format: 'NV12'
        Name        : Y/CbCr 4:2:0

When configured for RBG888 format:

$ v4l2-ctl -d /dev/video0 -D --list-formats-ext
Driver Info (not using libv4l2):
        Driver name   : xilinx-vipp
        Card type     : vcap_hdmi output 0
        Bus info      : platform:vcap_hdmi:0
        Driver version: 4.14.0
        Capabilities  : 0x84201000
                Video Capture Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04201000
                Video Capture Multiplanar
                Streaming
                Extended Pix Format
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture Multiplanar
        Pixel Format: 'XBGR'
        Name        : 32-bit XBGR 8-8-8-8

        Index       : 1
        Type        : Video Capture Multiplanar
        Pixel Format: 'XR24'
        Name        : 32-bit BGRX 8-8-8-8

        Index       : 2
        Type        : Video Capture Multiplanar
        Pixel Format: 'RGB3'
        Name        : 24-bit RGB 8-8-8

Related Links:

© Copyright 2019 - 2022 Xilinx Inc. Privacy Policy