Xilinx V4L2 Demosaic driver
Table of Contents
Introduction
The purpose of this page is to describe the Linux V4L2 driver for Xilinx Sensor Demosaic soft IP.
To understand more details about the Sensor Demosaic IP please see the Product Guide below.
The Linux V4L2 Demosaic driver is based on the V4L2 framework, and creates a subdev node(/dev/v4l-subdev*) which can be used to configure the Demosaic IP core.
The Xilinx V4L2 Demosaic driver is able to control the Sensor Demosaic Soft IP that can convert a Bayer video stream to RGB video stream.
The driver is present at https://github.com/Xilinx/linux-xlnx/blob/master/drivers/media/platform/xilinx/xilinx-demosaic.c
The driver comes up with default color format on sink pad as MEDIA_BUS_FMT_SRGGB8_1X8 and on source pad as MEDIA_BUS_FMT_RBG888_1X24.
The Source Pad (output) media bus format cannot be changed through media-ctl. However, the Sink pad's (input) media bus format can be changed to any Bayer format.
entity 16: a0060000.demosaic (2 pads, 2 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev4 pad0: Sink [fmt:SRGGB8/1280x720 field:none] <- "a3c30000.tpg":0 [ENABLED] pad1: Source [fmt:RBG24/1280x720 field:none] -> "a0050000.gamma":0 [ENABLED]
IP / Driver Features
IP Feature | 2018.1 to 2018.3 | 2019.1 to 2020.2 |
---|---|---|
IP version | 1.0 | 1.0 |
Compatible string | xlnx,v-demosaic(2023.1 and above) | |
1 / 2 / 4 / 8 pixels per clock support | Yes | Yes |
8 / 10 / 12 / 16 bits per component | Only 8 bpc supported | 8 / 10 / 12 / 16 bpc support |
Resolutions from 64x64 to 8192x4320 | Yes* | |
Supports 4Kp60 in all supported device families | Yes | |
Two interpolation algorithms High resolution and Fringe tolerant | NA for driver | |
Ability to remove horizontal zipper artifacts | NA for driver | |
Use UltraRAM for line buffers | NA for driver |
*Tested for 8192x4320, 3840x2160, 1920x1080, 1280x720
Kernel Configuration
CONFIG_VIDEO_XILINX_DEMOSAIC and CONFIG_VIDEO_XILINX should be enabled.
Device tree binding
The dts node should be defined with correct hardware configuration. How to define the DT node is documented here :https://github.com/Xilinx/linux-xlnx/blob/master/Documentation/devicetree/bindings/media/xilinx/xlnx%2Cv-demosaic.txt
Testing Procedure
This driver is a V4L2 sub-device driver, which means it is designed to operate with a video device in it's graph. The driver has been tested with a Xilinx video device (a V4L2 capture device) that is backed bya DMA element (like Framebuffer Write IP). The V4L2 device driver for this IP cannot be tested in isolation by itself. The successful operation of the driver can be tested in two phases :
- Successful Creation of a Media Device
- Successful Frame Captures (or writing of buffers by the Frame-buffer Write IP)
Successful Creation of a Media Device
media-ctl is a Linux user space utility that can control media entities, it is a part of a larger set of V4L2 tools called v4l-utilsmedia-ctl allows you to print the topology of your media pipeline and specify input and output pad properties.
Successful Frame Captures
Please see the Test Approach section of the Frame Buffer Write Wiki for more details on how to capture frames.
Test Design
We need a ZCU102 Rev 1.0 board. Enable TPG, Demosaic, Gamma LUT and Framebuffer Write in kernel defconfig.
Modify the demosaic driver as below as TPG output is RGB type and Demosaic requires SRGGB type.
44 diff --git a/drivers/media/platform/xilinx/xilinx-demosaic.c b/drivers/media/platform/xilinx/xilinx-demosaic.c 45 index 505fc14..8594dba 100644 46 --- a/drivers/media/platform/xilinx/xilinx-demosaic.c 47 +++ b/drivers/media/platform/xilinx/xilinx-demosaic.c 48 @@ -141,6 +141,7 @@ static bool 49 xdmsc_is_format_bayer(struct xdmsc_dev *xdmsc, u32 code) 50 { 51 switch (code) { 52 + case MEDIA_BUS_FMT_RBG888_1X24: 53 case MEDIA_BUS_FMT_SRGGB8_1X8: 54 xdmsc->bayer_fmt = XDEMOSAIC_RGGB; 55 break;
Setup the TPG to generate a pattern. Then run yavta to capture the frame. View this frame in a viewer like yuvplayer.exe
root@zcu102-zynqmp:/media# media-ctl -V '"a3c30000.v_tpg":0 [fmt:RBG24/1280x720 field:none]' root@zcu102-zynqmp:/media# ./tpg_diag_1080p_setup.sh 2 Device /dev/v4l-subdev2 opened. --- User Controls (class 0x00980001) --- control 0x0098c903 `Test Pattern: Color Mask' min 0 max 7 step 0 default 0 current 0. control 0x0098c907 `Test Pattern: Motion Speed' min 0 max 255 step 1 default 4 current 4. control 0x0098c908 `Test Pattern: Cross Hairs Row' min 0 max 4095 step 1 default 100 current 100. control 0x0098c909 `Test Pattern: Cross Hairs Colum' min 0 max 4095 step 1 default 100 current 100. control 0x0098c90a `Test Pattern: Zplate Horizontal' min 0 max 65535 step 1 default 30 current 30. control 0x0098c90b `Test Pattern: Zplate Horizontal' min 0 max 65535 step 1 default 0 current 0. control 0x0098c90c `Test Pattern: Zplate Vertical S' min 0 max 65535 step 1 default 1 current 1. control 0x0098c90d `Test Pattern: Zplate Vertical S' min 0 max 65535 step 1 default 0 current 0. control 0x0098c90e `Test Pattern: Box Size' min 0 max 4095 step 1 default 50 current 50. control 0x0098c90f `Test Pattern: Box Color(RGB/YCb' min 0 max 16777215 step 1 default 0 current 0. control 0x0098c912 `Test Pattern: Foreground Patter' min 0 max 2 step 1 default 0 current 0. 0: No Overlay (*) 1: Moving Box 2: Cross Hairs --- Image Source Controls (class 0x009e0001) --- control 0x009e0901 `Vertical Blanking' min 3 max 8159 step 1 default 100 current 100. control 0x009e0902 `Horizontal Blanking' min 3 max 8159 step 1 default 100 current 100. --- Image Processing Controls (class 0x009f0001) --- control 0x009f0903 `Test Pattern' min 0 max 16 step 1 default 9 current 9. 1: Horizontal Ramp 2: Vertical Ramp 3: Temporal Ramp 4: Solid Red 5: Solid Green 6: Solid Blue 7: Solid Black 8: Solid White 9: Color Bars (*) 10: Zone Plate 11: Tartan Color Bars 12: Cross Hatch 13: Color Sweep 14: Vertical/Horizontal Ramps 15: Black/White Checker Board 16: PseudoRandom 14 controls found. Device /dev/v4l-subdev2 opened. Control 0x009e0901 set to 45, is 45 Device /dev/v4l-subdev2 opened. Control 0x009f0903 set to 14, is 14 Device /dev/v4l-subdev2 opened. Control 0x0098c912 set to 1, is 1 Device /dev/v4l-subdev2 opened. Control 0x0098c90f set to 8453889, is 8453889 Device /dev/v4l-subdev2 opened. --- User Controls (class 0x00980001) --- control 0x0098c903 `Test Pattern: Color Mask' min 0 max 7 step 0 default 0 current 0. control 0x0098c907 `Test Pattern: Motion Speed' min 0 max 255 step 1 default 4 current 4. control 0x0098c908 `Test Pattern: Cross Hairs Row' min 0 max 4095 step 1 default 100 current 100. control 0x0098c909 `Test Pattern: Cross Hairs Colum' min 0 max 4095 step 1 default 100 current 100. control 0x0098c90a `Test Pattern: Zplate Horizontal' min 0 max 65535 step 1 default 30 current 30. control 0x0098c90b `Test Pattern: Zplate Horizontal' min 0 max 65535 step 1 default 0 current 0. control 0x0098c90c `Test Pattern: Zplate Vertical S' min 0 max 65535 step 1 default 1 current 1. control 0x0098c90d `Test Pattern: Zplate Vertical S' min 0 max 65535 step 1 default 0 current 0. control 0x0098c90e `Test Pattern: Box Size' min 0 max 4095 step 1 default 50 current 50. control 0x0098c90f `Test Pattern: Box Color(RGB/YCb' min 0 max 16777215 step 1 default 0 current 8453889. control 0x0098c912 `Test Pattern: Foreground Patter' min 0 max 2 step 1 default 0 current 1. 0: No Overlay 1: Moving Box (*) 2: Cross Hairs --- Image Source Controls (class 0x009e0001) --- control 0x009e0901 `Vertical Blanking' min 3 max 8159 step 1 default 100 current 45. control 0x009e0902 `Horizontal Blanking' min 3 max 8159 step 1 default 100 current 100. --- Image Processing Controls (class 0x009f0001) --- control 0x009f0903 `Test Pattern' min 0 max 16 step 1 default 9 current 14. 1: Horizontal Ramp 2: Vertical Ramp 3: Temporal Ramp 4: Solid Red 5: Solid Green 6: Solid Blue 7: Solid Black 8: Solid White 9: Color Bars 10: Zone Plate 11: Tartan Color Bars 12: Cross Hatch 13: Color Sweep 14: Vertical/Horizontal Ramps (*) 15: Black/White Checker Board 16: PseudoRandom 14 controls found. root@zcu102-zynqmp:/media# yavta -n 3 -c10 -f RGB24 -s 1280x720 --skip 9 -F /dev/video0 Device /dev/video0 opened. Device `vcap_gama output 0' on `platform:vcap_gama:0' is a video output (without mplanes) device. Video format set: RGB24 (33424752) 1280x720 field none, 1 planes: * Stride 3840, buffer size 2764800 Video format: RGB24 (33424752) 1280x720 field none, 1 planes: * Stride 3840, buffer size 2764800 3 buffers requested. length: 1 offset: 3944269120 timestamp type/source: mono/EoF Buffer 0/0 mapped at address 0x7f97c11000. length: 1 offset: 3944269120 timestamp type/source: mono/EoF Buffer 1/0 mapped at address 0x7f9796e000. length: 1 offset: 3944269120 timestamp type/source: mono/EoF Buffer 2/0 mapped at address 0x7f976cb000. 0 (0) [-] none 0 0 B 76.889014 76.889033 0.488 fps ts mono/EoF 1 (1) [-] none 1 0 B 77.912628 77.912645 0.977 fps ts mono/EoF 2 (2) [-] none 2 0 B 78.936243 78.936258 0.977 fps ts mono/EoF 3 (0) [-] none 3 0 B 79.959858 79.959873 0.977 fps ts mono/EoF 4 (1) [-] none 4 0 B 80.983388 80.983490 0.977 fps ts mono/EoF 5 (2) [-] none 5 0 B 82.007087 82.007104 0.977 fps ts mono/EoF 6 (0) [-] none 6 0 B 83.030702 83.030719 0.977 fps ts mono/EoF 7 (1) [-] none 7 0 B 84.054317 84.054334 0.977 fps ts mono/EoF 8 (2) [-] none 8 0 B 85.077921 85.078025 0.977 fps ts mono/EoF 9 (0) [-] none 9 0 B 86.101451 86.101468 0.977 fps ts mono/EoF Captured 10 frames in 11.259807 seconds (0.888115 fps, 0.000000 B/s). 3 buffers released. root@zcu102-zynqmp:/media#
The output looks as follows
Boards Supported
Driver has been tested on following boards:
- ZCU102 Rev 1.0
Known Issues
- AR68769 - LogiCORE Sensor Demosaic - Release Notes and Known Issues for the Vivado 2017.3 tool and later versions
Changelog
2024.2
- No changes
2024.1
- No changes
2023.2
- No changes
2023.1
- No changes
2022.2
- No changes
2022.1
- No changes
2021.2
- No changes
2021.1
- No changes
2020.2
- Summary
- Fix Coverity warnings
- Commits
2020.1
- Summary
- No changes
2019.2
- Summary
- No changes
2019.1
- Summary
- Adds support for 10,12,16 bpc RAW and RGB media bus formats.
- Make max width and height properties mandatory
- Commits
2018.3
- Summary
- Added xlnx,max-width and xlnx,max-height dt properties
- Added checks to set the correct default and max width and height from dt properties
- Commits
2018.2
- Summary
- No change
2018.1
- Summary
- Remove the redundant output format register
- Commits
- 6170785 v4l: xilinx-demosaic: Remove Output Video format register
2017.4
- Summary
- No change
2017.3
- Summary
- Initial version added to master branch
- Commits
- 50dbfdb v4l: xilinx-demosaic: driver support for xilinx video demosaic