Xilinx V4L2 Demosaic driver

Table of Contents

Introduction

The purpose of this page is to describe the Linux V4L2 driver for Xilinx Sensor Demosaic soft IP.
To understand more details about the Sensor Demosaic IP please see the Product Guide below.
The Linux V4L2 Demosaic driver is based on the V4L2 framework, and creates a subdev node(/dev/v4l-subdev*) which can be used to configure the Demosaic IP core.
The Xilinx V4L2 Demosaic driver is able to control the Sensor Demosaic Soft IP that can convert a Bayer video stream to RGB video stream.
The driver comes up with default color format on sink pad as MEDIA_BUS_FMT_SRGGB8_1X8 and on source pad as MEDIA_BUS_FMT_RBG888_1X24.
The Source Pad (output) media bus format cannot be changed through media-ctl. However, the Sink pad's (input) media bus format can be changed to any Bayer format.


entity 16: a0060000.demosaic (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev4
pad0: Sink
[fmt:SRGGB8/1280x720 field:none]
<- "a3c30000.tpg":0 [ENABLED]
pad1: Source
[fmt:RBG24/1280x720 field:none]
-> "a0050000.gamma":0 [ENABLED]

IP / Driver Features

IP Feature2018.1 to 2018.32019.1 to 2020.2
IP version1.01.0
1 / 2 / 4 / 8 pixels per clock supportYesYes
8 / 10 / 12 / 16 bits per componentOnly 8 bpc supported8 / 10 / 12 / 16 bpc support
Resolutions from 64x64 to 8192x4320Yes*
Supports 4Kp60 in all supported device familiesYes
Two interpolation algorithms High resolution and Fringe tolerantNA for driver
Ability to remove horizontal zipper artifactsNA for driver
Use UltraRAM for line buffersNA for driver

*Tested for 8192x4320, 3840x2160, 1920x1080, 1280x720


Kernel Configuration

CONFIG_VIDEO_XILINX_DEMOSAIC and CONFIG_VIDEO_XILINX should be enabled.

Device tree binding

The dts node should be defined with correct hardware configuration. How to define the DT node is documented here :
https://github.com/Xilinx/linux-xlnx/blob/master/Documentation/devicetree/bindings/media/xilinx/xlnx%2Cv-demosaic.txt


Testing Procedure

This driver is a V4L2 sub-device driver, which means it is designed to operate with a video device in it's graph. The driver has been tested with a Xilinx video device (a V4L2 capture device) that is backed by
a DMA element (like Framebuffer Write IP). The V4L2 device driver for this IP cannot be tested in isolation by itself. The successful operation of the driver can be tested in two phases :
  • Successful Creation of a Media Device
  • Successful Frame Captures (or writing of buffers by the Frame-buffer Write IP)

Successful Creation of a Media Device

media-ctl is a Linux user space utility that can control media entities, it is a part of a larger set of V4L2 tools called v4l-utils
media-ctl allows you to print the topology of your media pipeline and specify input and output pad properties.

Successful Frame Captures

Please see the Test Approach section of the Frame Buffer Write Wiki for more details on how to capture frames.
Test Design

We need a ZCU102 Rev 1.0 board.  Enable TPG, Demosaic, Gamma LUT and Framebuffer Write in kernel defconfig.

Modify the demosaic driver as below as TPG output is RGB type and Demosaic requires SRGGB type.

	 44 diff --git a/drivers/media/platform/xilinx/xilinx-demosaic.c b/drivers/media/platform/xilinx/xilinx-demosaic.c     
     45 index 505fc14..8594dba 100644
     46 --- a/drivers/media/platform/xilinx/xilinx-demosaic.c
     47 +++ b/drivers/media/platform/xilinx/xilinx-demosaic.c
     48 @@ -141,6 +141,7 @@ static bool
     49  xdmsc_is_format_bayer(struct xdmsc_dev *xdmsc, u32 code)
     50  {
     51         switch (code) {
     52 +       case MEDIA_BUS_FMT_RBG888_1X24:
     53         case MEDIA_BUS_FMT_SRGGB8_1X8:
     54                 xdmsc->bayer_fmt = XDEMOSAIC_RGGB;
     55                 break;

Setup the TPG to generate a pattern. Then run yavta to capture the frame. View this frame in a viewer like yuvplayer.exe

root@zcu102-zynqmp:/media# media-ctl -V '"a3c30000.v_tpg":0 [fmt:RBG24/1280x720 field:none]'
root@zcu102-zynqmp:/media# ./tpg_diag_1080p_setup.sh 2
Device /dev/v4l-subdev2 opened.
--- User Controls (class 0x00980001) ---
control 0x0098c903 `Test Pattern: Color Mask' min 0 max 7 step 0 default 0 current 0.
control 0x0098c907 `Test Pattern: Motion Speed' min 0 max 255 step 1 default 4 current 4.
control 0x0098c908 `Test Pattern: Cross Hairs Row' min 0 max 4095 step 1 default 100 current 100.
control 0x0098c909 `Test Pattern: Cross Hairs Colum' min 0 max 4095 step 1 default 100 current 100.
control 0x0098c90a `Test Pattern: Zplate Horizontal' min 0 max 65535 step 1 default 30 current 30.
control 0x0098c90b `Test Pattern: Zplate Horizontal' min 0 max 65535 step 1 default 0 current 0.
control 0x0098c90c `Test Pattern: Zplate Vertical S' min 0 max 65535 step 1 default 1 current 1.
control 0x0098c90d `Test Pattern: Zplate Vertical S' min 0 max 65535 step 1 default 0 current 0.
control 0x0098c90e `Test Pattern: Box Size' min 0 max 4095 step 1 default 50 current 50.
control 0x0098c90f `Test Pattern: Box Color(RGB/YCb' min 0 max 16777215 step 1 default 0 current 0.
control 0x0098c912 `Test Pattern: Foreground Patter' min 0 max 2 step 1 default 0 current 0.
  0: No Overlay (*)
  1: Moving Box
  2: Cross Hairs
--- Image Source Controls (class 0x009e0001) ---
control 0x009e0901 `Vertical Blanking' min 3 max 8159 step 1 default 100 current 100.
control 0x009e0902 `Horizontal Blanking' min 3 max 8159 step 1 default 100 current 100.
--- Image Processing Controls (class 0x009f0001) ---
control 0x009f0903 `Test Pattern' min 0 max 16 step 1 default 9 current 9.
  1: Horizontal Ramp
  2: Vertical Ramp
  3: Temporal Ramp
  4: Solid Red
  5: Solid Green
  6: Solid Blue
  7: Solid Black
  8: Solid White
  9: Color Bars (*)
  10: Zone Plate
  11: Tartan Color Bars
  12: Cross Hatch
  13: Color Sweep
  14: Vertical/Horizontal Ramps
  15: Black/White Checker Board
  16: PseudoRandom
14 controls found.
Device /dev/v4l-subdev2 opened.
Control 0x009e0901 set to 45, is 45
Device /dev/v4l-subdev2 opened.
Control 0x009f0903 set to 14, is 14
Device /dev/v4l-subdev2 opened.
Control 0x0098c912 set to 1, is 1
Device /dev/v4l-subdev2 opened.
Control 0x0098c90f set to 8453889, is 8453889
Device /dev/v4l-subdev2 opened.
--- User Controls (class 0x00980001) ---
control 0x0098c903 `Test Pattern: Color Mask' min 0 max 7 step 0 default 0 current 0.
control 0x0098c907 `Test Pattern: Motion Speed' min 0 max 255 step 1 default 4 current 4.
control 0x0098c908 `Test Pattern: Cross Hairs Row' min 0 max 4095 step 1 default 100 current 100.
control 0x0098c909 `Test Pattern: Cross Hairs Colum' min 0 max 4095 step 1 default 100 current 100.
control 0x0098c90a `Test Pattern: Zplate Horizontal' min 0 max 65535 step 1 default 30 current 30.
control 0x0098c90b `Test Pattern: Zplate Horizontal' min 0 max 65535 step 1 default 0 current 0.
control 0x0098c90c `Test Pattern: Zplate Vertical S' min 0 max 65535 step 1 default 1 current 1.
control 0x0098c90d `Test Pattern: Zplate Vertical S' min 0 max 65535 step 1 default 0 current 0.
control 0x0098c90e `Test Pattern: Box Size' min 0 max 4095 step 1 default 50 current 50.
control 0x0098c90f `Test Pattern: Box Color(RGB/YCb' min 0 max 16777215 step 1 default 0 current 8453889.
control 0x0098c912 `Test Pattern: Foreground Patter' min 0 max 2 step 1 default 0 current 1.
  0: No Overlay
  1: Moving Box (*)
  2: Cross Hairs
--- Image Source Controls (class 0x009e0001) ---
control 0x009e0901 `Vertical Blanking' min 3 max 8159 step 1 default 100 current 45.
control 0x009e0902 `Horizontal Blanking' min 3 max 8159 step 1 default 100 current 100.
--- Image Processing Controls (class 0x009f0001) ---
control 0x009f0903 `Test Pattern' min 0 max 16 step 1 default 9 current 14.
  1: Horizontal Ramp
  2: Vertical Ramp
  3: Temporal Ramp
  4: Solid Red
  5: Solid Green
  6: Solid Blue
  7: Solid Black
  8: Solid White
  9: Color Bars
  10: Zone Plate
  11: Tartan Color Bars
  12: Cross Hatch
  13: Color Sweep
  14: Vertical/Horizontal Ramps (*)
  15: Black/White Checker Board
  16: PseudoRandom
14 controls found.
root@zcu102-zynqmp:/media# yavta -n 3 -c10 -f RGB24 -s 1280x720 --skip 9 -F /dev/video0
Device /dev/video0 opened.
Device `vcap_gama output 0' on `platform:vcap_gama:0' is a video output (without mplanes) device.
Video format set: RGB24 (33424752) 1280x720 field none, 1 planes: 
 * Stride 3840, buffer size 2764800
Video format: RGB24 (33424752) 1280x720 field none, 1 planes: 
 * Stride 3840, buffer size 2764800
3 buffers requested.
length: 1 offset: 3944269120 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x7f97c11000.
length: 1 offset: 3944269120 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x7f9796e000.
length: 1 offset: 3944269120 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x7f976cb000.
0 (0) [-] none 0 0 B 76.889014 76.889033 0.488 fps ts mono/EoF
1 (1) [-] none 1 0 B 77.912628 77.912645 0.977 fps ts mono/EoF
2 (2) [-] none 2 0 B 78.936243 78.936258 0.977 fps ts mono/EoF
3 (0) [-] none 3 0 B 79.959858 79.959873 0.977 fps ts mono/EoF
4 (1) [-] none 4 0 B 80.983388 80.983490 0.977 fps ts mono/EoF
5 (2) [-] none 5 0 B 82.007087 82.007104 0.977 fps ts mono/EoF
6 (0) [-] none 6 0 B 83.030702 83.030719 0.977 fps ts mono/EoF
7 (1) [-] none 7 0 B 84.054317 84.054334 0.977 fps ts mono/EoF
8 (2) [-] none 8 0 B 85.077921 85.078025 0.977 fps ts mono/EoF
9 (0) [-] none 9 0 B 86.101451 86.101468 0.977 fps ts mono/EoF
Captured 10 frames in 11.259807 seconds (0.888115 fps, 0.000000 B/s).
3 buffers released.
root@zcu102-zynqmp:/media# 

The output looks as follows

Boards Supported

Driver has been tested on following boards:
  • ZCU102 Rev 1.0

Known Issues

  • AR68769 - LogiCORE Sensor Demosaic - Release Notes and Known Issues for the Vivado 2017.3 tool and later versions

Changelog

2023.2

  • No changes

2023.1

  • No changes

2022.2

  • No changes

2022.1

  • No changes

2021.2

  • No changes

2021.1

  • No changes

2020.2

2020.1

  • Summary
    • No changes

2019.2

  • Summary
    • No changes

2019.1

  • Summary
    • Adds support for 10,12,16 bpc RAW and RGB media bus formats.
    • Make max width and height properties mandatory
  • Commits
    • 6d7de7c v4l: xilinx-demosaic: Make max width and height properties mandatory
    • 899bb51 v4l: xilinx-demosaic: Add support for 10,12,16 bpc RAW and RGB bus formats

2018.3

  • Summary
    • Added xlnx,max-width and xlnx,max-height dt properties
    • Added checks to set the correct default and max width and height from dt properties
  • Commits
    • 4d0d5f2 v4l: xilinx-demosaic: Correct default values when width, height dt property absent
    • 734e627 v4l: xilinx-demosaic: Add check for max width and height

2018.2

  • Summary
    • No change

2018.1

  • Summary
    • Remove the redundant output format register
  • Commits
    • 6170785 v4l: xilinx-demosaic: Remove Output Video format register

2017.4

  • Summary
    • No change

2017.3

  • Summary
    • Initial version added to master branch
  • Commits
    • 50dbfdb v4l: xilinx-demosaic: driver support for xilinx video demosaic


Related Links


© Copyright 2019 - 2022 Xilinx Inc. Privacy Policy