Introduction

The purpose of this page is to describe the Linux V4L2 driver for Xilinx MIPI Camera Serial Interface 2 Receiver subsystem (MIPI CSI2 Rx SS) soft IP. The Linux MIPI CSI2 Rx Subsystem driver (xilinx-csi2rxss.c) is based on the V4L2 framework, and creates a subdev node(/dev/v4l-subdev*) which can be used to configure the MIPI CSI2 Rx Subsystem IP core. The general description of V4L2 framework is documented here. This subdev driver has 2 ports. One sink port which is connected to the image sensor's source port and one source port. The media format applied on the sink port is passed on to the source port. The media format applied is validated based on the IP configuration like RAW8, RAW10, etc. The driver also exposes a V4L2 control to set the number of active lanes, if the feature is enabled in the IP configuration. It also exposes V4L2 control to read number of frames transferred and to reset the event counters.

The MIPI CSI-2 RX subsystem allows you to quickly create systems based on the MIPI protocol. It interfaces between MIPI-based image sensors and an image sensor pipe. An internal high speed physical layer design, D-PHY, is provided that allows direct connection to image sources.

IP/Driver Features

IP Features	2018.1	2018.2	2018.3	2019.1	2019.2	2020.1	2020.2	2021.1
IP version	3.0	3.0	4.0	4.0	4.1	5.0		5.1
Compatible String	xlnx,mipi-csi2-rx-subsystem-5.0 (from 2023.1 and above)
Support for 1 to 4 D-PHY lanes	NA
Line rates ranging from 80 to 1500 Mb/s	NA
Multiple Data Type support (RAW, RGB, YUV)	IP allows RAW6/7/8/10/12/14, all RGB and YUV 422 8bpc Driver allows to set any format except when RAW10 and RAW12.			IP allows RAW6/7/8/10/12/14/16/20, all RGB and YUV 422 8/10 bpc Driver allows to set any format except when RAW10, RAW12 and RAW16			Added YUV 420 8 bpc support
AXI IIC support for Camera Control Interface (CCI)	NA					Removed
Filtering based on Virtual Channel Identifier	NA
Support for 1, 2, or 4 pixels per sample at the output	Yes**
AXI4-Lite interface for register access to configure different subsystem options	Yes
Dynamic selection of active lanes within the configured lanes during subsystem generation	Yes
Interrupt generation to indicate subsystem status information	Yes
Internal D-PHY allows direct connection to image sources	Yes
Resource optimization (removed register interface)	NA				No

*Only RAW8/10/12/16 media bus formats are tested.

**Tested for 1 and 2 pixels per sample

Update for 2021.1

Since 2021.1, the upstream driver is being used. This driver doesn't have any of the v4l2 controls as earlier.

Now pad 0 is sink pad and pad 1 is source pad. It uses the gpio connected to video-aresetn pin to reset.

In case of buffer overflow, the core is disabled.

It enumerates the bus formats. Only the sink pad format can be set. When trying to set source format, driver will return the format set on sink pad.

The driver just reads and prints the short packet data now to debug log when the Short packet FIFO is not empty.

It still keeps a counter of all types of events and prints them on log status.

Now on stream on, the driver not only enables itself but calls the stream on function of the driver for source element (reliance on Xilinx video pipeline is reduced).

On stopping stream, the core is disabled and a hard reset via gpio connected to video-aresetn pin is given to core.

Missing Features / Known Issues / Limitations in Driver

If Stream line buffer full prints / events come, the design needs to be checked.
There is no separate DPHY Phy driver.
When DPHY register interface is enabled, the driver only enables or disables the DPHY. It doesn't support modification of other DPHY parameters.
The driver is tested with fixed clocks.
The driver has always been tested with Video Format Bridge enabled.
Power suspend / resume are not tested.
Resource optimized IP (no register interface) not supported.

Other features supported in driver are -

Configure the number of active lanes. (removed in upstream)
Notification on reception of short packets. (removed in upstream)
Getting short packet data using new event type. (removed in upstream)
Notification on short packet FIFO overflow. (removed in upstream)
Notification on stream line buffer overflow. (removed in upstream, driver stops IP so system is not hung and pipeline can be restarted)
Get the number of frames received since streaming is enabled.
Print driver counters keeping track of number of interrupt / error events

MIPI CSI-2 Rx Subsystem IP Release Notes and Known Issues

MIPI CSI-2 Receiver Subsystem - Release Notes and Known Issues for the Vivado 2015.3 tool and later versions

Kernel Configuration

The following configs need to be enabled

CONFIG_VIDEO_XILINX

CONFIG_VIDEO_XILINX_CSI2RXSS

Device Tree Binding

The dts node should be defined with correct hardware configuration.

How to define the node is documented here, xlnx,csi2rxss.yaml

This driver is tested with IMX274 sensor based on MIPI DPHY protocol. The pipeline under test looks like

IMX274 sensor→ MIPI CSI2 RX→ Demosai→ GammaLUT→ VPSS CSC→ VPSS scaler→ framebuffer WR→ memory→ framebuffer RD→ MIPI DSI TX .

User need to refer attached system-user.dtsi which should be used as an overlay on top of generated pl.dtsi to point to sensor node and to point

sensor node as remote-endpoint to MIPI CSI2 Rx Subsystem node. This system.dts is an example of how exactly final device tree should look like.

Test Procedure

A video pipeline with MIPI CSI2 Rx connected to Demosaic, Gamma LUT, VPSS CSC, VPSS Scaler and Framebuffer Write IP is created for ZCU102 board.

An IMX74 sensor FMC card is used to capture image and send CSI stream to MIPI CSI 2 Rx Subsystem.

media-ctl is used to set the color format on the pads.

#Assuming the media device /dev/media1 has MIPI CSI2 Rx Subsystem entity name whose name is "a00f0000.csiss"
#To set RGGB RAW8 bayer data of size 1920x1080 on source pad
media-ctl -v -V '"a00f0000.csiss":0 [fmt:SRGGB8_1X8/1920x1080 field:none colorspace:srgb]' -d /dev/media1

Using v4l2-ctl to set the number of active lanes

#Assume MIPI CSI2 Rx subsystem which is registered as /dev/v4l-subdev1
#To set the number of active lanes as 4
v4l2-ctl -c mipi_csi2_rx_subsystem_active=4 -d /dev/v4l-subdev1

Using yavta to set the number of active lanes

#Assume MIPI CSI2 Rx subsystem which is registered as /dev/v4l-subdev1
#To set the number of active lanes as 4 (NA after 2020.2)
yavta -w '0x0098c981 4' /dev/v4l-subdev1

#To see all the controls
yavta -l /dev/v4l-subdev1

#Need to setup the media pipeline and start capture of video stream
media-ctl -v -V '"IMX274":0 [fmt:SRGGB8/1920x1080@1/60 field:none colorspace:srgb]' -d /dev/media1
media-ctl -v -V '"a00f0000.csiss":0 [fmt:SRGGB8_1X8/1920x1080 field:none colorspace:srgb]' -d /dev/media1
media-ctl -v -V '"a0250000.v_demosaic":1 [fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]' -d /dev/media1
media-ctl -v -V '"a0270000.v_gamma":1 [fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]' -d /dev/media1
media-ctl -v -V '"a0240000.csc":1 [fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]' -d /dev/media1
media-ctl -v -V '"a0200000.scaler":0 [fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]' -d /dev/media1
media-ctl -v -V '"a0200000.scaler":1 [fmt:UYVY/1920x1080 field:none colorspace:srgb]' -d /dev/media1

#Now start streaming
yavta -n 3 -c10 -f UYVY -s 1920x1080 --skip 7 -F /dev/video4

A custom application was implemented to test getting the short packets and other events using poll() on file descriptor returned on open() of the v4l subdev.

This was done as poll() mechanism on sub device wasn't present in any standard application.

Boards Supported

Driver has been tested on following boards:

ZCU102 Rev 1.0

Known Issues

AR67896 - MIPI CSI-2 Transmitter Subsystem - Release Notes and Known Issues for the Vivado 2016.3 tool and later versions

Change log

2024.2

No changes

2024.1

No changes

2023.2

Summary
- Add support for YUV420 8 BPC
Commit
- v4l: xilinx: xcsirxss: Fix YUV420 format

2023.1

No changes

2022.2

No changes

2022.1

No changes

2021.2

No changes

2019.2

Summary
- Add support to reset IP using external GPIO and stop streaming in case of stream line buffer full condition.
Commits
- c3e2148 v4l: xilinx: xcsi2rxss: Use external reset in a SLBF condition

2019.1

Summary
- Add support for common clock framework
- Add support for RAW16 format
Commits
- 9bd3a62 v4l: xilinx: xcsi2rxss: Add support for RAW16 format
- c3659f0 v4l: xilinx: xcsi2rxss: Add support for clock framework

2018.3

Summary
- Add support for up to 16 virtual channel VCX when enabled in IP configuration
- Fix to store format size to ensure link_validate passes.
Commits
- 3534b0f media: xilinx: csi2rxss: store format size
- f8ef0fc v4l: xilinx: xcsi2rxss: Add VCX support

2018.2

Summary
- No changes

2018.1

Summary
- Add xlnx,mipi-csi2-rx-subsystem-3.0 compatible string
- Fix compilation issue due to framework change
Commits
- 8b3a88f v4l: xilinx: csi2rxss: Add rev 3.0 compatible string
- 0323f75 v4l: xilinx: csi2rxss: Switch to fwnode to fix compilation error

2017.4

Summary
- No changes

2017.3

Summary
- Fix to handle failure case while creating a custom control
Commits
- 9921a92 v4l: xilinx: csi2rxss: Handle failing to create custom control

2017.2

Summary
- Fix a crash caused by update to events counters
Commit
- e3a7ef8 xcsi2rxss: Fix crash caused by update to event counter variable

2017.1

Summary
- Fix to support changing the bayer phase at run time
Commit
- 56e9ba9 xcsi2rxss: Support bayer phase change at run-time

2016.4

Summary
- No change

2016.3

Summary
- The driver is added
Commits
- 4db32c67 xcsi2rxss: Add support for Xilinx CSI-2 Receiver Subsystem

Xilinx Wiki

Xilinx V4L2 MIPI CSI driver

Table of Contents