Overview

Video encoder rate-control works based on previous history and stats gathered during the encoding session, whenever scene change happens the complexity of the frame changes and previously calculated stats may not hold good. The rate-control take a longer time to adjust to new content. Scene change detection is needed by Video encoder to know if it needs to update the reference frame for better performance. If the source provides scene change detection flag along with input buffer, encoder rate control can quickly adjust its stats and prevents overshoot/undershoot in the bitrate.

Video Scene change detection(SCD) IP provides a video processing block that implements a scene change detection algorithm. The IP core calculates histogram on a vertically subsampled luma frame for consecutive frames. The histogram of these frames is then compared using the sum of absolute difference(SAD).

IP/Driver Features

IP features	2019.1 or higher
IP version	1.0
One, two or four pixel-wide AXI4-Stream video interface	Yes*
Video resolution support up to 3840x2160 at 60 fps	Yes
8, 10, 12, and 16 bits per component support	Yes** Driver supports only 8 and 10 bpc formats using 8 and 10 bpc media bus formats
YUV 420, YUV 422, YUV 444 and RGB	Driver supports the listed color formats

Missing features / Known issues / Limitations in driver

- Streaming SCD: The SCD event is not aligned with buffer in case if the consumer (encoder) is slow.

Kernel Configuration

CONFIG_VIDEO_XILINX_SCD and CONFIG_VIDEO_XILINX should be enabled.

Device Tree Binding

The dts node should be defined with correct hardware configuration. How to define the node is documented here:
https://gitenterprise.xilinx.com/Linux/linux-xlnx/blob/master/Documentation/devicetree/bindings/media/xilinx/xlnx%2Cv-scd.txt

Testing Procedure

This driver is a V4L2 sub-device driver, which means it is designed to operate with a video device in it's graph. The driver has been tested with a Xilinx video device (a V4L2 capture device) that is backed by
a DMA element (like Framebuffer Write IP). The V4L2 device driver for this IP cannot be tested in isolation by itself. The successful operation of the driver can be tested in two phases :

Successful Creation of a Media Device
Successful Frame Captures (or writing of buffers by the Frame-buffer Write IP)

Successful Creation of a Media Device

media-ctl is a Linux user space utility that can control media entities, it is a part of a larger set of V4L2 tools called v4l-utils
media-ctl allows you to print the topology of your media pipeline and specify input and output pad properties.

Successful Frame Captures

Please see the Test Approach section of the Frame Buffer Write Wiki for more details on how to capture frames.

The driver has been tested with a design having HDMI-RX feeding video to SCD connected to Framebuffer Write.

Streaming based SCD

/* Streaming SCD: Hdmi Rx -> scd -> DMA */
xmedia-ctl -d /dev/media0 -V "\"xlnx-scdchan.0\":0 [fmt:VYYUYY8_1X24/3840x2160 field:none]"
xmedia-ctl -d /dev/media0 -V "\"xlnx-scdchan.0\":1 [fmt:VYYUYY8_1X24/3840x2160 field:none]"
yavta -n3 -c15 -f  RGB24 -s3840x2160 --skip 12 -F /dev/video0

This media-ctl and yavta commands capture frames are processed for scene change for 3840x2160 on NV12 format.

All color space conversions have been tested.

Memory based SCD

/* Memory SCD: VCU-decoder -> SCD -> VCU-Encoder */
gst-launch-1.0 filesrc location=input_4k.mp4 ! qtdemux ! h264parse ! omxh264dec ! queue ! xilinxscd io-mode=5 ! queue ! omxh264enc target-bitrate=20000 control-rate=2 cpb-size=5000 ! filesink location=output_4k.mp4

Scene change is tested for Formats that are listed below:

Sr No.	Video Input Resolution
1.	720x480
2.	1280x720
3.	1920x1080
4.	3840x2160

Boards Supported

Driver has been tested on following boards:

ZCU106 Rev 1.0

Known Issues

AR70293 - Video Scene Change Detection - Release Notes and Known Issues for the Vivado 2018.3 tool and later versions

Change log

Xilinx Wiki

Video Scene Change Detection(SCD)

Table of Contents