Video Scene Change Detection(SCD)
Table of Contents
Overview
Video encoder rate-control works based on previous history and stats gathered during the encoding session, whenever scene change happens the complexity of the frame changes and previously calculated stats may not hold good. The rate-control take a longer time to adjust to new content. Scene change detection is needed by Video encoder to know if it needs to update the reference frame for better performance. If the source provides scene change detection flag along with input buffer, encoder rate control can quickly adjust its stats and prevents overshoot/undershoot in the bitrate.
Video Scene change detection(SCD) IP provides a video processing block that implements a scene change detection algorithm. The IP core calculates histogram on a vertically subsampled luma frame for consecutive frames. The histogram of these frames is then compared using the sum of absolute difference(SAD).
IP/Driver Features
IP features | 2019.1 or higher | |
---|---|---|
IP version | 1.0 | |
One, two or four pixel-wide AXI4-Stream video interface | Yes* | |
Video resolution support up to 3840x2160 at 60 fps | Yes | |
8, 10, 12, and 16 bits per component support | Yes** Driver supports only 8 and 10 bpc formats using 8 and 10 bpc media bus formats | |
YUV 420, YUV 422, YUV 444 and RGB | Driver supports the listed color formats |
Missing features / Known issues / Limitations in driver
- Streaming SCD: The SCD event is not aligned with buffer in case if the consumer (encoder) is slow.
Kernel Configuration
Device Tree Binding
The dts node should be defined with correct hardware configuration. How to define the node is documented here:https://gitenterprise.xilinx.com/Linux/linux-xlnx/blob/master/Documentation/devicetree/bindings/media/xilinx/xlnx%2Cv-scd.txt
Testing Procedure
This driver is a V4L2 sub-device driver, which means it is designed to operate with a video device in it's graph. The driver has been tested with a Xilinx video device (a V4L2 capture device) that is backed bya DMA element (like Framebuffer Write IP). The V4L2 device driver for this IP cannot be tested in isolation by itself. The successful operation of the driver can be tested in two phases :
- Successful Creation of a Media Device
- Successful Frame Captures (or writing of buffers by the Frame-buffer Write IP)
Successful Creation of a Media Device
media-ctl is a Linux user space utility that can control media entities, it is a part of a larger set of V4L2 tools called v4l-utilsmedia-ctl allows you to print the topology of your media pipeline and specify input and output pad properties.
Successful Frame Captures
Streaming based SCD
/* Streaming SCD: Hdmi Rx -> scd -> DMA */ xmedia-ctl -d /dev/media0 -V "\"xlnx-scdchan.0\":0 [fmt:VYYUYY8_1X24/3840x2160 field:none]" xmedia-ctl -d /dev/media0 -V "\"xlnx-scdchan.0\":1 [fmt:VYYUYY8_1X24/3840x2160 field:none]" yavta -n3 -c15 -f RGB24 -s3840x2160 --skip 12 -F /dev/video0
This media-ctl and yavta commands capture frames are processed for scene change for 3840x2160 on NV12 format.
Memory based SCD
/* Memory SCD: VCU-decoder -> SCD -> VCU-Encoder */ gst-launch-1.0 filesrc location=input_4k.mp4 ! qtdemux ! h264parse ! omxh264dec ! queue ! xilinxscd io-mode=5 ! queue ! omxh264enc target-bitrate=20000 control-rate=2 cpb-size=5000 ! filesink location=output_4k.mp4
Sr No. | Video Input Resolution |
1. | 720x480 |
2. | 1280x720 |
3. | 1920x1080 |
4. | 3840x2160 |
Boards Supported
- ZCU106 Rev 1.0
Known Issues
- AR70293 - Video Scene Change Detection - Release Notes and Known Issues for the Vivado 2018.3 tool and later versions
Change log
2019.2
- Summary
- v4l: xilinx: scd: Fixed resolution switch issue
- v4l: xilinx: scd: Proper unregister from V4L2 framework
- v4l: xilinx: scd: Enable configuration of scene change threshold
- Commits
2019.1
- Summary
- v4l: xilinx: scd: Rewrite core start/stop and interrupt handling
- v4l: xilinx: scd: Don't mark channel as disabled in .s_stream(0)
- v4l: xilinx: scd: Rewrite core start/stop and interrupt handling
- v4l: xilinx: scd: Rewrite core start/stop and interrupt handling
- Commits
https://gitenterprise.xilinx.com/Linux/linux-xlnx/commit/45d6c73a3be65c72ec3a5559ccfb191d66b385f2
https://gitenterprise.xilinx.com/Linux/linux-xlnx/commit/9ca255ad4f8578c8d95f91a2a88a71f7a01bbae9
- https://gitenterprise.xilinx.com/Linux/linux-xlnx/commit/45d6c73a3be65c72ec3a5559ccfb191d66b385f2
- https://gitenterprise.xilinx.com/Linux/linux-xlnx/commit/45d6c73a3be65c72ec3a5559ccfb191d66b385f2
2018.3
- Summary
- xilinx: v4l: scd: Implement generic event notification
- xilinx: v4l: scd: Validate interrupt number
- Commits
Related Links
© Copyright 2019 - 2022 Xilinx Inc. Privacy Policy