The below sections describe the build and run flow tutorials of the DM4 - PL Video Capture and Display
design. For the overview, software tools and system requirements, and design files, follow the link below:
1.1 Board Setup
- Connect the Micro USB cable into the ZCU106 Board Micro USB port J83, and the other end into an open USB port on the host PC. This cable is used for UART over USB communication.
- Insert the SD card with the images copied into the SD card slot J100.
- Set the SW6 switches as shown in the below Figure. This configures the boot settings to boot from SD
- Connect 12V Power to the ZCU106 6-Pin Molex connector.
- Connect one end of Display Port (DP) cable to board’s U129 connector and the other end to DP port of 4K monitor.
- Connect one end of HDMI cable to board’s P7 stacked HDMI connector (lower port) and another end to HDMI source.
- Connect one end of HDMI cable to board’s P7 stacked HDMI connector (upper port) and another end to HDMI monitor.
- For USB storage device, connect USB hub along with the mouse. (Optional)
- For SATA storage device, connect SATA data cable to SATA 3.0 port. (Optional).
- For MIPI CSI-2, Insert the LI-IMX274MIPI-FMC image sensor daughter card into the FMC0 connector and set VADJ to 1.2V (See FMC VADJ Voltage Settings).
- Set up a terminal session between a PC COM port and the serial port on the evaluation board (See the Determine which COM to use to access the USB serial port on the ZCU106 board for more details).
- Switch ON SW1 to power the ZCU106 board.
The figure below shows the ZCU106 board connections for HDMI/MIPI design.
It takes about a minute for Linux to boot. The user may miss the boot sequence while the terminal session is being configured.FMC VADJ Voltage Settings
VADJ voltage needs to be set to 1.2V for LI-IMX274MIPI-FMC image sensor daughter card. Refer to AR# 67308
for setting VADJ value.Determine which COM to use to access the USB serial port on the ZCU106 board.
Make sure that the ZCU106 board is powered on and a micro USB cable is connected between ZCU106 board and host PC. This ensures that the USB-to-serial bridge is enumerated by the PC host.
Open your computer's Control Panel by clicking on Start > Control Panel
Note that the Start button is typically located in the lower left corner of the screen. Occasionally, it is in the upper left corner.
Launch any Terminal application like Tera term to view the serial messages
- Click Device Manager to open the Device Manager window. Note: You may be asked to confirm opening the Device Manager. If so, click YES.
- Expand Ports (COM & LPT).
- Locate the Silicon Labs Quad CP210x USB to UART Bridge: Interface 0 (COM#).
- Note down the COM Port number for further steps.
- Close the Device Manager by clicking the red X in the upper right corner of the window.
- Launch Tera Term and open the COM port that is associated to Silicon Labs Quad CP210x USB to UART Bridge: Interface 0 of the USB-to-serial bridge.
- Set the COM port to 115200 Baud rate, 8, none, 1 –Set COM port.
- Power ON the board which has SD card. It boots Linux on board.
1.2 Build Flow
The following tutorials assume that the $TRD_HOME environment variable is set as given below.NOTE
: It is recommended to follow the build steps in sequence.
1.2.1 Hardware Design
Refer to the Vivado Design Suite User Guide: Using the Vivado IDE
, UG893, for setting up Vivado environment.On Linux:
- Open a Linux terminal
- Change directory to $TRD_HOME/pl
- Apply 2018.1 Vivado patch related to HDMI from AR71203
- Run the following command in Vivado shell to create the Vivado IPI project and invoke the GUI for VCU TRD hardware design.
After executing the script, the Vivado IPI block design comes up as shown in the below figure.
- Click on “Generate Bitstream”.
If the user gets any pop-up with “No implementation Results available
”. Click “Yes
”. Then, if any pop-up comes up with “Launch runs
”, Click "OK
The design is implemented and a pop-up window comes up saying “Open Implemented Design
”. Click "OK
After opening implemented design, the window looks as shown in the below figure.Note:
The actual results might graphically look different than the image shown
- Go to File > Export > Export Hardware
- In the Export Hardware Platform for SDK window select "Include bitstream" and click "OK".
The HDF is created at $TRD_HOME/pl/vcu_trd_proj/zcu106_vcu_trd.sdk/zcu106_vcu_trd_wrapper.hdf
1.2.2 VCU PetaLinux BSP
This tutorial shows how to build the Linux image and boot image using the PetaLinux build tool.PetaLinux Installation
: Refer to the PetaLinux Tools Documentation
(UG1144) for installation.
Post PetaLinux installation $PETALINUX environment variable should be set.
- Configure and build the PetaLinux project for vcu trd design.
- If Vivado project is modified, the user is expected to configure the bsp with the modified .hdf file and build.
- Apply the following TMDS patches (Only for rev-b board) - Edit recipes-kernel/modules/kernel-module-hdmi_%.bbappend to include 0001-Subject-PATCH-xilinx-hdmi-rx-Add-HPD-hack-for-ZCU106.patch
- Build SDK components to use it as sysroot for the application development.
- Create a boot image (BOOT.BIN) including FSBL, ATF, bitstream, and u-boot.
- Copy the generated boot image and Linux image to the SD card directory. Replace rev-x with board revision
: rev-c binaries can be used for rev-d and later board versions.Custom EDID Support
The TRD design is tested/validated with ABOX 2017 and NVIDIA SHIELD Pro. If you wants to try with any new HDMI source, you need to generate the EDID of the new source and update it in the TRD bsp. Refer to Custom EDID Support
for adding the newly generated EDID file.
1.2.3 VCU APM library
This tutorial shows how to build the VCU APM library.
- Set the build environment variables. This requires that you have previously completed the PetaLinux SDK installation step.
- Note 1: Make sure you set the env variables in the same shell that is used to launch SDK. Also make sure the env variables are set before starting SDK, otherwise close and re-start SDK.
- Note 2: The below command might ask you execute unset LD_LIBRARY_PATH and then re-execute the command. Go ahead and follow those steps.
- Open the SDK workspace using the Xilinx SDK tool.
- From the menu bar, select File -> Import.., then select General -> Existing Projects into Workspace. Click Next.
- Browse to the $TRD_HOME/apu directory. Select “Deselect All” and then select vcu_apm_lib project. Click Finish.
- Right-click the vcu_apm_lib project and select 'Build Project'
1.2.4 VCU Video library
This tutorial shows how to build the VCU video library.
- From the menu bar, select File -> Import…, then select General -> Existing Projects into Workspace. Click Next.
- Browse to the $TRD_HOME/apu directory. Select “Deselect All” and select vcu_video_lib project. Click Finish.
- Right-click the vcu_video_lib project and select 'Build Project'
1.2.5 VCU GST library
This tutorial shows how to build the VCU GST library.
- From the menu bar, select File -> Import…, then select General -> Existing Projects into Workspace. Click Next.
- Browse to the $TRD_HOME/apu directory. Select “Deselect All” and select vcu_gst_lib project. Click Finish.
- Right-click the vcu_gst_lib project and select 'Build Project'
1.2.6 VCU GST APP
This tutorial shows how to build the VCU GST APP.
- From the menu bar, select File -> Import…, then select General -> Existing Projects into Workspace. Click Next.
- Browse to the $TRD_HOME/apu directory. Select “Deselect All” and select vcu_gst_app project. Click Finish.
- Right-click the vcu_gst_app project and select 'Build Project'
1.2.7 VCU Qt Application
This tutorial shows how to build the VCU Qt application.
- Source the Qt setup script to generate the Qt Makefile reflecting these changes.
- Copy the generated vcu_qt executable to the SD card directory.
1.3 Run Flow
Please note that Windows formatter can't be used and partition should be FAT32, not FAT.
- Copy all the files from the $TRD_HOME/images/rev-c SD card directory to a FAT32 formatted SD card. For ZCU106 Rev-b board, use $TRD_HOME/main/images/rev-b
- Power on the board; make sure INIT_B, done and all power rail LEDs are lit green.
- After ~30 seconds, the display will turn on and the application will start automatically, targeting the max supported resolution of the monitor (3840x2160).
- The user can now control the application from the GUI's control bar (top left) displayed on the monitor.
- The user can quit the application by clicking on exit [X] icon. To restart the application type run_vcu.sh on the shell.
: The SD card file system is mounted at /media/card. Optional storage medium SATA and USB are mounted at /media/sata and /media/usb respectively.
1.3.1 Graphic User Interface
A QT based graphical user interface (GUI) provides control and monitoring interface. By default, GUI will run on DP only.
Run the below command to launch GUI on Display port when connected.
The GUI is not supported for HDMI Tx. In HDMI 4kp60 mode, GUI requires additional ~2GB memory bandwidth. If you invoke the GUI cause memory bandwidth issue. So HDMI Tx, use only command line application (vcu_gst_app). Table below provides the supported display interfaces for both GUI and command line application.
|Display Interface||GUI||Command Line Application|
The figure below shows the home screen of DP when only DP is connected.
GUI only supports single stream playback. Supported resolutions are 1080p and 4kp. The supported frame rate for 1080p is 60 fps and 30fps for 4kp. For all multi-stream use case and 4kp60 single stream, the user has to run command line applications.
QT GUI has following options for the user to select:Number of InputThis determines the number of active video sources. Only one input source is supported.Output
This option allows the user to select sink for the pipeline. Supported output sink types is display port, record, and stream. For the Display Port , the available Codec options are either enc-dec or Pass-through. For Record and Stream Out, the available Codec option is Enc.Input Settings
The below figure shows the input settings window with configuration options.
Different configuration options available in the QT pop up window are:Input Source (Input 1)
The following video sources (4K) are available as part of input video selection:
- HDMI-Rx: GooBang Doo ABOX 2017 (Android 6.0 TV Box) with an optional splitter
- File: Mp4 streams reside in SD-Card/USB/SATA.
- Test Pattern Generator (TPG): Implemented in the PL
- MIPI: LI-IMX274 MIPI-FMC v1.1.
- SDI Receiver - Black Magic Teranex Mini HDMI to 12G converter
The following options are available as part of Codec functionality
- Enc-Dec: This option is to select encode and decode in the pipeline.
- Enc: This option is to select encode only in the pipeline for record or stream out use case.
- Pass-through: This option is to display raw video.
There are six predefined presets. If the user edits any control options preset mode switches to “Custom”.
Profile = Main, Rate Control= CBR, Filler Data = true, QP=uniform, L2 Cache = true, Latency mode = Normal, Low Bandwidth = true, Gop Mode = Basic, B-frame = 0, Slice = 4 and GoP Length = 30
- AVC Low Encoder type = H264, bitrate = 10Mbps
- AVC Medium Encoder type = H264, bitrate = 30Mbps
- AVC High Encoder type = H264, bitrate = 60Mbps
- HEVC Low Encoder type = H265, bitrate = 10Mbps
- HEVC Medium Encoder type = H265, bitrate = 30Mbps
- HEVC High Encoder type = H265, bitrate = 60Mbps
common for all above options.
- Custom User specific options
Please refer to the Supported Encoder Format
section for the supported encoder format in this release.Settings
A user can control the encoder, record and stream out configuration from the GUI, Settings option are enabled when the pipeline is in the stop state. User can control the encoder parameters when output option is selected as Display port/Record/Stream.Encoder
This can be either H264 or H265.Profile
The standard defines a set of capabilities, which are referred to as profiles, targeting specific classes of applications. These are declared as a profile code (profile_idc) and a set of constraints applied in the encoder. This allows a decoder to recognize the requirements to decode that specific stream.
H264 supports baseline, main and high profile. In H265 only main profile is supported.QP
Quantization in an encoder is controlled by a quantization parameter. It specifies how to generate the QP per coding unit (CU). Two modes are supported-
- Uniform: All CUs of the slice using the same QP
- Auto: VCU encoder changes the QP for each coding unit according to its content.
Constant bitrate (CBR), variable bitrate (VBR) and low-latency are supported. CBR mode generates a constant bitrate that you can predefine. CBR is recommended for limited bandwidth use cases.Bitrate
Encoding bit rate. In digital multimedia, bit rate often refers to the number of bits used per unit of playback time to represent a continuous medium such as audio or video after source coding (data compression).B-frame
Short for bi-directional frame, or bi-directional predictive frame, a video compression method used by the MPEG standard. It ranges from 0 to 4.Slice
The number of slices produced for each frame. Each slice contains one or more complete macroblock/CTU row(s). Slices are distributed over the frame as regularly as possible. If slice-size is defined as well more slices may be produced to fit the slice-size requirement
4-22 4kp resolution with HEVC codec
4-32 4kp resolution with AVC codec
4-32 1080p resolution with HEVC codec
4-32 1080p resolution with AVC codecGoP Length
In video coding, a group of pictures, or GOP structure, specifies the order in which intra and inter-frames are arranged. And GoP-length is a length between two Intra frames. The GOP is a collection of successive pictures within a coded video stream. Each coded video stream consists of successive GOPs, from which the visible frames are generated. Its range is from 1 to 1000. NOTE: the GoP-length must be a multiple of B-Frames+1 AR71253
Filler Data NAL units for CBR rate control to maintain constant bitrate throughout the playback. It can be enabled or disabled. It is applicable to CBR mode only.L2 Cache
Enable or Disable L2Cache buffer in encoding process.Latency Mode
Encoder latency mode. It can be normal or sub_frame mode.Low Bandwidth
If enabled, decrease the vertical search range used for P-frame motion estimation to reduce the bandwidth.Gop Mode
Group of Pictures mode. It can be Basic, low_delay_p, or low_delay_b.Record
When Output option is selected as Record, A tab will be enabled in settings window through which user can record the video into the storage medium.Storage
This option specifies storage device for the recorded file. The list is dynamically populated based on mounted storage devices. Supported storage devices are SD, USB, and SATA.Output File Name
Name of the output file. For H264, recorded file get saved as <Source name>_H264_rec_<timestamp>.mp4 and for H265 it will be <Source name>_H265_rec_<timestamp>.mp4Duration
This option will specify the Recording time duration. It ranges from 1-3 mins.NOTE
: Due to speed and storage constraints it is recommended to use USB/SATA storage devices for the record.Stream-out
The stream-out panel allows the user to configure streaming parameters.Stream-out Prerequisites:
- Host server machine (with GPU) capable of playing 4K60 stream.
- Setup Ethernet connection between ZCU106 board and the host machine. Ensure connection is alive by doing a ping from the board to host machine.
- VLC player installed on the host machine.
- Firewall is turned off on the host machine.
- Start VLC player on the host before streaming from GUI.
Provides sink option for the stream-out. It is set to PS Ethernet.Host IP
IP address of a computer or other Zynq® UltraScale+™ MPSoC board in which streaming video will be played on. It is set to 192.168.25.89 by default.IP
This option will show the IP address of the board if the Ethernet link is up. If no Ethernet link is connected then it will show “Not Connected”.Port
Port number of the Ethernet link. By default, it is set to 5004.Steps for Stream Out
Here are the steps to get the ZCU106 TRD to play out to VLC.
- Make sure that your machine can play 4Kp60 streams and turned off the Firewall.
- Connect your PC to the ZCU106 board. It is recommended a direct connection between ZCU106 board and PC to minimize the network traffic.
- Set the IP address for the ZCU106 board and PC to use the same subnet (i.e. ZCU106 - ifconfig eth0 192.168.25.90 and PC - 192.168.25.89).
- Create .sdp file on host. For H265 RTP client, add below configs in host_h265.sdp file.
- NOTE: For HEVC the 4th line will be H264 and for AVC the 4th line will be H264.
Depending on the network, change IP address and port number. IP address given in .sdp is ZCU106 board's IP. For H264 RTP client, change encoder to H264.
- Launch VLC on your PC and open the SDP file (Click on media -> Open File. Select host_h265.sdp file and click on open.)
- Setup output option to “Stream” in the ZCU106 VCU TRD GUI.
- Configure the Stream out settings and make sure that the ZCU106 VCU TRD GUI is pointing to the IP address of your PC.
- Select the HDMI-Rx/MIPI or the TPG as an input.
- Press play in VLC on your PC. (The Client must start first, but VLC times out if you wait too long before starting the ZCU106 VCU TRD Stream).
- Press play on the ZCU106 VCU TRD GUI
In video streaming use case it is recommended to set b-frame=0
1.3.2 GStreamer Application (vcu_gst_app)
The vcu_gst_app is a command line multi-threaded Linux application that uses the vcu_gst_lib interface similar to vcu_qt. The command line application requires an input configuration file (input.cfg) to be provided in the plain text. Refer to the Appendix A - Input Configuration File (input.cfg)
for the file format and description.
Before execution of vcu_gst_app
, we need to check the HDMI-Rx link status.
Refer Appendix B
for how to check the HDMI Link Status
- Execution of the application is shown below:
Required Storage medium SATA and USB are mounted at /media/sata and /media/usb respectively.
configuration files(input.cfg) for various Resolutions are placed in the following directory structure in /media/card/Ex:
4kp60 HEVC_HIGH Display Pipeline execution
- To measure the latency of the pipeline, run the below command. The latency data is huge, so dump it to a file.
When the number of input is more than one in the command line application, it is a multi-stream use case. The command line application supports multi-streaming, multi-recording, or multi-display.
The input source type can be HDMI and MIPI. For multi-streaming/multi-recording, the source type can be HDMI and MIPI.
2 Other Information
2.1 Supported Features
The table below provides the supported encoder feature in this release.
|Encoder Format||GUI||Command Line|
|Single Stream||Single Stream||Multi-stream|
|HEVC 4kp30||√||√||√ (Max 2)|
|AVC 4kp30||√||√||√ (Max 2)|
|HEVC 1080p60||X||√||√ (Max 4)|
|AVC 1080p60||X||√||√ (Max 4)|
√ - Supported
NA – Not applicable
x – Not supported
3 Appendix A - Input Configuration File (input.cfg)The example configuration files are stored at /media/card/config/ folder.Common Configuration:It is the starting point of common configuration.Num of Input:Provide the number of input. It is ranging from 1 to 4.Output:Select the video interface.Options: HDMI or DPOut Type:Options: display, record, and stream Display Rate:Pipeline frame rate.Options: 30 FPS or 60 FPS for each stream.Exit:It indicates to the application that the configuration is over.Input Configuration:It is the starting point of input configuration.Input Num:Starting Nth input configuration.Options: 1-4Input Type:Input source type.Options: TPG, HDMI, HDMI_2, HDMI_3, MIPI, FileUri:File path. Applicable for file playback only. File path should be appended by “file:/”Options: file:
/media/usb/abc.mp4Raw:To tell pipeline is processed or pass-through.Options: True, FalseWidth:The width of the live source.Options: 3840, 1920Height:The height of the live source.Options: 2160, 1080Exit:It indicates to the application that the configuration is over.Encoder Configuration:It is the starting point of encoder configuration.Encoder Num:Starting Nth encoder configuration.Options: 1-4Encoder Name:Name of the encoder.Options: AVC, HEVCProfile:Name of the profile.Options: baseline, main or high for AVC. Main for HEVC.Rate Control:Rate control options.Options: CBR, VBR, and low-latency.Filler Data:Filler Data NAL units for CBR rate control.Options: True, FalseQP:QP control mode used by the VCU encoder.Options: Uniform, AutoL2 Cache:Enable or Disable L2Cache buffer in encoding process.Options: True, FalseLatency Mode:Encoder latency mode.Options: normal, sub_frameLow Bandwidth:If enabled, decrease the vertical search range used for P-frame motion estimation to reduce the bandwidth.Options: True, FalseGop Mode:Group of Pictures mode.Options: Basic, low_delay_p, low_delay_bBitrate:Target bitrate in KbpsOptions: 1-60000B Frames:Number of B-frames between two consecutive P-framesOptions: 0-4Slice:The number of slices produced for each frame. Each slice contains one or more complete macroblock/CTU row(s). Slices are distributed over the frame as regularly as possible. If slice-size is defined as well more slices may be produced to fit the slice-size requirement. Options: 4-22 4kp resolution with HEVC codec 4-32 4kp resolution with AVC codec4-32 1080p resolution with HEVC codec4-32 1080p resolution with AVC codecGoP Length:The distance between two consecutive I framesOptions: 1-1000Format:The format of input data.Options: NV12Preset:Options: HEVC_HIGH, HEVC_MEDIUM, HEVC_LOW, AVC_HIGH, AVC_MEDIUM, AVC_LOW, CustomExitIt indicates to the application that the configuration is over.Record Configuration:It is the starting point of record configuration.Record Num:Starting Nth record configuration.Options: 1-4.Out-File Name:Record file path.Options: /media/usb/abc.mp4Duration:Duration in minutes.Options: 1-3ExitIt indicates to the application that the configuration is over.Streaming Configuration:It is the starting point of streaming configuration.Streaming Num:Starting Nth Streaming configuration.Options: 1-4.Host IP:The host to send the packets toOptions: 192.168.25.89 or Windows PC IPPort:The port to send the packets toOptions: 5004,5008,5012 and 5016.ExitIt indicates to the application that the configuration is over.Trace Configuration:It is the starting point of trace configuration.FPS Info:To display fps info on the console.Options: True, FalseAPM Info:To display APM counter number on the console.Options: True, FalsePipeline Info:To display pipeline info on console.Options: True, FalseExitIt indicates to the application that the configuration is over.
4 Appendix BSteps for GStreamer command line execution:
Support for only Single stream playback from the command line is available in this release.Note:
Using below commands, only limited configuration options are supported. Full-fledged command line support is available in next release.
- Kill the Qt GUI application running on target board by executing the below commands from the serial console.
- Run the below command to check HDMI link status and output format of the HDMI input source.
If HDMI-Rx link has an issue, it appears as below.