Versal AI Edge Series Gen 2 for ISP
1. Versal AI Edge Series Gen 2 - ISP Software Architecture
ISP is a hard block in a Multimedia Distributed (MMD) tile of the AMD Versal™ AI Edge Series Gen 2 SoC. The ISP software stack operates across both the Real-Time Processing Unit (RPU) and Application Processing Unit (APU) in a Split Architecture mode. The ISP software driver within the APU is responsible for extracting the ISP hardware configuration derived from the Device Tree Generator (DTG). These configurations are then transmitted to the RPU using Inter Process Communication (IPC) for the purpose of configuring the ISP instance.
The following diagram describes the structure of this Split Architecture.
The APU software encompasses the Linux operating system, configured to boot ARM Cortex® A78 cores. It includes essential drivers such as the remote proc driver, facilitating firmware loading onto the ARM Cortex R52 core-6 which is configured in lock-step mode. Additionally, an ISP driver extracts ISP configuration details from the device tree, registers itself within the V4L2 framework, and communicates with the RPU firmware through an IPC driver.
The APU operates by parsing an input JSON file that contains comprehensive information regarding active ISPs, their operational modes (for example, streaming, MCM), buffer quantities, input/output formats, and respective resolutions. Additionally, it manages the enabling of ISP filters and their configuration parameters. All of these crucial parameters are structured and transmitted via the mailbox to the RPU for execution.
The RPU firmware is the software running on the SRPU used for configuring the ISP instance, executing the 2A algorithm, configuring the sensor, IPC driver and other platform-related drivers such as I2C, IPI, and TTC.
RPU firmware supports asymmetric multi-processor (AMP) mode, allowing the APU to control two or more RPU cores independently to run multiple ISPs simultaneously.
It supports scalable, configurable Linux ISP/mailbox drivers and RPU firmware stack for AMP mode. Configuration is runtime-selectable based on hardware and performance requirements.
The device Tree populates multiple mailbox instances to support AMP mode based on design configuration. A single RPU can control multiple ISP tiles, while multiple RPUs manage and distribute ISP pipelines.
A single RPU project builds all RPU cores (6, 7, 8, and 9) using CMake.
To perform asymmetric multi-processing across RPU cores 6, 7, 8, and 9, synchronize access to resources shared by these cores.
Examples (Shared resource):
FMC
I2C controller
Shared memory
To protect access to these shared resources, multi-instance spinlock (1 instance defined for each shared resource) is implemented. All the clients in FW acquire and release the corresponding lock while using the shared resources.
ISP End to End Pipelines:
The ISP E2E pipeline contains a camera sensor, MIPI CSI 2 D-PHY, ISP (Processing IP), Video mixer (HLS IP) and HDMI 2.1 TX.
Non-MCM Use Case: In a Non-MCM use case, each ISP instance receives live data from the specific sensor.
MCM Use Case: In an MCM use case, each ISP instance can handle a maximum of four live input streams.
2. VEK385 Board Setup
The following image gives a visual representation of the connection between the Sensors, FMC, HDMI, COM Port and VEK385 board.
The basic board setup is as follows:
Connect the External Power Supply to the Power Input connector.
Connect the USB Type-C Connector labeled as FTDI USB to the host PC.
Connect the RJ45 Ethernet for DUT SoC to the local network.
Connect the RJ45 Ethernet for the System Controller labeled as SC Ethernet to the local network.
Powering the Board
Connect the Board External Power Supply to an outlet.
Plug in the External Supply to the board.
Turn on the Board using the power switch.
UART Connections - FTDI-USB
Evaluation boards have multiple UART connections. When the FTDI-USB cable is plugged in it will create multiple device nodes on the host PC.
For example, VEK385 has 4 serial / UART interfaces mapped as follows,
Device 0 (Versal AI Edge Series Gen 2 PS-UART0)
Device 1 (Versal AI Edge Series Gen 2 PS-UART1)
Device 2 (Versal AI Edge Series Gen 2 PL-UART)
Device 3 (System Controller UART)
For more details on Sensor, FMC, and De-serializer, refer to ISP Firmware [2025.2].
3. Boards Supported
VEK385-A1 (P5A)
VEK385 (Rev-B) supported from 2025.2
Xylon logiFMC-GMSL2-9296A-12C - 12-Ch GMSL2 FMC Board (Xylon Part Number: 11-00216-20-00001)
Xylon logiFMC-GMSL2-96716A-12C - 12-Ch GMSL2 FMC Board (Xylon Part Number: 11-00224-24-00001)
4. Reference Links
Versal AI Edge Series Gen 2 Documentation
Yocto Wiki – EDF Linux Disk Image Build Using Prebuilt Yocto Machine
AMD EDF Getting started - Operating System Integration and Development AMD Versal™ device portfolio Guide to build Linux disk images using a prebuilt Yocto Project machine for AMD Versal™ devices.
GA – AMD Embedded Development Framework (EDF)
AMD Embedded Development Framework (EDF) General availability landing page for the AMD Embedded Development Framework (EDF), with documentation and ecosystem information.
For more information related to Linux Driver, refer to the following ISP Linux Driver Documentation:
For more information related to Firmware, refer to the following ISP Firmware Documentation:
For more information related to Baremetal, refer to the following ISP BareMetal Documentation: