1. Versal AI Edge Series Gen 2 - ISP Software Architecture 

ISP is a hard block in a Multimedia Distributed (MMD) tile of the AMD Versal™ AI Edge Series Gen 2 SoC. The ISP software stack operates across both the Real-Time Processing Unit (RPU) and Application Processing Unit (APU) in a Split Architecture mode. The ISP software driver within the APU is responsible for extracting the ISP hardware configuration derived from the Device Tree Generator (DTG). These configurations are then transmitted to the RPU using Inter Process Communication (IPC) for the purpose of configuring the ISP instance.

The following diagram describes the structure of this Split Architecture.

Open

The APU software encompasses the Linux operating system, configured to boot ARM Cortex® A78 cores. It includes essential drivers such as the remote proc driver, facilitating firmware loading onto the ARM Cortex R52 core-6 which is configured in lock-step mode. Additionally, an ISP driver extracts ISP configuration details from the device tree, registers itself within the V4L2 framework, and communicates with the RPU firmware through an IPC driver.

The APU operates by parsing an input JSON file that contains comprehensive information regarding active ISPs, their operational modes (for example, streaming, MCM), buffer quantities, input/output formats, and respective resolutions. Additionally, it manages the enabling of ISP filters and their configuration parameters. All of these crucial parameters are structured and transmitted via the mailbox to the RPU for execution.

The RPU firmware is the software running on the SRPU used for configuring the ISP instance, executing the 2A algorithm, configuring the sensor, IPC driver and other platform-related drivers such as I2C, IPI, and TTC.

RPU firmware supports asymmetric multi-processor (AMP) mode, allowing the APU to control two or more RPU cores independently to run multiple ISPs simultaneously.

Open

It supports scalable, configurable Linux ISP/mailbox drivers and RPU firmware stack for AMP mode. Configuration is runtime-selectable based on hardware and performance requirements.
The device Tree populates multiple mailbox instances to support AMP mode based on design configuration. A single RPU can control multiple ISP tiles, while multiple RPUs manage and distribute ISP pipelines.
A single RPU project builds all RPU cores (6, 7, 8, and 9) using CMake.

To perform asymmetric multi-processing across RPU cores 6, 7, 8, and 9, synchronize access to resources shared by these cores.

Examples (Shared resource): 

1. FMC 

2. I2C controller

3. Shared memory

To protect access to these shared resources, multi-instance spinlock (1 instance defined for each shared resource) is implemented. All the clients in FW acquire and release the corresponding lock while using the shared resources.

ISP End to End Pipelines:

The ISP E2E pipeline contains a camera sensor, MIPI CSI 2 D-PHY, ISP (Processing IP), Video mixer (HLS IP) and HDMI 2.1 TX.

Open

Open

2. VEK385 Board Setup

The following image gives a visual representation of the connection between the Sensors, FMC, HDMI, COM Port and VEK385 board.

Open

VEK385 Board setup

The basic board setup is as follows:

1. Connect the External Power Supply to the Power Input connector.

2. Connect the USB-Type C Connector labeled as FTDI USB to the host PC.

3. Connect the RJ45 Ethernet for DUT SoC to the local network.

4. Connect the RJ45 Ethernet for the System Controller labeled as SC Ethernet to the local network.

Powering the Board

1. Connect the Board External Power Supply to an outlet

2. Plug in the External Supply to the board

3. Turn the Board ON with the power switch

Open

Powering the Board

UART connections - FTDI-USB
Evaluation boards have multiple UART connections. When the FTDI-USB cable is plugged in it will create multiple device nodes on the host PC.

For example, VEK385 has 4 serial / UART interfaces mapped as follows,

• Device 0 (Versal PS-UART0)

• Device 1 (Versal PS-UART1)

• Device 2 (Versal PL-UART)

• Device 3 (System Controller UART)

For more details on Sensor, FMC, and De-serializer, refer to ISP Firmware [2025.2].

3. Release Notes [2025.1]

3.1 Steps to Generate ISP Example Design and Build Software Components [2025.1]

Generate ISP Example Design: Begin by generating the ISP Vivado example design, according to the steps outlined in the PG432 ISP Product Guide.

Generate Yocto-based Linux Artifacts: Follow the instructions provided in the Linux section to Create Yocto based Linux Artifacts. In case MIPI is connected to the FPD domain in the Programmable Logic (PL) design, refer to the MIPI stability Answer Record AR000038130 for guidance on addressing the MIPI stability issue.
To address the CDO clock issues in example designs (Non-MCM LIMO/LILO with TILE0), apply the patch from this Answer Record AR000038118.

ISP Firmware Preparation: You can either use the pre-built ISP firmware binary from the generated root filesystem (rootfs) or generate the ISP firmware by following the steps detailed on the ISP Firmware [2025.1] page. If MIPI is connected to the Low Power Domain (LPD) in the PL design, refer to the Answer Record AR000038130 for solutions to the MIPI stability issue.

Load and Validate Artifacts: Once VEK385 board setup is complete (Xylon FMC Card and Sensor Setup), based on the boot mode, copy the artifacts to their respective location. See the Steps to Run ISP Pipeline MIMO/LIMO/LILO section in the Linux build page to load the artifacts onto the target.

3.2 ISP IP/Driver Features [2025.1]

The following table describes the list of ISP features, input/output formats, maximum resolutions, and output paths supported as part of the 2025.1 release.

3.3 Features Not Supported in ISP Stack for [2025.1]

• RGB-IR sensor (5 MP)

• Multiple ISP instance combinations for MCM mode:

  • 4 streams on ISP0 + 1 stream on ISP1 instance

  • 3 streams on ISP0 + 2 streams on ISP1 instance

  • 2 streams on ISP0 + 2 streams on ISP1 instance

• Multiple instances of ISP firmware running in AMP mode

3.4 Known Issues [2025.1]

• MCM is validated using a maximum of four sensors operating at 15 fps. Running a pipeline at 30 fps can in the long run lead to frame-drops or pipeline hangs.

• Closing a single pipeline in the MCM use case causes hangs on other running pipelines of the same ISP instance

• If a design includes AXI-Switch and AXI-Broadcaster IPs as part of ISP pipeline, errors are observed while converting pl.dtsi to dtbo because of end point linking issues. To overcome this issue, users have to edit the remote end point in pl.dtsi.

• Memory In Memory Out (MIMO) pipeline with fakesink is not working.

• Live In Live Out (LILO) driver is not validated with designs that have MIPI connected to the FPD in the PL design.

• Closing the media_server application using Ctrl+C causes a core dump. Instead use the pkill -9 isp_media_serve command to close the media_server application.

• Unloading of the ISP overlay using dfx-manager leads to Kernal Crashes. A reboot is needed for rerun.

• Use only the shared Tuning files, custom Tuning Parameters cannot be supported.

3.5 Limitations [2025.1]

• Only a single instance of firmware can be loaded and run on RPU cluster-3, core-0 (R52_6).

• In Non-MCM mode, only two ISP tiles can be validated with four sensors.

• In MCM-mode, the firmware can handle up to 4 MCM streams only on a single ISP.

• 3 ISP Tiles can support max 15 sensors. Due to a XylonFMC + VEK385 board pin limitation, only 4 streams are supported.

• On a VEK385 board, Xylon FMC supports only 3 MIPI interfaces, only MIPI-2 and MIPI-6 is validated. MIPI-5 is not validated due to design limitations.

• An End to End pipeline with 8MP (3840x2160) sensor can achieve up to 15 fps.

• In certain Vivado example designs, ISP SLCR Clocks are not being programmed. See the ISP SLCR Clocks Answer Record AR000038118 for guidance on addressing ISP SLCR Clocks.

• MIPI instability issues are observed with some VEK385 boards. Refer to AR000038130 for guidance on addressing the MIPI stability issue.

4. Release Notes [2025.2]

4.1 Steps to Generate ISP Example Design and Build Software Components [2025.2]

Generate ISP Example Design: Begin by generating the ISP Vivado example design, according to the steps outlined in the PG432 ISP Product Guide.

Generate Yocto-based Linux Artifacts: Follow the instructions provided in the Linux section to Create Yocto-based Linux Artifacts.

ISP Firmware Preparation: You can either use the pre-built ISP firmware binary from the generated root filesystem (rootfs) or generate the ISP firmware by following the steps detailed on the ISP Firmware [2025.2].

4.2 ISP IP/Driver Features [2025.2]

The following table describes the list of ISP features, input/output formats, maximum resolutions, and output paths supported as part of the 2025.2 release.

4.3 Newly Added Features in ISP Stack [2025.2]

• VISP BareMetal

  • Example APU BareMetal driver support for LIMO, LILO, and MIMO configuration

• Sensor Support

  • RGB-IR sensor (5 MP) supports Legacy and AB modes.

  • Added support for Sony 3 MP and 8 MP sensors.

• Firmware and Architecture

  • Multiple ISP firmware instances run in AMP mode.

  • Firmware loads to every core via Remoteproc from APUs Linux.

  • APU supports bare-metal code to run and control ISP firmware.

  • A single Vitis project builds all RPU firmware 6/7/8/9.

• MCM Configuration

  • MCM supports ISP tiles with 4+1 and 3+2 configurations within an ISP tile.

• ISP Firmware Functional Safety

  • Integrated and enabled ISP Software FuSa AoUs in RPU Firmware from APU Linux drivers.

• Performance

  • An end-to-end pipeline with an 8 MP (3840x2160) sensor achieves up to 15 fps (MCM) and 30 fps (Non-MCM) per stream.

  • In Non-MCM mode, six sensors validate on all ISP tiles.

• Linux Drivers

  • v4l2 controls support is added in LIMO and LILO drivers.

4.4 Known Issues [2025.2]

• AMP Mode

  • The Mailbox driver is providing intermittent prints of No message in shared memory for rpu_id X

• Source elements connected to the VISP node in the device tree are not included in the media pipeline, therefore they must be removed from the device tree and the VISP source node must be updated.

• Compatibility Issue: ISP capture → DP display pipeline does not work for NV12 format due to buffer alignment issues.

• MIMO Driver:

  • Does not run work with RPU8 and RPU9.

  • ISP can produce corrupted output frames. This issue is observed when the CMA node with address 0x500_0000_0000 is present in the DTS, even though the ISP driver does not use this node for any allocations.
    Work-around: As ISP driver does not allocate from this region, commenting the 0x500_0000_0000 CMA region resolves the issue.

• LILO Driver:

  • The OBA PPC value is not updated dynamically by the LILO Linux driver; it is fixed at 4 PPC.

  • The default FPS for pipelines is 30. To run at a different FPS, the value needs to be set with media-ctl for each run.

  • OBA programming calls in the RPU overwrite OBA registers on TILE0 when programming OBA for TILE1 or TILE2. Because of this, in multi-tile designs only RGB888 is supported (the default OBA configuration). In single-tile designs, either RGB888 and NV16 or NV12 alone are supported. (NV12 support is added in the driver but not validated with the design.)

  • When running both the primary output (MP) and the secondary output (SP) simultaneously, closing the primary output (MP) pipeline will invoke ISP stream off thus also stop the secondary stream (SP).

• ISP Tuning

  • Image flickering will be observed in stream when 2A is enabled, due to 2A algorithm issues.

• Baremetal Driver

  • Observing noise in MIMO mode

  • Resolutions above 1920×1080 are not supported.

  • Frame rate (FPS) is fixed at 30.

  • In LILO SP self-path configurations, the buffer address might appear as 0x0 in the dump command.

4.5 Bug Fixes [2025.2]

• MCM is validated with up to five sensors at 30 fps.

• Resolved closing pipeline in the MCM use case that caused hangs on other pipelines of the same ISP instance.

• Memory In Memory Out (MIMO) pipeline with fake sink works.

• Core dump issue on closing media-server is resolved.

• Kernel crash when unloading overlay is resolved.

4.6 Limitations [2025.2]

• 3 ISP Tiles can support maximum 15 sensors but due to existing XylonFMC + VEK385 board pin limitations, currently, only 6 streams are supported.

• On a VEK385 board, Xylon FMC supports only 3 MIPI interfaces.

• Use only the shared Tuning files, custom tuning Parameters cannot be supported.

• Tuning tool is not supported.

5. Boards Supported

• VEK385-A1 (P5A)

• VEK385 (RevB) supported from 2025.2

• Xylon logiFMC-GMSL2-9296A-12C - 12-Ch GMSL2 FMC Board (Xylon Part Number:  11-00216-20-00001)

6. Reference Links

• Versal AI Edge Series Gen 2 Documentation

  • PG432 – Versal AI Edge Series Gen 2 ISP Product Guide (ISP Architecture)

  • AM026 – Versal AI Edge Series Gen 2 and Prime Series Gen 2 Technical Reference Manual (TRM)

• Yocto Wiki – EDF Linux Disk Image Build Using Prebuilt Yocto Machine

  • AMD EDF Getting started - Operating System Integration and Development AMD Versal™ device portfolio Guide to build Linux disk images using a prebuilt Yocto Project machine for AMD Versal™ devices.

• GA – AMD Embedded Development Framework (EDF)

  • AMD Embedded Development Framework (EDF) General availability landing page for the AMD Embedded Development Framework (EDF), with documentation and ecosystem information.

• ISP Linux Driver Documentation

  • ISP Linux Driver [2025.1] Detailed documentation on the Linux driver support for the Image Signal Processor (ISP) on Versal AI Edge Series Gen 2 for 2025.1.

  • ISP Linux Driver [2025.2] Detailed documentation on the Linux driver support for the Image Signal Processor (ISP) on Versal AI Edge Series Gen 2 for 2025.2.

• ISP Firmware Documentation

  • ISP Firmware [2025.1] Detailed documentation on the Linux driver support for the Image Signal Processor (ISP) on Versal AI Edge Series Gen 2 for 2025.1.

  • ISP Firmware [2025.2] Detailed documentation on the Linux driver support for the Image Signal Processor (ISP) on Versal AI Edge Series Gen 2 for 2025.2.

• ISP BareMetal Documentation

  • ISP Baremetal Driver [2025.2] Detailed documentation on ISP BareMetal support for the Image Signal Processor (ISP) on Versal AI Edge Series Gen 2.