This page describes how to set up and run the libmetal demo on the Versal Platform, in order to demonstrate the communication between a user-space application running on A72 and bare-metal application running on R5.

Table of Contents

Introduction

This wiki provides the steps to use Vitis to build libmetal applications targeting the R5 and A72 processors. It also details how to set up a device-tree and build a sysroot with library support to build the Linux libmetal application. The application executables built in Vitis are installed in the PetaLinux project RootFS. The example demo shows the communication between a Linux user-space program on A72 and a bare-metal application running on the R5.

Libmetal provides a common user API across various OS environments for device access, interrupt handling and memory access. Using the standardized API, the same application can be run in different environments, such as Linux userspace, FreeRTOS or baremetal. Please refer to our user guide Access-Devices-with-Libmetal to understand how the Libmetal environment is set up, the API’s are used to access the devices, specify shared memory regions and interprocessor interrupts (IPI) .

PetaLinux

In this section, a Petalinux project will be modified to enable libmetal libraries; define shared memory, IPI and remoteproc nodes; and then export the resulting sysroot to Vitis for application development.

The PetaLinux project for this demo is built using a VCK190 BSP. The BSP can be downloaded here Downloads. You can also create a project using the template flow and your custom XSA.

1. Create the PetaLinux project using the VCK190 BSP:

   $ petalinux-create -t project -n vck190-libmetal-demo -s xilinx-vck190-v2022.1-final.bsp

2. Configure the project:

   $ cd vck190-libmetal-demo/ $ petalinux-config

3. Modify the system-user.dtsi

   1. Device-tree modifications for version 2022.x:
      Copy the content below into the system-user.dtsi. The reserved memory denotes the section of the memory being reserved in the DDR for running the RPU application. This region will also be reflected in the lscript.ld of the RPU application being built in Vitis. In the 2022.x releases and later, TCM and RPU nodes require the power-domain property as follows:
      
      <&versal_firmware power-domain-node>
      
      The power node numbers for Versal Adaptive SoC devices can be found here.

      $ vim project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi /include/ "system-conf.dtsi" / { }; / { reserved-memory { #address-cells = <2>; #size-cells = <2>; ranges; rproc_0_reserved: rproc@3ed00000 { no-map; reg = <0x0 0x3ed00000 0x0 0x40000>; }; }; tcm_0a@ffe00000 { no-map; reg = <0x0 0xffe00000 0x0 0x10000>; phandle = <0x40>; status = "okay"; compatible = "mmio-sram"; power-domain = <&versal_firmware 0x1831800b>; }; tcm_0b@ffe20000 { no-map; reg = <0x0 0xffe20000 0x0 0x10000>; phandle = <0x41>; status = "okay"; compatible = "mmio-sram"; power-domain = <&versal_firmware 0x1831800c>; }; rf5ss@ff9a0000 { compatible = "xlnx,zynqmp-r5-remoteproc"; #address-cells = <0x2>; #size-cells = <0x2>; ranges; xlnx,cluster-mode = <1>; reg = <0x0 0xff9a0000 0x0 0x10000>; r5f_0 { compatible = "xilinx,r5f"; #address-cells = <0x2>; #size-cells = <0x2>; ranges; sram = <0x40 0x41>; memory-region = <&rproc_0_reserved>; power-domain = <&versal_firmware 0x18110005>; }; }; /* Shared memory */ shm0: shm@0 { compatible = "shm_uio"; reg = <0x0 0x3ed80000 0x0 0x1000000>; }; /* IPI device */ ipi_amp: ipi@ff360000 { compatible = "ipi_uio"; reg = <0x0 0xff360000 0x0 0x1000>; interrupt-parent = <&gic>; interrupts = <0 33 4>; }; /* Triple Timer Counter */ ttc0 { compatible = "ttc-uio"; reg = <0x0 0xFF0E0000 0x0 0x1000>; }; };

   2. Device-tree modifications in 2021.2 .
      
      Add the content below to the system-user.dtsi for the reserved memory node, TCM memories of the RPU and the remoteproc node. Nodes are also added for IPI and shared memory (used as UIO devices).

      $ vim project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi /include/ "system-conf.dtsi" / { }; &amba { zyxclmm_drm { compatible = "xlnx,zocl-versal"; }; }; / { reserved-memory { #address-cells = <2>; #size-cells = <2>; ranges; rproc_0_reserved: rproc@3ed00000 { no-map; reg = <0x0 0x3ed00000 0x0 0x40000>; }; }; tcm_0a@ffe00000 { no-map; reg = <0x0 0xffe00000 0x0 0x10000>; phandle = <0x40>; status = "okay"; compatible = "mmio-sram"; }; tcm_0b@ffe20000 { no-map; reg = <0x0 0xffe20000 0x0 0x10000>; phandle = <0x41>; status = "okay"; compatible = "mmio-sram"; }; rf5ss@ff9a0000 { compatible = "xlnx,zynqmp-r5-remoteproc"; #address-cells = <0x2>; #size-cells = <0x2>; ranges; xlnx,cluster-mode = <1>; reg = <0x0 0xff9a0000 0x0 0x10000>; r5f_0 { compatible = "xilinx,r5f"; #address-cells = <0x2>; #size-cells = <0x2>; ranges; sram = <0x40 0x41>; memory-region = <&rproc_0_reserved>; power-domain = <0x18110005>; }; }; /* Shared memory */ shm0: shm@0 { compatible = "shm_uio"; reg = <0x0 0x3ed80000 0x0 0x1000000>; }; /* IPI device */ ipi_amp: ipi@ff360000 { compatible = "ipi_uio"; reg = <0x0 0xff360000 0x0 0x1000>; interrupt-parent = <&gic>; interrupts = <0 33 4>; }; /* Triple Timer Counter */ ttc0 { compatible = "ttc-uio"; reg = <0x0 0xFF0E0000 0x0 0x1000>; }; };

4. RootFS modifications:

   $ petalinux-config -c rootfs Filesystem Packages---> misc ---> sysfsutils ---> [*] libsysfs libs ---> libmetal---> [*] libmetal [*] libmetal-demos

5. Build and package sysroot:

   $ petalinux-build --sdk $ petalinux-package --sysroot

6. Install the libmetal application in the RootFS. Create a template app to install the Libmetal R5 firmware and Linux applications built in Vitis to the RootFS. Copy the application ELFs built in Vitis to the meta-user/recipes-apps/libmetal-apps/files/ directory.

   $ petalinux-create -t apps --template install -n libmetal-apps --enable $ cp <vitis-proj-libmetal-r5>/Debug/libmetal-r50.elf project-spec/meta-user/recipes-apps/libmetal-apps/files/ $ cp <vitis-proj-libmetal-linux>/Debug/libmetal-linux.elf project-spec/meta-user/recipes-apps/libmetal-apps/files/

7. Modify the libmetal-apps.bb recipes to include the application elf build in Vitis to the RootFS

   1. For the 2022.x versions and later please note the syntax changes in the Yocto recipe. The [:]colon character replaces the use of [ _ ] underscore when referring to an override.
      
      

      # # This file is the libmetal-apps recipe. # SUMMARY = "Simple libmetal-apps application" SECTION = "PETALINUX/apps" LICENSE = "MIT" LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" SRC_URI = "file://libmetal-linux.elf \ file://libmetal-r50.elf \ " S = "${WORKDIR}" INSANE_SKIP:${PN} = "arch" RDEPENDS:${PN} += " \ libmetal \ open-amp \ " do_install() { install -d ${D}/lib/firmware install -m 0644 ${S}/libmetal-r50.elf ${D}/lib/firmware/libmetal-r50.elf install -d ${D}/${bindir} install -m 0755 ${S}/libmetal-linux.elf ${D}/${bindir}/libmetal-linux.elf } FILES:${PN} = "/lib/firmware/libmetal-r50.elf ${bindir}/libmetal-linux.elf"

   2. For the 2021.x versions and earlier, the recipe is as shown below:

      $ vim project-spec/meta-user/recipes-apps/libmetal-apps/libmetal-apps.bb # This file is the libmetal-apps recipe. # SUMMARY = "Simple libmetal-apps application" SECTION = "PETALINUX/apps" LICENSE = "MIT" LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" SRC_URI = "file://libmetal-linux.elf \ file://libmetal-r50.elf \ " S = "${WORKDIR}" INSANE_SKIP_${PN} = "arch" RDEPENDS_${PN}_append = " \ libmetal \ open-amp \ " do_install() { install -d ${D}/lib/firmware install -m 0644 ${S}/libmetal-r50.elf ${D}/lib/firmware/libmetal-r50.elf install -d ${D}/${bindir} install -m 0755 ${S}/libmetal-linux.elf ${D}/${bindir}/libmetal-linux.elf } FILES_${PN} = "/lib/firmware/libmetal-r50.elf ${bindir}/libmetal-linux.elf"

8. Build the PetaLinux project and package the images.

   $ petalinux-build $ petalinux-package --boot --plm --psmfw --u-boot --dtb #QEMU $ petalinux-package --boot --u-boot --qemu-rootfs images/linux/rootfs.cpio.gz.u-boot --force

Vitis

Vitis is used to build the libmetal application examples for the R5 and A72 processors. The executables built in Vitis are then included in the PetaLinux recipes-apps generated above and installed to your RootFS.

Libmetal R5 App

1. Create a standalone Libmetal demo application targeting the R5-0 processor. Click browse and import your custom XSA, or you can select the vck190.xsa from fixed platforms.

2. Select the target processor versal_cips_0_pspmc_0_psv_cortexr5_0

3. Select Libmetal AMP Demo from the available templates.

4. Build the application. The generated .elf will be in the <vitis-app>/Debug/ directory. This .elf will need to be copied to the PetaLinux meta-user/recipes-apps folder as described in Step 6 of the PetaLinux project setup instructions.

Libmetal Linux App

1. Create a new application project for the Libmetal demo on Linux. You can use the same VCK190 platform created for your standalone application on R5

2. Check the box to show all processors in the hardware configuration. Select the target processor psv_cortexa72 SMP

3. Create a new domain and provide the sysroot path. The sysroot path will be <plnx-proj>/images/linux/sdk/sysroots/cortexa72-cortexa53-xilinx-linux/

4. Select Linux empty application from the available templates and click finish.

5. Clone the Libmetal repo. Copy the src files below to your Vitis libmetal-linux application's src folder

   $ git clone https://github.com/Xilinx/libmetal.git $ cd libmetal $ git checkout xlnx_rel_v2022.1 $ cd examples/system/linux/zynqmp/zynqmp_amp_demo/ $ cp common.h ipi_latency_demo.c ipi_shmem_demo.c ipi-uio.c shmem_atomic_demo.c shmem_demo.c shmem_latency_demo.c shmem_throughput_demo.c sys_init.c sys_init.h libmetal_amp_demo.c <vitis-proj-libmetal-linux>/src/

6. Add the metal library under ARM v8 Linux gcc linker Libraries.  

   Open

   

7. Add the compiler flag CONFIG_IPI_MASK=0x08 under the ARM v8 Linux gcc compiler symbols 

8. In common.h, make the following changes 

   #define IPI_DEV_NAME "ff360000.ipi" #define TTC_DEV_NAME "ff0e0000.ttc0"

9. Build the Libmetal Linux project.
   The generated .elf will be in the <vitis-app>/Debug/ directory. This .elf will need to be copied to the PetaLinux meta-user/recipes-apps folder as described in Step 6 of the PetaLinux project setup instructions.

Testing on Hardware/QEMU

Related Links

• https://docs.xilinx.com/r/en-US/ug1186-zynq-openamp-gsg