Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


This page is intended to complement  UG1186 "LibMetal and OpenAMP User Guide"  for Zynq-7000 and Zynq UltraScale+ MPSoC.


Quick try!

Here are the basic steps to boot Linux and run an openamp application using pre-built images.

e.g for ZCU102:
The echo-test application sends packets from Linux running on quad-core Cortex-A53 to a single cortex-R5 running FreeRTOS which send them back.
  • Extract files BOOT.BIN, image.ub and openamp.dtb files from a pre-built PetaLinux BSP to sdcard

  • Code Block
    themeMidnight
    host shell$ petalinux-create -s xilinx-zcu102-v2019.2-final.bsp --strip-components=4 --wildcards */BOOT.BIN */image.ub */openamp.dtb
    host shell$ cp BOOT.BIN image.ub openamp.dtb <your sd card>


Note: Alternatively, if you already created a PetaLinux project with a provided BSP for your board, pre-built images can also be found under the <your project>/pre-built/linux/images/ directory.
  • Go to u-boot prompt and boot Linux from sdcard

  • Code Block
    themeMidnight
    ...
    Hit any key to stop autoboot:  0 
    
    > mmcinfo && fatload mmc 0 ${netstart} ${kernel_img} &&  fatload mmc 0 0x14000000 openamp.dtb
    Device: sdhci@ff170000
    ...
    reading image.ub
    31514140 bytes read in 2063 ms (14.6 MiB/s)
    reading openamp.dtb
    38320 bytes read in 18 ms (2 MiB/s)
    > bootm $netstart $netstart 0x14000000
    ...


Note: As an alternative to all steps above to sd-boot, you can jtag-boot the board. For this you need to have connected a jtag cable, installed jtag drivers and created a PetaLinux project using a provided BSP. You would then go into the <your project>/pre-built/linux/images directory and replace file system.dtb by openamp.dtb, then enter: "petalinux-boot --jtag --prebuilt 3"
  • At Linux login prompt enter 'root' for user and 'root' for password and run echo-test demo

  • Code Block
    themeMidnight
    plnx_aarch64 login: root
    Password: 
    root@plnx_aarch64:~# echo image_echo_test > /sys/class/remoteproc/remoteproc0/firmware 
    root@plnx_aarch64:~# echo start > /sys/class/remoteproc/remoteproc0/state   
    [  177.375451] remoteproc remoteproc0: powering up ff9a0100.zynqmp_r5_rproc
    [  177.384705] remoteproc remoteproc0: Booting fw image image_echo_test, size 644144
    [  177.396832] remoteproc remoteproc0: registered virtio0 (type 7)
    [  177.399108] virtio_rpmsg_bus virtio0: rpmsg host is online
    [  177.412370] zynqmp_r5_remoteproc ff9a0100.zynqmp_r5_rproc: RPU boot from TCM.
    [  17Starting application...
    Try to init remoteproc resource
    Init remoteproc resource succeeded
    Waiting for events...
    7.422089] remoteproc remoteproc0: remote processor ff9a0100.zynqmp_r5_rproc is now up
    [  177.442121] virtio_rpmsg_bus virtio0: creating channel rpmsg-openamp-demo-channel addr 0x1
    root@plnx_aarch64:~# modprobe rpmsg_user_dev_driver
    [  188.089835] rpmsg_user_dev_driver virtio0:rpmsg-openamp-demo-channel: rpmsg_user_dev_rpmsg_drv_probe
    [  188.101250] rpmsg_user_dev_driver virtio0:rpmsg-openamp-demo-channel: new channel: 0x400 -> 0x1!
    root@plnx_aarch64:~# echo_test
     Echo test start 
     Open rpmsg dev! 
    [  190.364739] rpmsg_user_dev_driver virtio0:rpmsg-openamp-demo-channel: Sent init_msg to target 0x1.


Docs and source code

Documents

  • The following document describes libmetal APIs:
    View file
    namelibmetal-doc-20170418.pdf

URLs to source code


Xilinx Openamp and Libmetal related code

The following location provide access to the code:

Additional examples

OpenAMP Demos using RPMsg in kernel-space on Versal

  1. Configure PetaLinux to run the demo
    1. Download 2019.2 Versal BSP
    2. petalinux-config -c rootfs
      1. enable the following:
        • libmetal
        • open-amp
        • rpmsg-(demo name)
        • sysfsutils
        • libsysfs
        • openamp-echo-testd
    3. modify system-user.dtsi with the following

      1. Code Block
        /include/ "system-conf.dtsi"
        / {
        reserved-memory {
                #address-cells = <2>;
                #size-cells = <2>;
                ranges;
                rproc_0_dma: rproc@3ed40000 {
                    no-map;
                    compatible = "shared-dma-pool";
                    reg = <0x0 0x3ed40000 0x0 0x100000>;
                };
                rproc_0_reserved: rproc@3ed00000 {
                    no-map;
                    reg = <0x0 0x3ed00000 0x0 0x40000>;
                };
                rproc_1_dma: rproc@3ef40000 {
                    no-map;
                    compatible = "shared-dma-pool";
                    reg = <0x0 0x3ef40000 0x0 0x100000>;
                };
                rproc_1_reserved: rproc@3ef00000 {
                    no-map;
                    reg = <0x0 0x3ef00000 0x0 0x40000>;
                };
            };
         
            zynqmp-rpu {
                compatible = "xlnx,zynqmp-r5-remoteproc-1.0";
                core_conf = "split";
                #address-cells = <2>;
                #size-cells = <2>;
                ranges;
                r5_0: r5@0 {
                    #address-cells = <2>;
                    #size-cells = <2>;
                    ranges;
                    memory-region = <&rproc_0_reserved>, <&rproc_0_dma>;
                    pnode-id = <0x18110005>;
                    mboxes = <&ipi_mailbox_rpu0 0>, <&ipi_mailbox_rpu0 1>;
                    mbox-names = "tx", "rx";
                    tcm_0_a: tcm_0@0 {
                        reg = <0x0 0xFFE00000 0x0 0x10000>;
                        pnode-id = <0x1831800b>;
                    };
                    tcm_0_b: tcm_0@1 {
                        reg = <0x0 0xFFE20000 0x0 0x10000>;
                        pnode-id = <0x1831800c>;
                    };
                };
         
                r5_1: r5@1 {
                    #address-cells = <2>;
                    #size-cells = <2>;
                    ranges;
                    memory-region = <&rproc_1_reserved>, <&rproc_1_dma>;
                    pnode-id = <0x18110006>;
                    mboxes = <&ipi_mailbox_rpu1 0>, <&ipi_mailbox_rpu1 1>;
                    mbox-names = "tx", "rx";
                    tcm_1_a: tcm_1@0 {
                        reg = <0x0 0xFFE90000 0x0 0x10000>;
                        pnode-id = <0x603060d>;
                    };
                    tcm_1_b: tcm_1@1 {
                        reg = <0x0 0xFFEB0000 0x0 0x10000>;
                        pnode-id = <0x603060e>;
                    };
                };
            };
         
         
         zynqmp_ipi1 {
                compatible = "xlnx,zynqmp-ipi-mailbox";
                interrupt-parent = <&gic>;
                interrupts = <0 33 4>;
                xlnx,ipi-id = <5>;
                #address-cells = <1>;
                #size-cells = <1>;
                ranges;
         
                /* APU<->RPU0 IPI mailbox controller */
                ipi_mailbox_rpu0: mailbox@ff990600 {
                    reg = <0xff3f0ac0 0x20>,
                          <0xff3f0ae0 0x20>,
                          <0xff3f0740 0x20>,
                          <0xff3f0760 0x20>;
                    reg-names = "local_request_region",
                            "local_response_region",
                            "remote_request_region",
                            "remote_response_region";
                    #mbox-cells = <1>;
                    xlnx,ipi-id = <3>;
                };
                /* APU<->RPU1 IPI mailbox controller */
        
        		ipi_mailbox_rpu1: mailbox@ff990640 {
        			reg = <0xff3f0b00 0x20>,
        			      <0xff3f0b20 0x20>,
        			      <0xff3f0940 0x20>,
        			      <0xff3f0960 0x20>;
                   xlnx,ipi-id = <2>;
               }
        
                    reg-names = "local_request_region",
                            "local_response_region",
                            "remote_request_region",
                            "remote_response_region";
                    #mbox-cells = <1>;
                    xlnx,ipi-id = <4>;
                };
            };
        };
        


    4. build project with "petalinux-build"
    1. Running the Demo on Target
      1. After starting firmware on target the output from running Linux-side the output is as follows:

        Code Block
        $ echo_test -d <rpmsg channel name>
         Echo test start
         Open rpmsg dev /dev/rpmsg0!
         **************************************


Libmetal Demo on Versal

  1. R5 Baremetal Application
    1. source code can be found here
  2. Configure PetaLinux to run the demo
    1. Download 2019.2 Versal BSP
    2. petalinux-config -c rootfs
      1. enable the following:
        • libmetal
        • libmetal-demo
    3. install the libmetal R5 application into the PetaLinux -generated rootfs on target with the following:
      1. create PetaLinux app to install the firmware
        1. petalinux-create -t apps --template install --name libmetal-fw --enable
        2. modify recipe as follows:
          1. copy firmware into project-spec/meta-user/recipes-apps/libmetal-fw/files/libmetal-fw 
          2. modify libmetal-fw.bb as follows

            1. Code Block
              #
              # This file is the libmetal-fw recipe.
              #
               
              SUMMARY = "Simple libmetal-fw application"
              SECTION = "PETALINUX/apps"
              LICENSE = "MIT"
              LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
               
              SRC_URI = "file://libmetal-fw \
                  "  
              INSANE_SKIP_${PN} = "arch"
              S = "${WORKDIR}"
               
              do_install() {
                       install -d ${D}/lib/firmware/
                       install -m 644 ${S}/libmetal-fw ${D}/lib/firmware/
              }
              FILES_${PN} = " /lib/firmware/libmetal-fw "


    4. modify system-user.dtsi with the following

      1. Code Block
        / {
        
        	reserved-memory {
        		#address-cells = <2>;
        		#size-cells = <2>;
        		ranges;
        		rproc_0_reserved: rproc@3ed00000 {
        			no-map;
        			reg = <0x0 0x3ed00000 0x0 0x2000000>;
        		};
        	};
        	amba {
        		/* Shared memory (APU to RPU) */
        		shm0: shm@0 {
        			compatible = "shm";
        			reg = <0x0 0x3ed80000 0x0 0x01000000>;
        
        
        		};
        		/* IPI device */
        		ipi0: ipi@0 {
        			compatible = "ipi_uio";
        			reg = <0x0 0xff360000 0x0 0x1000>;
        			interrupt-parent = <&gic>;
        			interrupts = <0 33 4>;
        		};
        
        		timer@ff0e0000 {
                                compatible = "ttc-uio";
                                reg = <0x0 0xff0e0000 0x0 0x1000>;
                        };
        	};
        };
        
        &sdhci0 {
        	status = "disabled";
        };
        
        &sdhci1 {
        	status = "disabled";
        };
        
        


    5. build project with "petalinux-build"
  3. Running the Demo on Target
    1. After starting firmware on target the output from running Linux-side the output is as follows:

      1. Code Block
        # <linux libmetal application
        metal: warning: skipped page size 2097152 - invalid args
        CLIENT> ****** libmetal demo: shared memory ******
        metal: info: meta
        SERVER> Demo has started.
        SERVER> Shared memory test finished
        SERVER> ====== libmetal demo: atomic operation over shared memory ======
        SERVER> Starting atomic add on shared memory demo.
        l_uio_dev_open: No IRQ for device 3ed80000.shm.
        CLIENT> Setting up shared memory demo.
        CLIENT> Starting shared memory demo.
        CLIENT> Sending message: Hello World - libmetal shared memory demo
        CLIENT> Message Received: Hello World - libmetal shared memory demo
        CLIENT> Shared memory demo: Passed.



Libmetal IPC between RPU0 and RPU1 on Versal via OCM and/or DDR

  1. R5 Baremetal applications
    1. The source code for the demos and the binaries for RPU0 and RPU1 can be found in the attached zip file here. source for firmware can be found here 
      1. Note: the demos by default are hard-coded to use OCM for the shared memory location. To instead use DDR for the shared memory location do the following: modify within the sys_init.c the macro SHM_BASE_ADDR to 0X3ED40000UL
  2. Running the Demo

    1. Code Block
      SLAVE> Wait for Master.
      MASTER> Setting up shared memory demo.
      SLAVE> Demo has started.
      MASTER Starting shared memory demo.
      MASTER Message Received: test
      SLAVE> OCM Shared memory test finished
      MASTER Shared memory demo: Passed
      
      



OpenAMP demos with RPMSG in userspace with Shared Buffers and virtqueues in OCM

  1. Device Tree
    1. sample device tree for ZU+ 

      1. Code Block
        / {
            reserved-memory {
                #address-cells = <2>;
                #size-cells = <2>;
                ranges;
                rproc_0_reserved: rproc@3ed00000 {
                    no-map;
                    reg = <0x0 0x3ed00000 0x0 0x1000000>;
                };
            };
            amba {
                shm0: shm@0 {
                    compatible = "shm_uio";
                    reg = <0x0 0x3ed20000 0x0 0x0100000>;
                };
                ipi0: ipi@0 {
                    compatible = "ipi_uio";
                    reg = <0x0 0xff360000 0x0 0x1000>;
                    interrupt-parent = <&gic>;
                    interrupts = <0 33 4>;
                };
                ocm0 : ocm@0 {
                      compatible = "ocm_uio";
                      reg = <0 0xfffc0000 0x0 0x4000>;
                };
            };
        };
        
        


  2. R5 Baremetal Application
    1. attached is zipped source folder for r5 baremetal application
    2. changes to make for the application:
      1. platform_info.c

        1. Code Block
          #define SHARED_MEM_PA   0xFFFC0000UL
          #define SHARED_MEM_SIZE    0x20000UL


      2. rsc_table.c


        Code Block
        #define RING_TX                     0xfffc0000
        #define RING_RX                     0xfffc4000
        #define VRING_SIZE                  64


      3. linker script:

        1. Code Block
          + .resource_table 0x3ed20000 : {
          +     . = ALIGN(4);
          +     *(.resource_table)
          + } > psu_ddr_S_AXI_BASEADDR
          +


  3. Linux application
    1. attached is zipped source folder for linux application
      1. modify open-amp application code to use OCM

        1. Code Block
          # within build dir of yocto or root petalinux project
          devtool modify open-amp


        2.  apps/machine/zynqmp/platform_info.c. Here we will update shared buffer location, shared buffer size, vring location and vring size.


          1. Code Block
            -#define    VRING_MEM_PA    0x3ED40000UL
            +#define    VRING_MEM_PA    0xFFFC0000UL
             
            -#define    SHARED_BUF_PA   0x3ED48000UL
            +#define    SHARED_BUF_PA   0xFFFC8000UL
             
            -#define    SHARED_BUF_SIZE 0x40000UL
            +#define    SHARED_BUF_SIZE 0x20000UL
             
            +#define    OCM_DEV_NAME          "fffc0000.ocm"
             
            /* in  struct remoteproc_priv rproc_priv = { add the following field : */
            +   .ocm_name = OCM_DEV_NAME,


            1. if communicating with RPU1, change IPI_CHN_BITMASK from 0x08 to 0x09 in this file too.
          1. apps/machine/zynqmp/platform_info.h - add OCM information to the remoteproc_priv struct

            1. Code Block
              +   struct remoteproc_mem ocm_mem; /**< shared memory */
              +   struct metal_io_region *ocm_io; /**< pointer to OCM i/o
              +                        region */
              +   const char *ocm_name; /**< shared memory device name */
              +   struct metal_device *ocm_dev; /**< pointer to OCM device */


          2. apps/machine/zynqmp/zynqmp_linux_r5_proc.c - add the following after adding shared memory

            1. Code Block
              +    prproc->ocm_dev = NULL;
              +    /* Get shared memory device */
              +    ret = metal_device_open(prproc->shm_bus_name, prproc->ocm_name,
              +                &dev);
              +    if (ret) {
              +        fprintf(stderr, "ERROR: failed to open ocm device: %d.\r\n", ret);
              +        goto err1;
              +    }
              +    printf("Successfully open ocm device.\r\n");
              +    prproc->ocm_dev = dev;
              +    prproc->ocm_io = metal_device_io_region(dev, 0);
              +    if (!prproc->ocm_io)
              +        goto err2;
              +    mem_pa = metal_io_phys(prproc->ocm_io, 0);
              +    remoteproc_init_mem(&prproc->ocm_mem, "ocm", mem_pa, mem_pa,
              +       metal_io_region_size(prproc->ocm_io), prproc->ocm_io);
              +    remoteproc_add_mem(rproc, &prproc->ocm_mem);
              +    printf("Successfully added OCM memory\r\n");


  4. Running the demo
    1. expected output is as follows:

      1. Code Block
        root@virt-versal:~# matrix_multiply-shared
        metal: info: Registered sinitializing rpmsg shared buffer pool
        REMOTE> Waiting for events...
        REMOTE> Message received
        REMOTE> Send the result of matrix multiplication back to master.
        REMOTE> Message received
        REMOTE> Stopping application...
        rovider linux_shm.
        metal: info: Registered shmem provider ion.reserved.
        metal: info: Registered shmem provider ion.ion_system_contig_heap.
        metal: info: Registered shmem provider ion.ion_system_heap.
        metal: info: metal_uio_dev_open: No IRQ for device 3ed20000.shm.
        Successfully open shm device.
        Successfully added shared memory
        metal: info: metal_uio_dev_open: No IRQ for device fffc0000.ocm.
        Successfully open ocm device.
        Successfully added OCM memory
        Successfully probed IPI device
        Successfully initialized Linux r5 remoteproc.
        Successfully initialized remoteproc
        Calling mmap resource table.
        pa 3ed20000 rsc_size RSC_MEM_SIZE 2000
        Successfully mmap resource table.
        Successfully set resource table to remoteproc.
        CLIENT> Compute thread unblocked ..
        CLIENT> It will generate two random matrices.
        CLIENT> Send to the remote and get the computation result back.
        CLIENT> It will then check if the result is expected.
        CLIENT> RPMSG endpoint is binded with remote.
        CLIENT> Input matrix 0
        5 9 9 7 8 3
        4 4 8 9 6 1
        1 5 7 1 0 2
        3 3 0 4 0 6
        3 9 0 6 1 2
        8 8 3 7 7 3
        CLIENT> Input matrix 1
        3 1 8 1 2 6
        4 5 3 1 6 3
        6 2 9 8 6 9
        4 9 0 5 7 3
        9 6 3 2 5 1
        5 8 4 5 0 9
        CLIENT> Matrix multiply: sent : 296
        CLIENT> Input matrix 0
        3 4 4 7 8 3
        2 6 5 3 4 1
        4 8 2 4 5 0
        9 4 8 3 8 3
        6 6 2 2 3 4
        1 7 0 6 4 0
        CLIENT> Input matrix 1
        9 8 6 4 2 0
        7 6 9 1 3 4
        3 2 1 1 7 9
        5 5 5 9 8 9
        3 9 8 5 5 2
        6 6 2 2 2 4
        CLIENT> Matrix multiply: sent : 296
        CLIENT> **********************************
        CLIENT> Test Results: Error count = 0
        CLIENT> **********************************
        CLIENT> Quitting application .. Matrix multiplication end
        CLIENT> Stopping application...
        root@virt-versal:~#
        


...

2. Enable the OpenAMP and libmetal packages with "petalinux-config -c rootfs":

Code Block
themeMidnight
Filesystem Packages --->
   libs --->
       libmetal   --->
         [*] libmetal
       open-amp   --->
         [*] open-amp
 
 

...

  1. copy firmware (.elf file) into project-spec/meta-user/recipes-apps/<app_name>/files/ directory
  2. Modify the project-spec/meta-user/recipes-apps/<app_name>/<app_name>.bb to install the remote processor firmware in the RootFS as follows:


Code Block
themeMidnight
SUMMARY = "Simple test application"
SECTION = "PETALINUX/apps"
LICENSE = "MIT"
LIC_FILES_CHKSUM ="file:${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
SRC_URI = "file:<myfirmware>"
S = "${WORKDIR}"
INSANE_SKIP_${PN} = "arch"
do_install() {
  install -d ${D}/lib/firmware
  install -m 0644 ${S}/<myfirmware> ${D}/lib/firmware/<myfirmware>
}
FILES_${PN} = "/lib/firmware/<myfirmware>
 
 

...