This document covers how to configure RPMsg in Kernelspace or Userspace.
Table of Contents Introduction The OpenAMP RPMsg API allows inter-process communications (IPC) between software running on independent cores in an AMP system. This is also compliant with the RPMsg BUS infrastructure present in the Linux Kernel version 3.18 and later.
There are different implementations of RPMsg,
Linux kernel rpmsg (primary) and RPU OpenAMP remote processor (secondary).
Linux userspace OpenAMP application (primary) and RPU OpenAMP application (secondary).
Some important configuration elements to take care of when dealing with RPMsg are the resource table (for firmware), shared memory with shared buffers and vring buffers. There is also another configuration element which is the Inter-Processor Interrupt (IPI) which is used by one core to notify the other.
RPMsg in Kernelspace Figure: RPMsg Implementation in Kernel Space
The Linux rpmsg driver parses the RPU firmware ELF and updates the resource table in the ELF with vring address locations (based on what is provided in the kernel device tree). The shared memory address in the RPU firmware should match the shared memory address in the kernel device tree.
Modifications for RPMsg in kernel space Refer to the OpenAMP Application Build Process page for Building Linux Distribution using PetaLinux , for modifying Demo Tests Firmware , and for booting up and running demo tests in kernelspace.
OpenAMP version specific modifications are covered below; for example, changes in the device tree, RPU firmware, Linux application, etc.
2022.1 Example Device Tree
Device tree file location:
<plnx-proj-root>/project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi
Start with the sample DTSI files published at Xilinx/meta-openamp/zynqmp-openamp.dtsi following sample device tree
Device Tree Source Include file for ZynqMP with RPMsg in kernelspace and Remoteproc in Lock-step mode
/ {
reserved-memory {
#address-cells = <2>;
#size-cells = <2>;
ranges;
rpu0vdev0vring0: rpu0vdev0vring0@3ed40000 {
no-map;
reg = <0x0 0x3ed40000 0x0 0x4000>;
};
rpu0vdev0vring1: rpu0vdev0vring1@3ed44000 {
no-map;
reg = <0x0 0x3ed44000 0x0 0x4000>;
};
rpu0vdev0buffer: rpu0vdev0buffer@3ed48000 {
no-map;
reg = <0x0 0x3ed48000 0x0 0x100000>;
};
rproc_0_reserved: rproc@3ed00000 {
no-map;
reg = <0x0 0x3ed00000 0x0 0x40000>;
};
};
tcm_0a@ffe00000 {
no-map;
reg = <0x0 0xffe00000 0x0 0x10000>;
phandle = <0x40>;
status = "okay";
compatible = "mmio-sram";
power-domain = <&zynqmp_firmware 15>;
};
tcm_0b@ffe20000 {
no-map;
reg = <0x0 0xffe20000 0x0 0x10000>;
phandle = <0x41>;
status = "okay";
compatible = "mmio-sram";
power-domain = <&zynqmp_firmware 16>;
};
rf5ss@ff9a0000 {
compatible = "xlnx,zynqmp-r5-remoteproc";
xlnx,cluster-mode = <0>; /*Lock-step mode*/
ranges;
reg = <0x0 0xFF9A0000 0x0 0x10000>;
#address-cells = <0x2>;
#size-cells = <0x2>;
r5f_0 {
compatible = "xilinx,r5f";
#address-cells = <2>;
#size-cells = <2>;
ranges;
sram = <0x40 0x41>;
memory-region = <&rproc_0_reserved>, <&rpu0vdev0buffer>, <&rpu0vdev0vring0>, <&rpu0vdev0vring1>;
power-domain = <&zynqmp_firmware 7>;
mboxes = <&ipi_mailbox_rpu0 0>, <&ipi_mailbox_rpu0 1>;
mbox-names = "tx", "rx";
};
};
zynqmp_ipi1 {
compatible = "xlnx,zynqmp-ipi-mailbox";
interrupt-parent = <&gic>;
interrupts = <0 29 4>;
xlnx,ipi-id = <7>;
#address-cells = <1>;
#size-cells = <1>;
ranges;
/* APU<->RPU0 IPI mailbox controller */
ipi_mailbox_rpu0: mailbox@ff990600 {
reg = <0xff990600 0x20>,
<0xff990620 0x20>,
<0xff9900c0 0x20>,
<0xff9900e0 0x20>;
reg-names = "local_request_region",
"local_response_region",
"remote_request_region",
"remote_response_region";
#mbox-cells = <1>;
xlnx,ipi-id = <1>;
};
};
}; Device Tree Source Include file for ZynqMP with RPMsg in kernelspace and Remoteproc in Split mode
/ {
reserved-memory {
#address-cells = <2>;
#size-cells = <2>;
ranges;
rpu0vdev0vring0: rpu0vdev0vring0@3ed40000 {
no-map;
reg = <0x0 0x3ed40000 0x0 0x4000>;
};
rpu0vdev0vring1: rpu0vdev0vring1@3ed44000 {
no-map;
reg = <0x0 0x3ed44000 0x0 0x4000>;
};
rpu0vdev0buffer: rpu0vdev0buffer@3ed48000 {
no-map;
reg = <0x0 0x3ed48000 0x0 0x100000>;
};
rproc_0_reserved: rproc@3ed00000 {
no-map;
reg = <0x0 0x3ed00000 0x0 0x40000>;
};
rproc_1_reserved: rproc@3ef00000 {
no-map;
reg = <0x0 0x3ef00000 0x0 0x40000>;
};
rpu1vdev0vring0: rpu1vdev0vring0@3ef40000 {
no-map;
reg = <0x0 0x3ef40000 0x0 0x4000>;
};
rpu1vdev0vring1: rpu1vdev0vring1@3ef44000 {
no-map;
reg = <0x0 0x3ef44000 0x0 0x4000>;
};
rpu1vdev0buffer: rpu1vdev0buffer@3ef48000 {
no-map;
compatible = "shared-dma-pool";
reg = <0x0 0x3ef48000 0x0 0x100000>;
};
};
tcm_0a@ffe00000 {
no-map;
reg = <0x0 0xffe00000 0x0 0x10000>;
phandle = <0x40>;
status = "okay";
compatible = "mmio-sram";
power-domain = <&zynqmp_firmware 15>;
};
tcm_0b@ffe20000 {
no-map;
reg = <0x0 0xffe20000 0x0 0x10000>;
phandle = <0x41>;
status = "okay";
compatible = "mmio-sram";
power-domain = <&zynqmp_firmware 16>;
};
tcm_1a@ffe90000 {
no-map;
reg = <0x0 0xffe90000 0x0 0x10000>;
phandle = <0x42>;
status = "okay";
compatible = "mmio-sram";
power-domain = <&zynqmp_firmware 17>;
};
tcm_1b@ffeb0000 {
no-map;
reg = <0x0 0xffeb0000 0x0 0x10000>;
phandle = <0x43>;
status = "okay";
compatible = "mmio-sram";
power-domain = <&zynqmp_firmware 18>;
};
rf5ss@ff9a0000 {
compatible = "xlnx,zynqmp-r5-remoteproc";
xlnx,cluster-mode = <1>; /*Split mode*/
ranges;
reg = <0x0 0xFF9A0000 0x0 0x10000>;
#address-cells = <0x2>;
#size-cells = <0x2>;
r5f_0 {
compatible = "xilinx,r5f";
#address-cells = <2>;
#size-cells = <2>;
ranges;
sram = <0x40 0x41>;
memory-region = <&rproc_0_reserved>, <&rpu0vdev0buffer>, <&rpu0vdev0vring0>, <&rpu0vdev0vring1>;
power-domain = <&zynqmp_firmware 7>;
mboxes = <&ipi_mailbox_rpu0 0>, <&ipi_mailbox_rpu0 1>;
mbox-names = "tx", "rx";
};
r5f_1 {
compatible = "xilinx,r5f";
#address-cells = <2>;
#size-cells = <2>;
ranges;
sram = <0x42 0x43>;
memory-region = <&rproc_1_reserved>, <&rpu1vdev0buffer>, <&rpu1vdev0vring0>, <&rpu1vdev0vring1>;
power-domain = <&zynqmp_firmware 8>;
mboxes = <&ipi_mailbox_rpu1 0>, <&ipi_mailbox_rpu1 1>;
mbox-names = "tx", "rx";
};
};
zynqmp_ipi1 {
compatible = "xlnx,zynqmp-ipi-mailbox";
interrupt-parent = <&gic>;
interrupts = <0 29 4>;
xlnx,ipi-id = <7>;
#address-cells = <1>;
#size-cells = <1>;
ranges;
/* APU<->RPU0 IPI mailbox controller */
ipi_mailbox_rpu0: mailbox@ff990600 {
reg = <0xff990600 0x20>,
<0xff990620 0x20>,
<0xff9900c0 0x20>,
<0xff9900e0 0x20>;
reg-names = "local_request_region",
"local_response_region",
"remote_request_region",
"remote_response_region";
#mbox-cells = <1>;
xlnx,ipi-id = <1>;
};
};
zynqmp_ipi2 {
compatible = "xlnx,zynqmp-ipi-mailbox";
interrupt-parent = <&gic>;
interrupts = <0 30 4>;
xlnx,ipi-id = <8>;
#address-cells = <1>;
#size-cells = <1>;
ranges;
/* APU<->RPU1 IPI mailbox controller */
ipi_mailbox_rpu1: mailbox@ff990640 {
reg = <0xff3f0b00 0x20>,
<0xff3f0b20 0x20>,
<0xff3f0940 0x20>,
<0xff3f0960 0x20>;
reg-names = "local_request_region",
"local_response_region",
"remote_request_region",
"remote_response_region";
#mbox-cells = <1>;
xlnx,ipi-id = <2>;
};
};
}; Device Tree Source Include file for Versal with RPMsg in kernelspace and Remoteproc in Lock-step mode
/ {
reserved-memory {
#address-cells = <2>;
#size-cells = <2>;
ranges;
rproc_0_reserved: rproc@3ed00000 {
no-map;
reg = <0x0 0x3ed00000 0x0 0x40000>;
};
rpu0vdev0vring0: rpu0vdev0vring0@3ed40000 {
no-map;
reg = <0x0 0x3ed40000 0x0 0x4000>;
};
rpu0vdev0vring1: rpu0vdev0vring1@3ed44000 {
no-map;
reg = <0x0 0x3ed44000 0x0 0x4000>;
};
rpu0vdev0buffer: rpu0vdev0buffer@3ed48000 {
no-map;
reg = <0x0 0x3ed48000 0x0 0x100000>;
};
};
tcm_0a@ffe00000 {
no-map;
reg = <0x0 0xffe00000 0x0 0x10000>;
phandle = <0x40>;
status = "okay";
compatible = "mmio-sram";
power-domain = <&versal_firmware 0x1831800b>;
};
tcm_0b@ffe20000 {
no-map;
reg = <0x0 0xffe20000 0x0 0x10000>;
phandle = <0x41>;
status = "okay";
compatible = "mmio-sram";
power-domain = <&versal_firmware 0x1831800c>;
};
rf5ss@ff9a0000 {
compatible = "xlnx,zynqmp-r5-remoteproc";
#address-cells = <0x2>;
#size-cells = <0x2>;
ranges;
xlnx,cluster-mode = <0>; /*Lock-step mode*/
reg = <0x0 0xff9a0000 0x0 0x10000>;
r5f_0 {
compatible = "xilinx,r5f";
#address-cells = <0x2>;
#size-cells = <0x2>;
ranges;
sram = <0x40 0x41>;
memory-region = <&rproc_0_reserved>, <&rpu0vdev0buffer>, <&rpu0vdev0vring0>, <&rpu0vdev0vring1>;
power-domain = <&versal_firmware 0x18110005>;
mboxes = <&ipi_mailbox_rpu0 0>, <&ipi_mailbox_rpu0 1>;
mbox-names = "tx", "rx";
};
};
zynqmp_ipi1 {
compatible = "xlnx,zynqmp-ipi-mailbox";
interrupt-parent = <&gic>;
interrupts = <0 33 4>;
xlnx,ipi-id = <5>;
#address-cells = <1>;
#size-cells = <1>;
ranges;
/* APU<->RPU0 IPI mailbox controller */
ipi_mailbox_rpu0: mailbox@ff990600 {
reg = <0xff3f0ac0 0x20>,
<0xff3f0ae0 0x20>,
<0xff3f0740 0x20>,
<0xff3f0760 0x20>;
reg-names = "local_request_region",
"local_response_region",
"remote_request_region",
"remote_response_region";
#mbox-cells = <1>;
xlnx,ipi-id = <3>;
};
};
};
Device Tree Source Include file for Versal with RPMsg in kernelspace and Remoteproc in Split mode RPU Firmware
In rsc_table.c , ensure RING_TX and RING_RX are set as FW_RSC_U32_ADDR_ANY, where FW_RSC_U32_ADDR_ANY = (-1). The kernel driver relies on this value to update the resource table with vring addresses.
In platform_info.c you can find the below address, size and offset assignment which should match the device tree description of shared buffers.
RPMsg in Userspace Figure: OpenAMP RPMsg Implementation in Linux Userspace
The OpenAMP library can also be used in Linux userspace. However, in this scenario, its important to match up the Linux userspace application, kernel device tree and RPU firmware with respect to the vring address locations and shared memory address.
Modifications for RPMsg in user space Refer to the OpenAMP Application Build Process page for Building Linux Distribution using PetaLinux , for modifying Demo Tests Firmware , and for booting up and running demo tests in userspace. OpenAMP version specific modifications are covered below; for example, changes in the device tree, RPU firmware, Linux application, etc.
2022.1 Example Device Tree
Device tree file location: <plnx-proj-root>/project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi
Zynqmp - RPMsg in user space Linux application
RPU Firmware
OpenAMP's implementation of RPMsg in userspace only allows for static vring entries so make sure to change it to the address as detailed below.
RPU firmware resource table location
