Subsystem Steps

• Define the subsystem using the isolation configuration menu in Vivado and export to hdf.
• Prepare Petalinux project with following changes w.r.t default 2018.3 bsp
  • Modify device tree source file (dts) of Linux to exclude PL IPs that are not part of APU subsystem.
  • Add rpu_firmware application binaries 
  • Add remoteproc.dtsi to allow remoteproc to load the rpu binaries
  • Add pmu-fw and ATF flags to enable warm restart, recovery, escalation
  • Add pmu-fw code patches to idle PL peripheral for given master
• For RPU subsystem restart, make PM aware rpu applications. Make linker script inline with remoteproc.dtsi
• Build using Petalinux.

An Example

The following figure shows the example design described by this document.  It consists of an APU subsystem, RPU subsystem, and miscellaneous PL connections.  It was built from the Xilinx ZCU102 Board Support Package (BSP) hardware design, and enhanced to have more hardware components added and isolation enabled.

The APU masters consist of an AXI CDMA and an AXI DMA, each with their own interconnect and firewall as shown below.

APU slaves consist of a BRAM controller, GPIO, and a blinking LED using pl_clk0 from Zynq UltraScale+ MPSoC.

The RPU slaves block contains a BRAM controller, AXI GPIO, and blinking LED as shown below.

1. A hardware design file (hdf) which defines three subsystems, APU (including APU_secure), RPU and RPU1.  PMU Firmware subsystem is included by default. Details on subsystems can be found here.
2. The hdf also contains PL components added to APU and RPU subsystems, including some DMAs to demonstrate how to idle a PL master.
   
   • Note: Custom hooks are needed to idle the PL masters of the given subsystem.  Refer to the UG1137 Software Developer's Guide, Chapter 12 for description of what is required.
3. Recovery with healthy bit escalation scheme.
   • Recovery and escalation are enabled with build flags in pmu recipe and warm restart flag in ATF.
   • Includes application (wdt-heartbeat) to provide heartbeats to the FPD wdt.
4. Enable XMPU / XPPU based configuration only for DDR and OCM with flag in FSBL recipe. No XMPU / XPPU protection for any other components.
5. Helper script (installed through mpsrm-init) to facilitate setting /unsetting of healthy bit, and starting/stopping of wdt-heartbeat application.
6. RPU application support. Application recipe to copy prebuild RPU application for r5-0 and r5-1 in the /lib/firmware directory of the rootfs

• Note: The device-tree changes needed for the R5 application are not needed in meta-user as the default device tree generated with rpu subsystem will generate the necessary overlay.

1. Download the default 2018.3 ZCU102 BSP from the Xilinx web page and save to your working directory.   

2. Create new project from the default BSP.

   # Create the Petalinux project from the BSP
   $ petalinux-create -t project -s xilinx-zcu102-v2018.3-final.bsp -n my_project
   

3. Change working directory to newly created Petalinux project directory.

   $ cd my_project
   

4. Configure Petalinux to use new hardware hdf (pre-built/newly created).

   # Point Petalinux to the new hardware, which is provided in the zip file, so that the appropriate drivers are included in the Petalinux build.
   # Use this path if using hardware from the example package
   $ petalinux-config --oldconfig --get-hw-description=</path/to/package>/hardware/sdk_export/
   
   # If using hardware from another source, point to that location instead of the package location
   $ petalinux-config --oldconfig --get-hw-description=</path/to/hdf>/
   

5. Copy (overwrite) meta-user directory from the package to project-spec.

   # Add / modify meta-user recipes as needed for the example.
   $ cp -r </path/to/package>/software/petalinux-patch/meta-user  project-spec/
   

6. (optional) Copy newly created RPU firmware images. Default images are already present.

   # Copy R5 Images
   $ cp </path/to/sdk/workspace>/r50_app/Debug/r50_app.elf  project-spec/meta-user/recipes-apps/rpu-firmware/files/data/r50_app
   $ cp </path/to/sdk/workspace>/r51_app/Debug/r51_app.elf  project-spec/meta-user/recipes-apps/rpu-firmware/files/data/r51_app
   

7. Add rpu-firmware to rootfs

   $ petalinux-config -c rootfs
   # enable user-packages > rpu-firmware

          

8. Build the petalinux project

   # Build all the petalinux components
   $ petalinux-build
   

   This will generate <plnx-root>/images/linux/image.ub file which is kernel image (along with dtb and rootfs).

9. Generate the BOOT image.

   # Create BOOT.BIN for sd boot (Add --force flag if BOOT.BIN already exist)
   $ petalinux-package --boot --fsbl --pmufw --fpga --atf --uboot
   

   This will generate <plnx-root>/images/linux/BOOT.BIN

Copy image.ub and BOOT.BIN to the sd card.

For more information:

How To Run

To run the design, load the SD card with images mentioned above and boot the zcu102 board with sd boot mode.

To log in, use root as login and password.

xilinx-zcu102-2018-3 login: root
Password: root

$ set_reboot apu
$ reboot

$ echo stop > /sys/class/remoteproc/remoteproc1/state

Recovery and Escalations

Normal Healthy System

1. Set the Healthy bit
2. Start watchdog application as daemon to kick the WDT

Following are the description of various argument that watchdog-int script (available at /etc/init.d/watchdog-init) can take

# Description:       This script runs the daemon for providing heartbeat
#                    to FPD wdt, failing which PMU will trigger restart
#                    escalation. This script also mark the current boot
#                    as healthy by setting healthy bit.
#             Following arguments can be passed to this script:
#            start --> Set the healthy bit and start wdt heartbeats
#            stop --> Stop wdt heartbeats (Note: this gracefully shutdowns the wdt and disable the recovery)
#            restart --> stop followed by start
#            set_healthy -> mark healthy status as true
#            unset_healthy -> mark healthy status as false
#            start_hb --> start wdt heartbeats
#            stop_hb    --> stop wdt heartbeats (Note: this gracefully shutdowns the wdt and disable the recovery)

How to Build Example Design Hardware

The following steps can be used to generate the hardware description file (HDF) used in this example. 

Copy sources from the package to a working directory, then open the project in Vivado using the provided Tcl script:

$ cp </path/to/package>/hardware/sources/* </path/to/working/directory>/.
$ vivado -source create_warm_restart.tcl

When the script finishes, you should see components in your block design for MPSOC, APU_SS, RPU_SS, and PL_SS.  You should also have a constraint file added under the Sources tab.  You can double-click the Zynq UltraScale+ MPSOC component, click on the box for Switch to Advanced Mode, then click on Isolation Configuration to see how the subsystems are configured.

Click on Generate Bitstream using the link under Flow Navigator, saving the block design when prompted.

When the bitstream is ready, go to File → Export → Export Hardware and select Include Bitstream.  Use the default location for the exported hardware. Then go to File → Launch SDK using the default location and workspace to finish creating the HDF needed for building Petalinux.  Follow the steps for Building Petalinux Project with new hardware (Step 4) to use this new hardware in the Petalinux image.