This tutorial explains procedure to measure transition times and respected power values when either PS or PL suspends or wake up. By following below procedure, suspend and wake up time of APU, These procedures are for 2017.1 and later releases. There are some differences for later releases which are indicated when appropriate.
RPU application: rpu_deep_sleep.elf
Workspace for RPU baremetal: sdk_ws.zip
Vitis workspace for RPU application for v2020.1: vitis-ws-20.1.zip
Petalinux 2020.1 generated images (including rpu_0.elf, boot.bif and BOOT.BIN): linux_images_2020.1.tar.gz
Petalinux 2019.1 generated images (including rpu_0.elf, boot.bif and BOOT.BIN): linux_images_2019.1.tar.gz
Petalinux 2018.3 generated images (including rpu_0.elf, boot.bif and BOOT.BIN): linux_images_2018.3.tar.gz
Petalinux 2018.2 generated images (including rpu_0.elf, boot.bif and BOOT.BIN): linux_images_2018.2.tar.gz
To generate binaries on your own please refer below steps for generating required images/binaries.
Run below commands from bash terminal to create petalinux project.
source <petalinux-install-dir>/settings.sh petalinux-create -t project -s xilinx-zcu102-v20xx.x.bsp |
Create directory <plnx-proj-root>/project-spec/meta-user/recipes-kernel/linux/linux-xlnx/ (if not present)
Copy below patch files to <plnx-proj-root>/project-spec/meta-user/recipes-kernel/linux/linux-xlnx/. (v2018.x - later)
Rename the downloaded patch file to pm_wakeup_latency.patch.
pm_wakeup_latency.patch (for v2017.1-4)
pm_wakeup_latency_2018_1_3.patch (for v2018.x)
pm_wakeup_latency_2019_1_x.patch (for v2019.1 - later)
mailbox.patch (for v2018.2 only)
pm_firmware_debug.cfg (for v2018.3 - later)
Open file <plnx-proj-root>/project-spec/meta-user/recipes-kernel/linux/linux-xlnx_%.bbappend (create if not present)
Add below lines in file:
For v2017.x - v2018.1
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
SRC_URI_append = " file://pm_wakeup_latency.patch" |
For v2018.2 only
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
SRC_URI_append = " file://mailbox.patch"
SRC_URI_append = " file://pm_wakeup_latency.patch" |
For v2018.3 - 2019.1
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
SRC_URI_append = " file://pm_wakeup_latency.patch"
SRC_URI_append = " file://pm_firmware_debug.cfg" |
For v2019.2-later
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
SRC_URI += " file://pm_wakeup_latency.patch"
SRC_URI_append = " file://pm_firmware_debug.cfg" |
Copy kernel min config file (defconfig) to project-spec/meta-user/recipes-kernel/linux/files/defconfig. (v2018.1 & v2018.2) Rename the downloaded config file to 'defconfig'.
defconfig (for v2017.1-4)
defconfig_2018_x (for v2018.1 & v2018.2)
Add below line in project-spec/meta-user/recipes-kernel/linux/linux-xlnx_%.bbappend file:
SRC_URI_append = " file://defconfig" |
Isolate SATA node from Linux use by removing its node from device tree. Add the following to <plnx-proj-root>/project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi.
&amba {
/delete-node/ ahci@fd0c0000;
}; |
First enable ECC in FSBL code:
Create directory <plnx-proj-root>/project-spec/meta-user/recipes-bsp/fsbl/files/ (if not present)
Copy patch file to <plnx-proj-root>/project-spec/meta-user/recipes-bsp/fsbl/files/ (Patch attached here: ecc_fsbl.patch)
Open file <plnx-proj-root>/project-spec/meta-user/recipes-bsp/fsbl/fsbl_%.bbappend (create if not present)
Add below lines in file:
For 2017.x-2019.1
FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
SRC_URI_append = " file://ecc_fsbl.patch" |
For 2019.2-later
FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
SRC_URI += " file://ecc_fsbl.patch" |
Now build petalinux using below command
petalinux-build -c kernel |
Start with an empty RPU application (like the Hello World example here).
Configure the RPU to run from TCM in split mode. (Refer ZU＋ Example - PM Hello World#RPU to run from TCM)
Replace the main.c with this file (rpu.c (for 2017.x-2019.1), rpu-2019.2.c (for 2019.2-later))
Build application elf.
Create RPU_0 application rpu_0.elf from SDK as described in above section.
Create a new folder and copy pmufw.elf, zynqmp_fsbl.elf, bl31.elf and u-boot.elf from petalinux generated images (present at <plnx-proj-root>/images/linux/). Also copy rpu_0.elf into same folder.
Create boot.bif file in same folder as shown below.
for 2017.x-2019.2
the_ROM_image:
{
[fsbl_config] a53_x64
[pmufw_image] pmufw.elf
[bootloader,destination_cpu=a53-0] zynqmp_fsbl.elf
[bootloader,destination_cpu=r5-0] rpu_0.elf
[destination_cpu=a53-0, exception_level=el-3,trustzone] bl31.elf
[destination_cpu=a53-0, exception_level=el-2] u-boot.elf
} |
For 2020.1-later
the_ROM_image:
{
[bootloader, destination_cpu=a53-0] fsbl.elf
[pmufw_image] pmufw.elf
[destination_cpu=r5-0] rpu.elf
[destination_cpu=a53-0,exception_level=el-3,trustzone] bl31.elf
[destination_cpu=a53-0, load=0x00100000] system.dtb
[destination_cpu=a53-0,exception_level=el-2] u-boot.elf
} |
Create BOOT.BIN file using following command.
bootgen -arch zynqmp -image boot.bif -w -o BOOT.BIN |
Create a boot partition in SD card and copy BOOT.BIN and image.ub file (present at <plnx-proj-root>/images/linux/) to boot partition.
Boot the ZCU102 board in SD boot mode.
Once boot start U-boot prompt will appear on serial. Press Enter again and again to interrupt u-boot till ZynqMP prompt appear. Run below command from u-boot to disable cpuidle.
ZynqMP > setenv bootargs ${bootargs} cpuidle.off=1 |
After that start the linux by writing "boot" on the U-boot prompt
Use Power Advantage tool for measuring power values.
Run below command to load all APU from Linux terminal on board and measure the power using Power Advantage Tool. That will give power values for All on (PS+PL).
root@plnx_aarch64:~# fulload() { dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null &};fulload; read; |
Turn OFF PL
Run below command from Linux terminal on board to power off PL.
(For v2017.1-4)
root@plnx_aarch64:~# time echo release_node 69 > /sys/kernel/debug/zynqmp_pm/power |
(For v2018.2 and above)
root@plnx_aarch64:~# time echo pm_release_node 69 > /sys/kernel/debug/zynqmp-firmware/pm |
Turn ON PL
Run below command from Linux terminal on board to power on PL.
(For v2017.1-4)
root@plnx_aarch64:~# time echo request_node 69 > /sys/kernel/debug/zynqmp_pm/power |
(For v2018.2 and above)
root@plnx_aarch64:~# time echo pm_request_node 69 > /sys/kernel/debug/zynqmp-firmware/pm |
Power OFF 3 APUs
Run below command from Linux terminal on board to power off 3 APUs. Use Power Advantage Tool to measure power for PS all OFF.
root@plnx_aarch64:~# time echo 0 > /sys/devices/system/cpu/cpu1/online root@plnx_aarch64:~# time echo 0 > /sys/devices/system/cpu/cpu2/online root@plnx_aarch64:~# time echo 0 > /sys/devices/system/cpu/cpu3/online |
Power ON 3 APUs
Run below command from Linux terminal on board to power on 3 APUs. Use Power Advantage Tool to measure power for PS all ON.
root@plnx_aarch64:~# time echo 1 > /sys/devices/system/cpu/cpu1/online root@plnx_aarch64:~# time echo 1 > /sys/devices/system/cpu/cpu2/online root@plnx_aarch64:~# time echo 1 > /sys/devices/system/cpu/cpu3/online |
Run following command to get available frequencies.
root@plnx_aarch64:~# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies |
Set minimum frequency
Run following command to set minimum frequency
root@plnx_aarch64:~# time echo <min freq. from above command> > /sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed |
Set maximum frequency
Run following command to set maximum frequency
root@plnx_aarch64:~# time echo <max freq. from above command> > /sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed |
Power off all 3 secondary APUs and set frequency of primary APU0 to minimum using above commands for minimum latency measurement.
Run following command from Linux terminal to release unnecessary nodes and signal the RPU_0:
(For v2017.1-4)
root@plnx_aarch64:~# echo request_wakeup 8 1 0 1 > /sys/kernel/debug/zynqmp_pm/power root@plnx_aarch64:~# echo force_powerdown 8 > /sys/kernel/debug/zynqmp_pm/power root@plnx_aarch64:~# echo release_node 69 > /sys/kernel/debug/zynqmp_pm/power root@plnx_aarch64:~# echo 0 > /sys/module/printk/parameters/console_suspend root@plnx_aarch64:~# echo MMIO_WRITE 0xFFD80064 1 1 > /sys/kernel/debug/zynqmp_pm/power |
(For v2018.2)
root@plnx_aarch64:~# echo pm_request_wakeup 8 1 0 1 > /sys/kernel/debug/zynqmp-firmware/pm root@plnx_aarch64:~# echo pm_force_powerdown 8 > /sys/kernel/debug/zynqmp-firmware/pm root@plnx_aarch64:~# echo pm_release_node 69 > /sys/kernel/debug/zynqmp-firmware/pm root@plnx_aarch64:~# echo 0 > /sys/module/printk/parameters/console_suspend root@plnx_aarch64:~# echo 0x1 0x1 > /sys/firmware/zynqmp/pggs3 |
(For v2018.3)
root@plnx_aarch64:~# echo disabled > /sys/devices/platform/amba/ffa60000.rtc/power/wakeup root@plnx_aarch64:~# echo disabled > /sys/devices/platform/amba/ff000000.serial/power/wakeup root@plnx_aarch64:~# echo disabled > /sys/devices/platform/amba/ff010000.serial/power/wakeup root@plnx_aarch64:~# echo pm_release_node 69 > /sys/kernel/debug/zynqmp-firmware/pm root@plnx_aarch64:~# echo 0 > /sys/module/printk/parameters/console_suspend root@plnx_aarch64:~# echo 0x1 0x1 > /sys/firmware/zynqmp/pggs3 |
(For v2019.1 - Later)
root@plnx_aarch64:~# echo disabled > /sys/devices/platform/amba/ffa60000.rtc/power/wakeup root@plnx_aarch64:~# echo disabled > /sys/devices/platform/amba/ff000000.serial/power/wakeup root@plnx_aarch64:~# echo disabled > /sys/devices/platform/amba/ff010000.serial/power/wakeup root@plnx_aarch64:~# echo disabled > /sys/devices/platform/amba/ff0a0000.gpio/power/wakeup root@plnx_aarch64:~# echo pm_release_node 69 > /sys/kernel/debug/zynqmp-firmware/pm root@plnx_aarch64:~# echo 0 > /sys/module/printk/parameters/console_suspend root@plnx_aarch64:~# echo 0x1 0x1 > /sys/firmware/zynqmp/pggs3 |
Now, Linux will be suspended and it will be resumed by RPU after suspend. RPU has requested NODE_SATA from FPD, so Linux will be suspended with FPD on.
After 10 second of Linux suspend, RPU application will release NODE_SATA. Once NODE_SATA is released, FPD will be off. Use Power Advantage Tool in these 10 seconds to get power value for Suspend to RAM.
Following are two print messages to identify these 10 second window.
RPU0: ******************** RPU ON, APU suspend with FPD ON ****************** RPU0: (10 seconds delay) RPU0: Release NODE_SATA, FPD will be off after releasing NODE_SATA |
Once FPD is off, RPU will resume APU after 10 second of FPD off. To measure power with FPD off, Use Power Advantage Tool in these 10 seconds.
Following are two print messages to identify these 10 second window.
RPU0: ******************** RPU ON, APU suspend with FPD OFF ****************** RPU0: Suspended. RPU0: (10 seconds delay) RPU0: Resuming APU. |
Above loop will run for 3 times and print minimum, maximum and average values for latency. On console, you should able to see suspend/wakeup latency of APU from RPU print message as shown below.
RPU0: Request Suspend Latency in useconds of Node NODE_APU_0: Min: 42701, Max: 45728, Avg: 45058 RPU0: FPD off Latency in useconds of Node NODE_APU_0: Min: 5902, Max: 5902, Avg: 5902 RPU0: FPD on Latency in useconds of Node NODE_APU_0: Min: 121025, Max: 121251, Avg: 121116 RPU0: APU0 Wakeup Latency in useconds of Node NODE_APU_0: Min: 8061, Max: 8067, Avg: 8063 RPU0: Wakeup Latency in useconds of Node NODE_APU_0: Min: 14062, Max: 14088, Avg: 14080 |
Here,
RPU0: Request Suspend Latency in useconds of Node NODE_APU_0: Latency for power state transition from "3 APU off (min frequency)" to "Suspend to RAM".
RPU0: FPD off Latency in useconds of Node NODE_APU_0: Latency for power state transition from "Suspend to RAM" to "FPD off".
RPU0: FPD on Latency in useconds of Node NODE_APU_0: Latency for power state transition from FPD off to "Suspend to RAM".
RPU0: APU0 Wakeup Latency in useconds of Node NODE_APU_0: Time require to wakeup APU_0 after RequestWakeup call.
RPU0: Wakeup Latency in useconds of Node NODE_APU_0: Latency for power state transition from "Suspend to RAM" to "3 APU off (min frequency).
R5 Sleep and Deep-Sleep mode
Once above measurements are done, RPU puts FPD in off state and goes to idle mode (wfi state) for 10 seconds.
To measure power with R5 sleep mode, Use Power Advantage Tool in these 10 seconds.
Following are two print messages to identify these 10 second window.
RPU0: ******************** RPU idle, APU suspend with FPD OFF ****************** RPU0: Go to RPU Sleep mode RPU0: (10 seconds timer) |
After 10 seconds RPU gets timer interrupt after that it suspend itself and goes to deep sleep mode for 10 seconds.
After RPU goes to deep sleep, Use Power Advantage tool for measuring power values within these 10 seconds.
Following are two print messages to identify these 10 second window.
RPU0: *************************** Deep Sleep Mode ********************* |
After 10 seconds timer will generate interrupt and Resumes RPU. Then RPU resumes APU. This state is same as 3 APU off with minimum frequency.
After resuming, RPU will automatically execute the demo again.
RPU0: Resume APU RPU0: **************** Demo executed successfully ******************** RPU0: ******** Executing Typical power state demo again ************** |
baremetal.rar (for v2017.1-4)
v2018.3 SDK workspace for APU and RPU applications: ws_measure_rpu_latency_2018_3.zip; Binaries generated for v2018.3: rpu_latency_measurement_2018_3.tar.gz
v2019.1 SDK workspace for APU and RPU applications: ws_measure_rpu_latency_2019_1.zip; Binaries generated for v2019.1: rpu_latency_measurement_2019_1.tar.gz
v2020.1 Vitis workspace for APU and RPU applications: ws_measure_rpu_latency_2020.1.zip; Binaries generated for v2020.1: rpu_latency_measurement_2020.1.tar.gz
To generate images on your own please refer below steps for generating required images/binaries.
Start with an empty RPU application (like the Hello World example here).
Configure the RPU to run from TCM in split mode. (Refer ZU＋ Example - PM Hello World#RPU to run from TCM)
Replace the main.c with this file (rpu_0_target.c)
Build rpu_0 application elf.
Start with an empty APU application (like the Hello World example here).
Replace the main.c with this file (apu_host.c)
Build apu application elf.
Create rpu_0.elf and apu.elf from SDK as described in above section.
Create a new folder and copy pmufw.elf and zynqmp_fsbl.elf from petalinux generated images (present at <plnx-proj-root>/images/linux/). Also copy rpu_0.elf and apu.elf into same folder.
Create boot.bif file in same folder as shown below.
the_ROM_image:
{
[fsbl_config] a53_x64
[pmufw_image] pmufw.elf
[bootloader,destination_cpu=a53-0] zynqmp_fsbl.elf
[destination_cpu=r5-0] rpu_0.elf
[destination_cpu=a53-0] apu.elf
} |
Create BOOT.BIN file using following command.
bootgen -arch zynqmp -image boot.bif -w -o BOOT.BIN |
Create a boot partition in SD card and copy BOOT.BIN to boot partition.
Boot the ZCU102 board in SD boot mode and observe the output on uart console.
APU suspends and resumes RPU and measures its transition time.