/
Zynq UltraScale＋ MPSoC Power Management - Linux Kernel

Zynq UltraScale＋ MPSoC Power Management - Linux Kernel

May 27, 2024

Table of Contents

Prerequisites

  1. You have installed Petalinux 2018.1 (or later).

  2. You have created a Petalinux project using the BSP for your board.

  3. You were able to boot the pre-built images on your board.

  4. You have a stand-alone application running on the RPU that supports power management using the XilPM library. If not, see Minimal RPU Applications on how to run a minimal application on the RPU.

  5. The overviw of EEMI APIs described in https://www.xilinx.com/support/documentation/user_guides/ug1199-zynq-power-management.pdf

 

Out-of-box PM Demo

A script has been included in the pre-built PetaLinux images that demonstrate the power management features available to the Linux console. To run the demo script:

  • Boot the PetaLinux pre-built image on the board.

  • Log in on the Linux console, type: "hellopm"

Runtime PM

See:https://www.kernel.org/doc/Documentation/power/runtime_pm.txt

Quoting from https://www.kernel.org/doc/html/v4.12/driver-api/pm/devices.html:

  • "Devices may be put into low-power states while the system is running, independently of other power management activity in principle. However, devices are not generally independent of each other (for example, a parent device cannot be suspended unless all of its child devices have been suspended)."


See documentation on the Linux drivers for any specific runtime PM handling: Linux Drivers

CPU PM

CPU Hotplug

Kernel documentation: https://www.kernel.org/doc/Documentation/power/suspend-and-cpuhotplug.txt

The required kernel configuration options are:

  • Kernel Features

    • [*] Support for hot-pluggable CPUs

  • Power management options

    • [*] Suspend to RAM and standby


The user may take one or more APU cores on-line and off-line as needed via the CPU Hotplug control interface.

For example, to take CPU3 off-line:

$ echo 0 > /sys/devices/system/cpu/cpu3/online

CPU Idle

CPU cores are powered off when they are idling.

Kernel documentation:



The required kernel configuration options are:

  • CPU Power Management

    • CPU Idle

      • [*] CPU idle PM support

      • [*] Ladder governor (for periodic timer tick)

      • ARM CPU Idle Drivers

        • [*] Generic ARM/ARM64 CPU idle Driver


Below is the sysfs interface for cpuidle.

$ ls -lR /sys/devices/system/cpu/cpu0/cpuidle/ /sys/devices/system/cpu/cpu0/cpuidle/: drwxr-xr-x 2 root root 0 Nov 19 17:24 driver drwxr-xr-x 2 root root 0 Nov 19 17:24 state0 drwxr-xr-x 3 root root 0 Nov 19 17:24 state1 /sys/devices/system/cpu/cpu0/cpuidle/state0: -r--r--r-- 1 root root 4096 Nov 19 17:24 above -r--r--r-- 1 root root 4096 Nov 19 17:24 below -r--r--r-- 1 root root 4096 Nov 19 17:24 default_status -r--r--r-- 1 root root 4096 Nov 19 17:24 desc -rw-r--r-- 1 root root 4096 Nov 19 17:24 disable -r--r--r-- 1 root root 4096 Nov 19 17:24 latency -r--r--r-- 1 root root 4096 Nov 19 17:24 name -r--r--r-- 1 root root 4096 Nov 19 17:24 power -r--r--r-- 1 root root 4096 Nov 19 17:24 rejected -r--r--r-- 1 root root 4096 Nov 19 17:24 residency -r--r--r-- 1 root root 4096 Nov 19 17:24 time -r--r--r-- 1 root root 4096 Nov 19 17:24 usage /sys/devices/system/cpu/cpu0/cpuidle/state1: -r--r--r-- 1 root root 4096 Nov 19 17:27 above -r--r--r-- 1 root root 4096 Nov 19 17:27 below -r--r--r-- 1 root root 4096 Nov 19 17:27 default_status -r--r--r-- 1 root root 4096 Nov 19 17:27 desc -rw-r--r-- 1 root root 4096 Nov 19 17:27 disable -r--r--r-- 1 root root 4096 Nov 19 17:27 latency -r--r--r-- 1 root root 4096 Nov 19 17:27 name -r--r--r-- 1 root root 4096 Nov 19 17:27 power -r--r--r-- 1 root root 4096 Nov 19 17:27 rejected -r--r--r-- 1 root root 4096 Nov 19 17:27 residency drwxr-xr-x 2 root root 0 Nov 19 17:27 s2idle -r--r--r-- 1 root root 4096 Nov 19 17:27 time -r--r--r-- 1 root root 4096 Nov 19 17:27 usage

where,

  • desc : Small description about the idle state (string)

  • disable : Option to disable this idle state (bool) -> see note below

  • latency : Latency to exit out of this idle state (in microseconds)

  • name : Name of the idle state (string)

  • power : Power consumed while in this idle state (in milliwatts)

  • time : Total time spent in this idle state (in microseconds)

  • usage : Number of times this state was entered (count)


Below is the sysfs interface for cpuidle governors.

$ ls -lR /sys/devices/system/cpu/cpuidle/ /sys/devices/system/cpu/cpuidle/: -r--r--r-- 1 root root 4096 Nov 19 17:25 available_governors -r--r--r-- 1 root root 4096 Nov 19 17:25 current_driver -rw-r--r-- 1 root root 4096 Nov 19 17:25 current_governor -r--r--r-- 1 root root 4096 Nov 19 17:25 current_governor_ro

CPU Frequency

Adjust CPU frequency at runtime.

Kernel documentation:



The required kernel configuration options are:

  • CPU Power Management

    • CPU Frequency scaling

      • [*] CPU Frequency scaling

        • Default CPUFreq governor

          • Userspace

        • <*> Generic DT based cpufreq driver


Read current CPU frequency (same for all cores):

$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq


Look up the available governors speeds (same for all cores):

$ cat /sys/devices/system/cpu/cpu1/cpufreq/scaling_available_governors


Select the 'userspace' governor for CPU frequency control (same for all cores):

$ echo userspace > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor


Look up the available CPU speeds (same for all cores):

$ cat /sys/devices/system/cpu/cpu1/cpufreq/scaling_available_frequencies


Change the CPU speed (same for all cores):

$ echo <frequency> > /sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed


Zynq UltraScale+ MPSoC Power Management - Linux Kernel

This page provides tips and examples of Linux kernel power management solutions for the Zynq UltraScale+ MPSoC.

System PM

Suspend

The kernel is suspended when the CPU and most of the peripherals are powered down. The system run states needed to resume from suspend is stored in the DRAM, which is put into self-refresh mode.

The required kernel configuration option settings are:

  • Power management options

    • [*] Suspend to RAM and standby

    • [*] User space wakeup sources interface

    • [*] Device power management core functionality

 

  • Device Drivers

    • SOC (System On Chip) specific Drivers

      •  

        • XIlinx SoC drivers

          • Zynq MPSoC SoC

            • [*] Enable Xilinx Zynq MPSoC Power Management Driver

            • [*] Enable Zynq MPSoC generic PM domains

  • Firmware Drivers

    • Zynq MPSoC Firmware Drivers

      • -*- Enable Xilinx Zynq MPSoC firmware interface



Type the following command to suspend the kernel. (Note: you must enable a wake-up source first otherwise the kernel will never resume.)

$ echo mem > /sys/power/state

Wake-up Source

The kernel resumes from the suspend mode when a wake-up event occurs.

UART

To wake up the APU on UART input:

  • For ZynMP

$ echo enabled > /sys/devices/platform/axi/ff000000.serial/tty/ttyPS0/power/wakeup

Note: For version before 2022.1:

$ echo enabled > /sys/devices/platform/amba/ff000000.serial/tty/ttyPS0/power/wakeup

 

  • For Versal

$ echo enabled > /sys/devices/platform/axi/ff000000.serial/tty/ttyAMA0/power/wakeup

Note: For version before 2022.1:

$ echo enabled > /sys/devices/platform/amba/ff000000.serial/tty/ttyAMA0/power/wakeup

USB

To wake up the APU on USB input:

$ echo 0 > /sys/module/printk/parameters/console_suspend (optional) $ echo enabled > /sys/devices/platform/axi/ff000000.serial/tty/ttyPS0/power/wakeup (optional, this is UART wakeup source) #Note: For version before 2022.1: #$ echo enabled > /sys/devices/platform/amba/ff000000.serial/tty/ttyPS0/power/wakeup $ echo enabled > /sys/bus/usb/devices/usb1/power/wakeup (set USB as wakeup source) /* Based on the device connected the below needs to be done */ $ echo enabled > /sys/bus/usb/devices/usb1/1-1/power/wakeup Suspend Linux: $ echo mem > /sys/power/state

Note: To Wake up using USB, Generate wakeup signal from the connected USB device.

Wake on LAN

To wake up the APU on Ethernet input:

  • For ZynqMP

 

// without FPD off / Select GEM as wake device $ echo enabled > /sys/devices/platform/axi/ff0e0000.ethernet/net/eth0/power/wakeup # Note: For version before 2022.1: # $ echo enabled > /sys/devices/platform/amba/ff0e0000.ethernet/net/eth0/power/wakeup / Suspend $ echo mem > /sys/power/state //for FPD off $ echo pm_request_wakeup 8 1 0 1 > /sys/kernel/debug/zynqmp-firmware/pm $ echo pm_force_powerdown 6 > /sys/kernel/debug/zynqmp-firmware/pm / Select GEM as wake device $ echo enabled > /sys/devices/platform/axi/ff0e0000.ethernet/net/eth0/power/wakeup # Note: For version before 2022.1: # $ echo enabled > /sys/devices/platform/amba/ff0e0000.ethernet/net/eth0/power/wakeup // Suspend $ echo mem > /sys/power/state

  • For Versal

/ Select GEM as wake device $ echo enabled > /sys/devices/platform/axi/ff0c0000.ethernet/net/eth0/power/wakeup # Note: For version before 2022.1: # $ echo enabled > /sys/devices/platform/amba/ff0c0000.ethernet/net/eth0/power/wakeup $ echo mem > /sys/power/state

Note: To Wake up using Ethernet, Please ping the specific interface's IP address.

GPIO

The required kernel configuration options are:

  • Device Drivers

    • [*] GPIO Support

      • [*] /sys/class/gpio/... (sysfs interface)

      • Memory mapped GPIO drivers

        • [*] Xilinx GPIO support

        • [*] Xilinx Zynq GPIO support

To wake up the APU on the GPIO pin:

$ echo enabled > /sys/devices/platform/gpio-keys/power/wakeup

RTC

The required kernel configuration options are:

  • Device Drivers

    • Real Time Clock

      • [*] Xilinx Zynq Ultrascale+ MPSoC RTC


To wake up the RTC to wake up the APU after 10 seconds:

$ echo +10 > /sys/class/rtc/rtc0/wakealarm

Global Storage Registers

Global General Storage Registers and Persistent Global General Storage Registers available for general use can be accessed through firmware sysfs files.

Path to device file (firmware-node) for 2018.1 and above release is:

/sys/firmware/zynqmp/

Path to device file (firmware-node) for 2017.4 and earlier release is:

/sys/devices/platform/firmware/

Global General Storage Registers

Four 32-bit storage registers are available for general use. Their values are not preserved across after software reboots.

For ZynqMP:

Device file

MMIO Register

MMIO Access

Valid Value Range

<firmware-node>/ggs0

GLOBAL_GEN_STORAGE0

0xFFD80030

0x00000000 - 0xFFFFFFFF

<firmware-node>/ggs1

GLOBAL_GEN_STORAGE1

0xFFD80034

0x00000000 - 0xFFFFFFFF

<firmware-node>/ggs2

GLOBAL_GEN_STORAGE2

0xFFD80038

0x00000000 - 0xFFFFFFFF

<firmware-node>/ggs3

GLOBAL_GEN_STORAGE3

0xFFD8003C

0x00000000 - 0xFFFFFFFF

For Versal:

Note: PLM uses PMC RAM for storing GGS values as Life of GGS registers and PMC RAM is same. So there is no fixed hardware register is involved.

Device file

MMIO Register

MMIO Access

Valid Value Range

<firmware-node>/ggs0

NA

NA

0x00000000 - 0xFFFFFFFF

<firmware-node>/ggs1

NA

NA

0x00000000 - 0xFFFFFFFF

<firmware-node>/ggs2

NA

NA

0x00000000 - 0xFFFFFFFF

<firmware-node>/ggs3

NA

NA

0x00000000 - 0xFFFFFFFF


Read the value of a global storage register:

$ cat /sys/devices/platform/firmware:zynqmp-firmware/ggs0 # Note: For version before 2022.1 # cat /sys/firmware-zynqmp/ggs0

Write the mask and value of a global storage register:

$ echo 0xFFFFFFFF 0x1234ABCD > /sys/devices/platform/firmware:zynqmp-firmware/ggs0 # Note: For version before 2022.1 # echo 0xFFFFFFFF 0x1234ABCD > /sys/firmware-zynqmp/ggs0

Persistent Global General Storage Registers

Four 32-bit persistent global storage registers are available for general use. Their values are preserved across after software reboots.

For ZynqMP

Device Name

MMIO Register

MMIO Access

Valid Value Range

<firmware-node>/pggs0

PERS_GLOB_GEN_STORAGE0

0xFFD80050

0x00000000 - 0xFFFFFFFF

<firmware-node>/pggs1

PERS_GLOB_GEN_STORAGE1

0xFFD80054

0x00000000 - 0xFFFFFFFF

<firmware-node>/pggs2

PERS_GLOB_GEN_STORAGE2

0xFFD80058

0x00000000 - 0xFFFFFFFF

<firmware-node>/pggs3

PERS_GLOB_GEN_STORAGE3

0xFFD8005C

0x00000000 - 0xFFFFFFFF

For Versal

Device Name

MMIO Register

MMIO Access

Valid Value Range

<firmware-node>/pggs0

PMC_PERS_GLOB_GEN_STORAGE3

0xF111005C

0x00000000 - 0xFFFFFFFF

<firmware-node>/pggs1

PMC_PERS_GLOB_GEN_STORAGE4

0xF1110060

0x00000000 - 0xFFFFFFFF

Read the value of a persistent global storage register:

$ cat /sys/devices/platform/firmware\:zynqmp-firmware/pggs0 # Note: For version before 2022.1 # cat /sys/firmware-zynqmp/pggs0

Write the mask and value of a persistent global storage register:

$ echo 0xFFFFFFFF 0x1234ABCD > /sys/devices/platform/firmware\:zynqmp-firmware/pggs0 # Note: For version before 2022.1 # echo 0xFFFFFFFF 0x1234ABCD > /sys/firmware-zynqmp/pggs0

Debugfs

Kernel documentation:

 

The debugfs interface is intended for testing and debugging the integration between the Linux kernel and the Zynq UltraScale+ MPSoC power management framework. This interface must be used with a lot of care. In fact, accessing this interface during normal PM operation will very likely cause unexpected problems. Please refer to UG1137 for the usage of PM API.

Starting with 2018.1, this interface is disabled by default. To enable this interface, change the following kernel configurations (in this order):

  • Power management options

    • [*] Suspend to RAM and standby

    • [*] User space wakeup sources interface

    • [*] Device power management core functionality

  • Kernel hacking

    • Compile-time checks and compiler options

      • [*] Debug Filesystem

  • Firmware Drivers

    • Zynq MPSoC Firmware Drivers

      • -*- Enable Xilinx Zynq MPSoC firmware interface

      • [*] Enable Xilinx Zynq MPSoC firmware debug APIs



Access to the debugfs interface is provided through the following node in the sysfs:

For 2018.1 and later releases:

/sys/kernel/debug/zynqmp-firmware/pm

Note: For 2018.1 and later releases, output of debug commands (if any) would not be printed on console in dmesg. it needs to be explicitly read by reading above "pm" debugfs file as below:

cat /sys/kernel/debug/zynqmp-firmware/pm


For 2017.4 and earlier releases:

/sys/kernel/debug/zynqmp_pm/power

For example,

$ echo pm_request_node 22 1 100 1 > (sysfs node)

See UG1137 for the input parameters to the APIs for ZynqMP Platform. For Versal refer UG1304.


Note: For 2018.1 and later releases, API names are updated with "pm_" prefix.

Get API Version

Get the API version.

$ echo pm_get_api_version > (sysfs node)

Request Suspend

Request another PU to suspend itself.

$ echo pm_request_suspend <node> > (sysfs node)

Self Suspend

Notify PMU that this PU is about to suspend itself.

$ echo pm_self_suspend <node> > (sysfs node)

Force Power Down

Force another PU to power down.

$ echo pm_force_powerdown <node> > (sysfs node)

Abort Suspend

Notify PMU that the attempt to suspend has been aborted.

$ echo pm_abort_suspend > (sysfs node)

Request Wake-up

Request another PU to wake up from suspend state.

$ echo pm_request_wakeup <node> <set_address> <address> > (sysfs node)

Set Wake-up Source

Set up a node as the wake-up source.

$ echo pm_set_wakeup_source <target> <wkup_node> <enable> > (sysfs node)

System Shutdown

Shut down the PU's own sub-system.

$ echo pm_system_shutdown <action> <scope> > (sysfs node)

Action:

  • 0: Shutdown

  • 1: Reset

  • 2: Set reset scope

Scope:

  • 0: APU

  • 1: PS

  • 2: System

Request Node

Request to use a node.

$ echo pm_request_node <node> > (sysfs node)

Release Node

Free a node that is no longer being used.

$ echo pm_release_node <node> > (sysfs node)

Set Requirement

Set the power requirement on the node.

$ echo pm_set_requirement <node> <capabilities> > (sysfs node)

Set Max Latency

Set the maximum wake-up latency requirement for a node.

$ echo pm_set_max_latency <node> <latency> > (sysfs node)

Get Node Status

Get status information of a node. (Any PU can check the status of any node, regardless of the node assignment.)

$ echo pm_get_node_status <node> > (sysfs node)

Returns (printed to screen):

  • Status: Node power status (see below.)

  • Usage: The master(s) using this node (bit-map):

    • Bit 0: Used by this master

    • Bit 1: Used by other master(s)

  • Requirement: Capabilities required (bit-map):

    • Bit 0: Accessible.

    • Bit 1: Retain context.

    • Bit 2: Wake-up source.

Status

Node Type

0

1

2

3

Memory

OFF

RETENTION

ON

 

CPU PP

OFF

ON

 

 

Processor

FORCED OFF

ACTIVE

SLEEP

SUSPENDING