Zynq UltraScale+ MPSoC Power Management - Linux Kernel

Table of Contents

Prerequisites

  1. You have installed Petalinux 2018.1 (or later).
  2. You have created a Petalinux project using the BSP for your board.
  3. You were able to boot the pre-built images on your board.
  4. You have a stand-alone application running on the RPU that supports power management using the XilPM library. If not, see Minimal RPU Applications on how to run a minimal application on the RPU.
  5. The overviw of EEMI APIs described in https://www.xilinx.com/support/documentation/user_guides/ug1199-zynq-power-management.pdf

Out-of-box PM Demo

A script has been included in the pre-built PetaLinux images that demonstrate the power management features available to the Linux console. To run the demo script:
  • Boot the PetaLinux pre-built image on the board.
  • Log in on the Linux console, type: "hellopm"

Runtime PM

See:https://www.kernel.org/doc/Documentation/power/runtime_pm.txt

Quoting from https://www.kernel.org/doc/html/v4.12/driver-api/pm/devices.html:

  • "Devices may be put into low-power states while the system is running, independently of other power management activity in principle. However, devices are not generally independent of each other (for example, a parent device cannot be suspended unless all of its child devices have been suspended)."

See documentation on the Linux drivers for any specific runtime PM handling: Linux Drivers

CPU PM

CPU Hotplug

Kernel documentation: https://www.kernel.org/doc/Documentation/power/suspend-and-cpuhotplug.txt

The required kernel configuration options are:
  • Kernel Features
    • [*] Support for hot-pluggable CPUs
  • Power management options
    • [*] Suspend to RAM and standby

The user may take one or more APU cores on-line and off-line as needed via the CPU Hotplug control interface.

For example, to take CPU3 off-line:
$ echo 0 > /sys/devices/system/cpu/cpu3/online

CPU Idle

CPU cores are powered off when they are idling.

Kernel documentation:


The required kernel configuration options are:
  • CPU Power Management
    • CPU Idle
      • [*] CPU idle PM support
      • [*] Ladder governor (for periodic timer tick)
      • ARM CPU Idle Drivers
        • [*] Generic ARM/ARM64 CPU idle Driver

Below is the sysfs interface for cpuidle.
$ ls -lR /sys/devices/system/cpu/cpu0/cpuidle/
/sys/devices/system/cpu/cpu0/cpuidle/:
drwxr-xr-x    2 root     root             0 Jun 10 21:55 state0
drwxr-xr-x    2 root     root             0 Jun 10 21:55 state1
 
/sys/devices/system/cpu/cpu0/cpuidle/state0:
-r--r--r--    1 root     root          4096 Jun 10 21:55 desc
-rw-r--r--    1 root     root          4096 Jun 10 21:55 disable
-r--r--r--    1 root     root          4096 Jun 10 21:55 latency
-r--r--r--    1 root     root          4096 Jun 10 21:55 name
-r--r--r--    1 root     root          4096 Jun 10 21:55 power
-r--r--r--    1 root     root          4096 Jun 10 21:55 residency
-r--r--r--    1 root     root          4096 Jun 10 21:55 time
-r--r--r--    1 root     root          4096 Jun 10 21:55 usage
 
/sys/devices/system/cpu/cpu0/cpuidle/state1:
-r--r--r--    1 root     root          4096 Jun 10 21:55 desc
-rw-r--r--    1 root     root          4096 Jun 10 21:55 disable
-r--r--r--    1 root     root          4096 Jun 10 21:55 latency
-r--r--r--    1 root     root          4096 Jun 10 21:55 name
-r--r--r--    1 root     root          4096 Jun 10 21:55 power
-r--r--r--    1 root     root          4096 Jun 10 21:55 residency
-r--r--r--    1 root     root          4096 Jun 10 21:55 time
-r--r--r--    1 root     root          4096 Jun 10 21:55 usage
where,
  • desc : Small description about the idle state (string)
  • disable : Option to disable this idle state (bool) -> see note below
  • latency : Latency to exit out of this idle state (in microseconds)
  • name : Name of the idle state (string)
  • power : Power consumed while in this idle state (in milliwatts)
  • time : Total time spent in this idle state (in microseconds)
  • usage : Number of times this state was entered (count)

Below is the sysfs interface for cpuidle governors.
$ ls -lR /sys/devices/system/cpu/cpuidle/
/sys/devices/system/cpu/cpuidle/:
-r--r--r--    1 root     root          4096 Jun 10 21:55 current_driver
-r--r--r--    1 root     root          4096 Jun 10 21:55 current_governor_ro

CPU Frequency

Adjust CPU frequency at runtime.

Kernel documentation:


The required kernel configuration options are:
  • CPU Power Management
    • CPU Frequency scaling
      • [*] CPU Frequency scaling
        • Default CPUFreq governor
          • Userspace
        • <*> Generic DT based cpufreq driver

Read current CPU frequency (same for all cores):
$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq

Look up the available governors speeds (same for all cores):
$ cat /sys/devices/system/cpu/cpu1/cpufreq/scaling_available_governors

Select the 'userspace' governor for CPU frequency control (same for all cores):
$ echo userspace > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

Look up the available CPU speeds (same for all cores):
$ cat /sys/devices/system/cpu/cpu1/cpufreq/scaling_available_frequencies
 

Change the CPU speed (same for all cores):
$ echo <frequency> > /sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed

Zynq UltraScale+ MPSoC Power Management - Linux Kernel

This page provides tips and examples of Linux kernel power management solutions for the Zynq UltraScale+ MPSoC.

System PM

Suspend

The kernel is suspended when the CPU and most of the peripherals are powered down. The system run states needed to resume from suspend is stored in the DRAM, which is put into self-refresh mode.

The required kernel configuration option settings are:
  • Power management options
    • [*] Suspend to RAM and standby
    • [*] User space wakeup sources interface
    • [*] Device power management core functionality

  • Device Drivers
    • SOC (System On Chip) specific Drivers
        • XIlinx SoC drivers
          • Zynq MPSoC SoC
            • [*] Enable Xilinx Zynq MPSoC Power Management Driver
            • [*] Enable Zynq MPSoC generic PM domains
  • Firmware Drivers
    • Zynq MPSoC Firmware Drivers
      • -*- Enable Xilinx Zynq MPSoC firmware interface


Type the following command to suspend the kernel. (Note: you must enable a wake-up source first otherwise the kernel will never resume.)
$ echo mem > /sys/power/state

Wake-up Source

The kernel resumes from the suspend mode when a wake-up event occurs.

UART

To wake up the APU on UART input:
  • For ZynMP
$ echo enabled > /sys/devices/platform/amba/ff000000.serial/tty/ttyPS0/power/wakeup

  • For Versal
$ echo enabled > /sys/devices/platform/amba/ff000000.serial/tty/ttyAMA0/power/wakeup

USB

To wake up the APU on USB input:
$ echo 0 > /sys/module/printk/parameters/console_suspend (optional)
$ echo enabled > /sys/devices/platform/amba/ff000000.serial/tty/ttyPS0/power/wakeup  (optional, this is UART wakeup source)
$ echo enabled > /sys/bus/usb/devices/usb1/power/wakeup (set USB as wakeup source)
 
/* Based on the device connected the below needs to be done */
$ echo enabled  > /sys/bus/usb/devices/usb1/1-1/power/wakeup
 
Suspend Linux:
$ echo mem > /sys/power/state
 
Note: To Wake up using USB, Generate wakeup signal from the connected USB device.

Wake on LAN

To wake up the APU on Ethernet input:
  • For ZynqMP


// without FPD off
/ Select GEM as wake device
$ echo enabled > /sys/devices/platform/amba/ff0e0000.ethernet/net/eth0/power/wakeup
/ Suspend
$ echo mem > /sys/power/state
 
//for FPD off
$ echo pm_request_wakeup 8 1 0 1 > /sys/kernel/debug/zynqmp-firmware/pm
$ echo pm_force_powerdown 6 > /sys/kernel/debug/zynqmp-firmware/pm
/ Select GEM as wake device
$ echo enabled > /sys/devices/platform/amba/ff0e0000.ethernet/net/eth0/power/wakeup
// Suspend
$ echo mem > /sys/power/state
 
  • For Versal
/ Select GEM as wake device
$ echo enabled > /sys/devices/platform/amba/ff0c0000.ethernet/net/eth0/power/wakeup
$ echo mem > /sys/power/state

Note: To Wake up using Ethernet, Please ping the specific interface's IP address.

GPIO

The required kernel configuration options are:

  • Device Drivers
    • [*] GPIO Support
      • [*] /sys/class/gpio/... (sysfs interface)
      • Memory mapped GPIO drivers
        • [*] Xilinx GPIO support
        • [*] Xilinx Zynq GPIO support


To wake up the APU on the GPIO pin:

$ echo enabled > /sys/devices/platform/gpio-keys/power/wakeup

RTC

The required kernel configuration options are:

  • Device Drivers
    • Real Time Clock
      • [*] Xilinx Zynq Ultrascale+ MPSoC RTC


To wake up the RTC to wake up the APU after 10 seconds:

$ echo +10 > /sys/class/rtc/rtc0/wakealarm

Global Storage Registers

Global General Storage Registers and Persistent Global General Storage Registers available for general use can be accessed through firmware sysfs files.

Path to device file (firmware-node) for 2018.1 and above release is:

/sys/firmware/zynqmp/

Path to device file (firmware-node) for 2017.4 and earlier release is:

/sys/devices/platform/firmware/

Global General Storage Registers

Four 32-bit storage registers are available for general use. Their values are not preserved across after software reboots.

For ZynqMP:

Device fileMMIO RegisterMMIO AccessValid Value Range
<firmware-node>/ggs0GLOBAL_GEN_STORAGE00xFFD800300x00000000 - 0xFFFFFFFF
<firmware-node>/ggs1GLOBAL_GEN_STORAGE10xFFD800340x00000000 - 0xFFFFFFFF
<firmware-node>/ggs2GLOBAL_GEN_STORAGE20xFFD800380x00000000 - 0xFFFFFFFF
<firmware-node>/ggs3GLOBAL_GEN_STORAGE30xFFD8003C0x00000000 - 0xFFFFFFFF

For Versal:

Note: PLM uses PMC RAM for storing GGS values as Life of GGS registers and PMC RAM is same. So there is no fixed hardware register is involved.

Device fileMMIO RegisterMMIO AccessValid Value Range
<firmware-node>/ggs0NANA0x00000000 - 0xFFFFFFFF
<firmware-node>/ggs1NANA0x00000000 - 0xFFFFFFFF
<firmware-node>/ggs2NANA0x00000000 - 0xFFFFFFFF
<firmware-node>/ggs3NANA0x00000000 - 0xFFFFFFFF


Read the value of a global storage register:

$ cat /sys/firmware/zynqmp/ggs0

Write the mask and value of a global storage register:

$ echo 0xFFFFFFFF 0x1234ABCD > /sys/firmware/zynqmp/ggs0

Persistent Global General Storage Registers

Four 32-bit persistent global storage registers are available for general use. Their values are preserved across after software reboots.

For ZynqMP

Device NameMMIO RegisterMMIO AccessValid Value Range
<firmware-node>/pggs0PERS_GLOB_GEN_STORAGE00xFFD800500x00000000 - 0xFFFFFFFF
<firmware-node>/pggs1PERS_GLOB_GEN_STORAGE10xFFD800540x00000000 - 0xFFFFFFFF
<firmware-node>/pggs2PERS_GLOB_GEN_STORAGE20xFFD800580x00000000 - 0xFFFFFFFF
<firmware-node>/pggs3PERS_GLOB_GEN_STORAGE30xFFD8005C0x00000000 - 0xFFFFFFFF

For Versal

Device NameMMIO RegisterMMIO AccessValid Value Range
<firmware-node>/pggs0PMC_PERS_GLOB_GEN_STORAGE30xF111005C0x00000000 - 0xFFFFFFFF
<firmware-node>/pggs1PMC_PERS_GLOB_GEN_STORAGE40xF11100600x00000000 - 0xFFFFFFFF
<firmware-node>/pggs2PSM_PERS_GLOB_GEN_STORAGE00xFFC900500x00000000 - 0xFFFFFFFF
<firmware-node>/pggs3PSM_PERS_GLOB_GEN_STORAGE10xFFC900540x00000000 - 0xFFFFFFFF

Read the value of a persistent global storage register:

$ cat /sys/firmware/zynqmp/pggs0

Write the mask and value of a persistent global storage register:

$ echo 0xFFFFFFFF 0x1234ABCD > /sys/firmware/zynqmp/pggs0

Debugfs

Kernel documentation:


The debugfs interface is intended for testing and debugging the integration between the Linux kernel and the Zynq UltraScale+ MPSoC power management framework. This interface must be used with a lot of care. In fact, accessing this interface during normal PM operation will very likely cause unexpected problems. Please refer to UG1137 for the usage of PM API.

Starting with 2018.1, this interface is disabled by default. To enable this interface, change the following kernel configurations (in this order):

  • Power management options
    • [*] Suspend to RAM and standby
    • [*] User space wakeup sources interface
    • [*] Device power management core functionality
  • Kernel hacking
    • Compile-time checks and compiler options
      • [*] Debug Filesystem
  • Firmware Drivers
    • Zynq MPSoC Firmware Drivers
      • -*- Enable Xilinx Zynq MPSoC firmware interface
      • [*] Enable Xilinx Zynq MPSoC firmware debug APIs



Access to the debugfs interface is provided through the following node in the sysfs:

For 2018.1 and later releases:

/sys/kernel/debug/zynqmp-firmware/pm

Note: For 2018.1 and later releases, output of debug commands (if any) would not be printed on console in dmesg. it needs to be explicitly read by reading above "pm" debugfs file as below:

cat /sys/kernel/debug/zynqmp-firmware/pm


For 2017.4 and earlier releases:

/sys/kernel/debug/zynqmp_pm/power

For example,

$ echo pm_request_node 22 1 100 1 > (sysfs node)

See UG1137 for the input parameters to the APIs for ZynqMP Platform. For Versal refer UG1304.


Note: For 2018.1 and later releases, API names are updated with "pm_" prefix.

Get API Version

Get the API version.
$ echo pm_get_api_version > (sysfs node)

Request Suspend

Request another PU to suspend itself.
$ echo pm_request_suspend <node> > (sysfs node)

Self Suspend

Notify PMU that this PU is about to suspend itself.
$ echo pm_self_suspend <node> > (sysfs node)

Force Power Down

Force another PU to power down.
$ echo pm_force_powerdown <node> > (sysfs node)

Abort Suspend

Notify PMU that the attempt to suspend has been aborted.
$ echo pm_abort_suspend > (sysfs node)

Request Wake-up

Request another PU to wake up from suspend state.
$ echo pm_request_wakeup <node> <set_address> <address> > (sysfs node)

Set Wake-up Source

Set up a node as the wake-up source.
$ echo pm_set_wakeup_source <target> <wkup_node> <enable> > (sysfs node)

System Shutdown

Shut down the PU's own sub-system.
$ echo pm_system_shutdown <action> <scope> > (sysfs node)
Action:
  • 0: Shutdown
  • 1: Reset
  • 2: Set reset scope
Scope:
  • 0: APU
  • 1: PS
  • 2: System

Request Node

Request to use a node.
$ echo pm_request_node <node> > (sysfs node)

Release Node

Free a node that is no longer being used.
$ echo pm_release_node <node> > (sysfs node)

Set Requirement

Set the power requirement on the node.
$ echo pm_set_requirement <node> <capabilities> > (sysfs node)

Set Max Latency

Set the maximum wake-up latency requirement for a node.
$ echo pm_set_max_latency <node> <latency> > (sysfs node)

Get Node Status

Get status information of a node. (Any PU can check the status of any node, regardless of the node assignment.)
$ echo pm_get_node_status <node> > (sysfs node)
Returns (printed to screen):
  • Status: Node power status (see below.)
  • Usage: The master(s) using this node (bit-map):
    • Bit 0: Used by this master
    • Bit 1: Used by other master(s)
  • Requirement: Capabilities required (bit-map):
    • Bit 0: Accessible.
    • Bit 1: Retain context.
    • Bit 2: Wake-up source.
Status
Node Type0123
MemoryOFFRETENTIONON
CPU PPOFFON

ProcessorFORCED OFFACTIVESLEEPSUSPENDING
USBNOT USEDOFF (Could be a wake-up source)ON

Other NodesOFFON

Get Operating Characteristic

Get operating characteristic information of a node.
$ echo pm_get_operating_characteristic <node> > (sysfs node)

Reset Assert

To perform assert, release or pulse reset on specific reset line.

$ echo pm_reset_assert <reset> <action> > (sysfs node)

Reset Get Status

To get the status of specific reset line.

$ echo pm_reset_get_status <reset> > (sysfs node)

MMIO Read

To read a value from an address that is not directly accessible from Linux/APU.

$ echo mmio_read <address> > (sysfs node)
Address is in hex format (e.g. 0xFFFF0000). Not all addresses are accessible by all masters.
Note: MMIO read has been removed since 2018.1 release.

MMIO Write

To write a value to a specific address that is not directly accessible from Linux/APU.

$ echo mmio_write <mask> <address> <value> > (sysfs node)
Mask, address and value are in hex format (e.g. 0xFFFF0000). Not all addresses are accessible by all masters.
Note: MMIO write has been removed since 2018.1 release.

Get Chip ID

Get the chip ID.
$ echo pm_get_chipid > (sysfs node)

Get pin control functions

Get current selected function for given pin.
$ echo pm_pinctrl_get_function <pin-number> > (sysfs node)

Set pin control functions

Set requested function for given pin.
$ echo pm_pinctrl_set_function <pin-number> <function-id> > (sysfs node)

Get configuration parameters for the pin

Get value of requested configuration parameter for given pin.
$ echo pm_pinctrl_config_param_get <pin-number> <parameter to get> > (sysfs node)

Set configuration parameters for the pin

Set value of requested configuration parameter for given pin.
$ echo pm_pinctrl_config_param_set <pin-number> <parameter to set> <param value> > (sysfs node)

Control device and configurations

Control device and configurations and get configurations values.
$ echo pm_ioctl <node id> <ioctl id> <arg1> <arg2> > (sysfs node)

Query Data

Request data from firmware.
$ echo pm_query_data <query id> <arg1> <arg2> <arg3> > (sysfs node)

Enable Clock

Enable the clock for given clock node id.
$ echo pm_clock_enable <clock id> > (sysfs node)

Disable Clock

Disable the clock for given clock node id.
$ echo pm_clock_disable <clock id> > (sysfs node)

Get Clock State

Get the state of clock for given clock node id.
$ echo pm_clock_getstate <clock id> > (sysfs node)
$ cat(sysfs node)

Set Clock Divider

Set the divider value of clock for given clock node id.
$ echo pm_clock_setdivider <clock id> <divider value> > (sysfs node)

Get Clock Divider

Get the divider value of clock for given clock node id.
$ echo pm_clock_getdivider <clock id> > (sysfs node)

Set Clock Rate

Set the rate of clock for given clock node id.
$ echo pm_clock_setrate <clock id> <clock rate> > (sysfs node)
Note: For 2018.1 release, Set Clock Rate API is not implemented and has no effect.

Get Clock Rate

Get the rate of clock for given clock node id.
$ echo pm_clock_getrate <clock id> > (sysfs node)
Note: For 2018.1 release, Get Clock Rate API is not implemented and has no effect.

Set Clock Parent

Set the parent of clock for given clock node id.
$ echo pm_clock_setparent <clock id> <parent clock id> > (sysfs node)

Get Clock Parent

Get the parent of clock for given clock node id.
$ echo pm_clock_getparent <clock id> > (sysfs node)

CSU/PMU Register Access 

Read/Write the CSU and PMU global registers.

Register Write :

$ echo  <address> <mask> <value> > /sys/firmware/zynqmp/config_reg

Register Read :

$ echo <address> > /sys/firmware/zynqmp/config_reg
$ cat /sys/firmware/zynqmp/config_reg

CSU and PMU global registers are classified to two lists. White list (accessed all the time by default), Black list (accessed when a compile time flag is set)

we have a #define option (SECURE_ACCESS_VAL) that provides access to black list. To Access black list registers build the PMUFW with  SECURE_ACCESS_VAL flag set.

White List Registers :

CSU Module :

  • Csu_status
  • Csu_multi_boot
  • Csu_tamper_trig
  • Csu_ft_status
  • Jtag_chain_status
  • Idcode
  • Version
  • Csu_rom_digest(0:11)
  • Aes_status
  • Pcap_status

PMU Global Module :

  • Global_control
  • Global_Gen_Storage0 - 6
  • Pers_Glob_Gen_Storage0-6
  • Req_Iso_Status
  • Req_SwRst_Status
  • Csu_Br_Error
  • Safety_Chk

Black List Registers :

  • Every other register in CSU Module, that is not covered in the above white list will be a black register.
  • Every other register in PMU_GLOBAL Module,  that is not covered in the above white list will be a black register.
  • RSA and RSA_CORE module registers are black.

Related Links