Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The purpose of this page is to describe how the ARM MALI driver is integrated into Xilinx

Table of Contents

Table of Contents

Overview

Xilinx ZynqMPSoC has the MALI 400MP GPU from ARM. The ARM MALI 400MP is an OpenGLES 2.0 capable GPU.

Driver access and license

The driver for MALI 400MP consists of Linux kernel driver and user library. The user space library is proprietary licensed and will have to be distributed as binaries. The user space library will be provided through Xilinx PetaLinux release. The Linux kernel driver is GPL licensed, and downloadable from http://malideveloper.arm.com/. Until 2016.4, the kernel driver was hosted on Xilinx github. From 2017.1, the kernel driver hosted on Xilinx github is deprecated. This will now be downloaded from ARM website and packaged into the PetaLinux BSP.

Note: From 2019.2 onwards Xilinx will not host the user-space binaries in the lounge. Customers can download from https://github.com/Xilinx/mali-userspace-binaries with appropriate branch.

Release version
The driver is released periodically by ARM. Thus there can be multiple versions available. The default version in Kconfig points to the latest working release.

Device tree binding

The DT binding documentation is included in the driver release. Download the kernel driver tarball from http://malideveloper.arm.com/., unzip it and the DT binding documentation is available at the folder path.
"driver/documentation/devicetree/bindings/arm/mali-utgard.txt"

Runtime power management

The MALI driver supports fine grained runtime power management based on Linux runtime PM APIs, with its own scheduler. This section describes the flow of specific driver version, r7p0-00rel0, to give an overview.
First the driver implements the Linux runtime PM callbacks (in mali_kernel_linux.c), and the runtime pm is enabled in device initialization (arm.c)
Code Block
themeMidnight
static const struct dev_pm_ops mali_dev_pm_ops = { #ifdef CONFIG_PM_RUNTIME .runtime_suspend = mali_driver_runtime_suspend, .runtime_resume = mali_driver_runtime_resume,

The purpose of this page is to describe how the ARM MALI driver is integrated into Zynq™ UltraScale+™ MPSoC


Table of Contents

Table of Contents

Overview

Zynq™ UltraScale+™ MPSoC has the MALI 400MP GPU from ARM. The ARM MALI 400MP is an OpenGLES 2.0 capable GPU.

Driver access and license

The driver for MALI 400MP consists of Linux kernel driver and user library. The user space library is proprietary licensed and will have to be distributed as binaries. The user space library will be provided through AMD's PetaLinux release. The Linux kernel driver is GPL licensed, and downloadable from http://malideveloper.arm.com/. Until 2016.4, the kernel driver was hosted on github. From 2017.1, the kernel driver hosted on github is deprecated. This will now be downloaded from ARM website and packaged into the PetaLinux BSP.

Note: From 2019.2 onwards the user-space binaries in the lounge are not hosted. Customers can download from https://github.com/Xilinx/mali-userspace-binaries with appropriate branch.


Release version
The driver is released periodically by ARM. Thus there can be multiple versions available. The default version in Kconfig points to the latest working release.

Device tree binding

The DT binding documentation is included in the driver release. Download the kernel driver tarball from http://malideveloper.arm.com/., unzip it and the DT binding documentation is available at the folder path.
"driver/documentation/devicetree/bindings/arm/mali-utgard.txt"

Runtime power management

The MALI driver supports fine grained runtime power management based on Linux runtime PM APIs, with its own scheduler. This section describes the flow of specific driver version, r7p0-00rel0, to give an overview.

First the driver implements the Linux runtime PM callbacks (in mali_kernel_linux.c), and the runtime pm is enabled in device initialization (arm.c)
Code Block
themeMidnight
static const struct dev_pm_ops mali_dev_pm_ops = {
#ifdef CONFIG_PM_RUNTIME
        .runtime_suspend = mali_driver_runtime_suspend,
        .runtime_resume = mali_driver_runtime_resume,
        .runtime_idle = mali_driver_runtime_idle,
#endif
        .suspend = mali_driver_suspend_scheduler,
        .resume = mali_driver_resume_scheduler,
        .freeze = mali_driver_suspend_scheduler,
        .thaw = mali_driver_resume_scheduler,
        .poweroff = mali_driver_suspend_scheduler,
};
#endif


Code Block
themeMidnight
mali_platform_device_register()
{
        ...
        pm_runtime_set_autosuspend_delay(&&(mali_gpu_device.dev), 1000);
        pm_runtime_use_autosuspend(&&(mali_gpu_device.dev));
#endif
        pm_runtime_enable(&&(mali_gpu_device.dev));
        ...
}


Then, the driver has its own scheduler (mali_scheduler.c) that tracks any activities of all GPU processors (GP: geometry processor, PP: pixel processor). All activities on those processors are created as a job (ex, gp job / pp job) and scheduled through this scheduler. The scheduler tracks the completion of the job as well. Based on the status, the scheduler sets the runtime pm reference count accordingly (mali_scheduler.c). Below is an example for GP. Equivalent functions exist for PP.
Code Block
themeMidnight
mali_scheduler_queue_gp_job()
{
        ...
        _mali_osk_pm_dev_ref_get_async()
        .runtime_idle =..
}
 
mali_scheduler_drivercomplete_runtime_idle,
#endifgp_job()
{
        ...suspend
= mali_driver_suspend_scheduler,         .resume = mali_driver_resume_scheduler,_mali_osk_pm_dev_ref_pet_async()
        ..freeze = .
}

When the reference count reaches to 0, the runtime_suspend callback will be triggered: runtime_suspend callback -> mali_driver_runtime_suspend_scheduler, .thaw = mali_driver_resume_scheduler, .poweroff = mali_driver_suspend_scheduler, }; #endif() -> mali_pm_runtime_suspend() -> mali_pm_common_suspend(). mali_pm_common_suspend() performs a series of operations to put all relevant modules, ex, l2 cache and mmu, in idle state. Reverse operations is performed when resuming.
Code Block
themeMidnight
mali_platformpm_devicecommon_registersuspend()
{
        ...
         pm_runtime_set_autosuspend_delay(&&(mali_gpu_device.dev), 1000);if (0 < num_groups_down) {
                pmmali_executor_runtimegroup_usepower_autosuspend(&&(mali_gpu_device.dev)down(groups_down, num_groups_down);
#endif          pm_runtime_enable(&&(mali_gpu_device.dev));      }
 
...
}
Then, the driver has its own scheduler (mali_scheduler.c) that tracks any activities of all GPU processors (GP: geometry processor, PP: pixel processor). All activities on those processors are created as a job (ex, gp job / pp job) and scheduled through this scheduler. The scheduler tracks the completion of the job as well. Based on the status, the scheduler sets the runtime pm reference count accordingly (mali_scheduler.c). Below is an example for GP. Equivalent functions exist for PP.
Code Block
themeMidnight
mali_scheduler_queue_gp_job()
{
        ...
        _mali_osk_pm_dev_ref_get_async()
        ...
}
 
mali_scheduler_complete_gp_job()
{
        ...
        _mali_osk_pm_dev_ref_pet_async()
        ...
}
When the reference count reaches to 0, the runtime_suspend callback will be triggered: runtime_suspend callback -> mali_driver_runtime_suspend() -> mali_pm_runtime_suspend() -> mali_pm_common_suspend(). mali_pm_common_suspend() performs a series of operations to put all relevant modules, ex, l2 cache and mmu, in idle state. Reverse operations is performed when resuming.
Code Block
themeMidnight
mali_pm_common_suspend()
{
        ...
        if (0 < num_groups_down) {
                mali_executor_group_power_down(groups_down, num_groups_down);
                }
 
        for (i = 0; i < num_l2_down; i++) {
                mali_l2_cache_power_down(l2_down[i]);
        }
 
        ...
}
The driver level handling eventually triggers the platform level power domain management. Underlying runtime pm and genpd implementation triggers the firmware APIs at the end. It's not scope of this documentation.
Changelog

Selecting particular backends:

Wayland/GBM backend:

2019.1:

From 2019.1 release, selecting Mali backend is decoupled from 'DISTRO_FEATURES', another variable is introduced 'MALI_BACKEND_DEFAULT' to create proper link. By default, plnx build system will try to package all the backends in the rootfs and depending upon the value of 'MALI_BACKEND_DEFAULT' we create a link to the correct backend. Fbdev, X11, wayland and headless are the choices we have, assigning incorrect name would yield a link to headless.

For example: Once you have selected libmali through 'petalinux-config -c rootfs', add following parameters to <plnx-proj-root>/project-spec/meta-user/conf/petalinuxbsp.conf to select wayland. 

Code Block
themeMidnight
MALI_BACKEND_DEFAULT = "wayland"

Users can still modify DISTRO_FEATURES, in order to reduce the size of rootfs, but remember selecting Mali backend wont be impacted.

2018.3:

From 2018.3 release, Mali will support wayland/GBM backend in addition to fbdev and X11.

For example: Once you have selected libmali through 'petalinux-config -c rootfs' unselect 'packagegroup-petalinux-matchbox' and 'packagegroup-petalinux-x11' and add following parameters to <plnx-proj-root>/project-spec/meta-user/conf/petalinuxbsp.conf 

Code Block
themeMidnight
DISTRO_FEATURES_append = " wayland"
IMAGE_INSTALL_append = " packagegroup-petalinux-weston"

This packagegroup ensures all the essential wayland/weston packages are packaged into the rootfs for having a wayland/weston application work out of the box. On boot, export following parameter in your terminal console.

Code Block
themeMidnight
export XDG_RUNTIME_DIR=/run/user/0/

Now, you can run sample benchmarking application glmark2-es2-wayland and glmark2-es2-drm.

X11 backend:

By default, Mali supports X11 backend. Just select libmali-xlnx package from 'petalinux-config -c rootfs'. The root filesystem should now have libmali with X11 support. Also, please select at least one window manager.

For example: packagegroup-petalinux-matchbox.

Fbdev backend:

2019.1:

From 2019.1 release,to select fbdev, Just select libmali-xlnx package from 'petalinux-config -c rootfs' and add the following lines to <plnx-proj-root>/project-spec/meta-user/conf/petalinuxbsp.conf

Code Block
themeMidnight
MALI_BACKEND_DEFAULT = "fbdev"

2018.3:

Up to 2018.3, in order to have Mali fbdev backend, please add following lines to <plnx-proj-root>/project-spec/meta-user/conf/petalinuxbsp.conf

Code Block
themeMidnight
DISTRO_FEATURES_remove = "x11"

Headless-EGL backend:

2019.1:

From 2019.1 onwards, please add following lines to <plnx-proj-root>/project-spec/meta-user/conf/petalinuxbsp.conf.

Code Block
themeMidnight
MALI_BACKEND_DEFAULT = "headless"

2018.3:

This backend is available from 2018.3 release, please add following lines to <plnx-proj-root>/project-spec/meta-user/conf/petalinuxbsp.conf. 

Code Block
themeMidnight
DISTRO_FEATURES_remove = "x11 fbdev"

Benchmark:

To run a benchmark example for any backend (except headless-egl) please add below lines to build/conf/local.conf

Code Block
themeMidnight
IMAGE_INSTALL_append = " glmark2"

QT wayland/KMS/GBM plugin:

In addition to enable Wayland Mali backend support above, select 'packagegroup-petalinux-qt' in 'petalinux-config -c rootfs' and perform a petalinux-build.

Wayland plugin:

Once weston shows up, export the following environment variables for wayland plugins. 

Code Block
themeMidnight
export XDG_RUNTIME_DIR=/run/user/0/ && export QT_QPA_PLATFORM=wayland && export QT_WAYLAND_SHELL_INTEGRATION=wl-shell

Now you can run any example applications like  /usr/share/qt5/examples/opengl/textures/textures

KMS/GBM plugin:

For this plugin ensure that weston is not running and export the following environment variables

Code Block
themeMidnight
export QT_QPA_PLATFORM=eglfs && EGLFS_DEVICE_INTEGRATION=eglfs_kms

Now you can run any example applications like /usr/share/qt5/examples/opengl/textures/textures.

Update-alternatives:

2019.1:

If DISTRO_FEATURES are not modified you should have all backends packaged in the rootfs. Now, you can switch between muliple backends using update-alternatives commands as follows:

To Update a link:

Code Block
themeMidnight
update-alternatives --install /usr/lib/libMali.so.8.0 libmali /usr/lib/fbdev/libMali.so.8.0 90

To remove a link:

Code Block
themeMidnight
update-alternatives --remove libmali /usr/lib/fbdev/libMali.so.8.0

For more info, perform update-alternatives --help

Alternate solution: Users can always modify links using "ln" commands

Related Links
        for (i = 0; i < num_l2_down; i++) {
                mali_l2_cache_power_down(l2_down[i]);
        }
 
        ...
}


The driver level handling eventually triggers the platform level power domain management. Underlying runtime pm and genpd implementation triggers the firmware APIs at the end. It's not scope of this documentation.

GPU software stack


Image Added

Changelog
  • 2023.1
  • 2021.1
    • Update EGL headers
    • Fix for compatibility with 5.10 Linux kernel


Selecting particular backends:

You can select particular backend while configuring rootfs

File: project-spec/config/rootfs_config

Code Block
themeMidnight
petalinux-config -c rootfs
  1. Wayland/GBM backend:

    By default, plnx build system will try to package all the backends in the rootfs and depending upon the rootfs config, we create a link to the correct backend. Fbdev, X11, wayland and headless are the choices we have.

    For example: Once you have selected libmali through 'petalinux-config -c rootfs', select backend to wayland and unselect 'packagegroup-petalinux-matchbox' and 'packagegroup-petalinux-x11' and select 'packagegroup-petalinux-weston'. After selection your rootfs_config will look as below. 

    Code Block
    themeMidnight
    CONFIG_libmali-xlnx=y
    CONFIG_mali-backend-wayland=y

    This packagegroup ensures all the essential wayland/weston packages are packaged into the rootfs for having a wayland/weston application work out of the box. On boot, export following parameter in your terminal console.


    Code Block
    themeMidnight
    export XDG_RUNTIME_DIR=/run/


    Now, you can run sample benchmarking application glmark2-es2-wayland.

  2. X11 backend:

    By default, Mali supports X11 backend. Just select libmali-xlnx package from 'petalinux-config -c rootfs'. The root filesystem should now have libmali with X11 support. Also, please select at least one window manager. For example: packagegroup-petalinux-matchbox.

    Once you have selected libmali through 'petalinux-config -c rootfs', and selected backend to x11, your rootfs_config will look as below.


    Code Block
    themeMidnight
    CONFIG_libmali-xlnx=y
    CONFIG_mali-backend-x11=y


  3. Fbdev backend:

    Just select libmali-xlnx package from 'petalinux-config -c rootfs' and select fbdev backend.

    Once you have selected libmali through 'petalinux-config -c rootfs' unselect 'packagegroup-petalinux-matchbox' and 'packagegroup-petalinux-x11'. Your rootfs_config will look as below.


    Code Block
    themeMidnight
    CONFIG_libmali-xlnx=y
    CONFIG_mali-backend-fbdev=y


  4. Headless-EGL backend:

    Just select libmali-xlnx package from 'petalinux-config -c rootfs' and select headless backend. Unselect 'packagegroup-petalinux-matchbox' and 'packagegroup-petalinux-x11'.

    Once you have selected libmali through 'petalinux-config -c rootfs', and selected backend to headless, your rootfs_config will look as below.

    Code Block
    themeMidnight
    CONFIG_libmali-xlnx=y
    CONFIG_mali-backend-headless=y

    Please find more details for Headless rendering on below page.

    Mali 400 Headless rendering


Benchmark:

To run a benchmark example for x11 and wayland backend, please add below lines to build/conf/local.conf


Code Block
themeMidnight
IMAGE_INSTALL:append = "glmark2"


Update-alternatives:

You should have all backends packaged in the rootfs and you can switch between multiple backends using update-alternatives commands as follows:

To Update a link:

Code Block
themeMidnight
update-alternatives --install /usr/lib/libMali.so.9.0 libmali /usr/lib/fbdev/libMali.so.9.0 90

To remove a link:

Code Block
themeMidnight
update-alternatives --remove libmali /usr/lib/libMali.so.9.0

For more info, perform update-alternatives --help

Alternate solution: Users can always modify links using "ln" commands

Running GPU applications:

Create project 


Code Block
themeMidnight
source settings.sh
petalinux-create -t project -s xilinx-zcu106-v2023.1-final.bsp


Setting Petalinux BSP configuration

File: project-spec/meta-user/conf/petalinuxbsp.conf

For generating rootfs for compiling application or any libraries add below configs


Code Block
themeMidnight
EXTRA_IMAGE_FEATURES = "debug-tweaks dev-pkgs"


Enabling and disabling rootfs configs


Code Block
themeMidnight
petalinux-config -c rootfs


For enabling qt application for any backend
CONFIG_packagegroup-petalinux-qt=y

  1. x11
    CONFIG_packagegroup-petalinux-matchbox=y
    CONFIG_packagegroup-petalinux-x11=y
  2. fbdev
    # CONFIG_packagegroup-petalinux-matchbox is not set
    # CONFIG_packagegroup-petalinux-x11 is not set
  3. wayland
    CONFIG_packagegroup-petalinux-weston=y
    # CONFIG_packagegroup-petalinux-matchbox is not set
    # CONFIG_packagegroup-petalinux-x11 is not set
  4. headless backend
    CONFIG_libmali-xlnx=y
    CONFIG_mali-backend-headless=y
    # CONFIG_packagegroup-petalinux-x11 is not set
    # CONFIG_packagegroup-petalinux-matchbox is not set


Building project


Code Block
themeMidnight
petalinux-build


Running applications

  1. GLMARK2 application
    1. x11 backend
      1. Run Xorg application in backend or start xserver service if not running

        Code Block
        themeMidnight
        Xorg -depth 16 &
        systemctl start xserver-nodm


      2. Run glmark application

        Code Block
        themeMidnight
        export DISPLAY=:0.0
        glmark2-es2


    2. wayland backend 
      1. Start weston if not running

        Code Block
        themeMidnight
        systemctl start weston 


      2. Run glmark application

        Code Block
        themeMidnight
        export XDG_RUNTIME_DIR=/run/
        glmark2-es2-wayland


  2. QT applications

    1. x11 backend (Xorg should be running or xserver-nodm service should be started)
      1. Run Xorg or xserver service

        Code Block
        themeMidnight
        /usr/bin/Xorg -depth 16 &    (if Xorg or xserver-nodm service is not running)
        or
        systemctl start xserver-nodm


      2. Set environment variables

        Code Block
        themeMidnight
        export DISPLAY=:0.0
        export QT_QPA_PLATFORM=eglfs
        export QT_QPA_EGLFS_WIDTH=1920
        export QT_QPA_EGLFS_HEIGHT=1080
        export QT_QPA_GENERIC_PLUGINS=evdevmouse,evdevkeyboard
        export QT_QPA_ENABLE_TERMINAL_KEYBOARD=1
        export QT_QPA_FONTDIR=/usr/share/fonts/truetype
        export QT_QPA_PLATFORM_PLUGIN_PATH=/usr/lib/qt5/plugins
        export QML2_IMPORT_PATH=/usr/lib/qt5/qml


      3. Run example (eg: analogclock)

        Code Block
        themeMidnight
        cd /usr/share/examples/gui/analogclock/
        ./analogclock



    2. fbdev backend (Disable any Xorg or weston if running)
      1. Set environment variables

        Code Block
        themeMidnight
        export DISPLAY=:0.0
        export QT_QPA_EGLFS_INTEGRATION=eglfs_mali
        export QT_QPA_PLATFORM=eglfs::fb=/dev/fb0
        export QT_QPA_EGLFS_FB=/dev/fb0
        export QT_QPA_EGLFS_WIDTH=1920
        export QT_QPA_EGLFS_HEIGHT=1080
        export QT_QPA_GENERIC_PLUGINS=evdevmouse,evdevkeyboard
        export QT_QPA_ENABLE_TERMINAL_KEYBOARD=1
        export QT_QPA_FONTDIR=/usr/share/fonts/truetype
        export QT_QPA_PLATFORM_PLUGIN_PATH=/usr/lib/qt5/plugins
        export QML2_IMPORT_PATH=/usr/lib/qt5/qml


      2. Run application (eg: dockwidget)

        Code Block
        themeMidnight
        cd /usr/share/examples/widgets/mainwindows/dockwidgets
        ./dockwidgets





    3. wayland backend (weston should be running)

      1. If weston is not running, enable weston service

        Code Block
        themeMidnight
        systemctl start weston


      2. Set environment variables

        Code Block
        themeMidnight
        export DISPLAY=:0.0
        export XDG_RUNTIME_DIR=/run/
        export QT_QPA_PLATFORM=wayland
        export QT_WAYLAND_SHELL_INTEGRATION=xdg-shell (for >2023.1 release)
        export QT_WAYLAND_SHELL_INTEGRATION=wl-shell (till 2022.2 and older releases)


      3. Run applications

        Code Block
        themeMidnight
        cd /usr/share/examples/widgets/widgets/sliders/
        ./sliders
         
        cd /usr/share/examples/gui/analogclock/
        ./analogclock
         
        cd /usr/share/examples/widgets/mainwindows/dockwidgets/
        ./dockwidgets 



Enabling LIMA for 2023.1

  1. Add next line to the project-spec/meta-user/conf/petalinuxbsp.conf: 
    • DISTRO_FEATURES_BACKFILL_CONSIDERED += "libmali"
  2. Applications ran:
    1. x11:
      • vivid_tex, glmark2-es2, glcts, QT applications
    2. fbdev:
      • eglfbdev, eglfbdev_gears, QT applications
    3. wayland:
      • glmark2-es2-wayland, glcts, QT applications
  3. Limitations:
    1. x11:
      • All x11 applications are failing with segmentation fault while running.
    2. fbdev:
      • Application compilation issues for eglfbdev and eglfbdev_gears
    3. wayland:
      • glcts application compilation issues