The real-time performance of Linux or of specific, developer-identified tasks that run on Linux can be improved through either a native-Linux approach where services from the existing Xilinx kernel can be used, where low latency and preemption enhancements are made to the Linux kernel, or through an Asymmetric Multi Processing (AMP) approach where a real-time operating system has the ability to pre-empt the whole Linux OS, or it has the ability to run independently on one of the two processor cores.

Readers of this wiki who seek improved real-time performance might also want to review our Multi OS (AMP and Hypervisor) page.

Table of Contents

Xilinx Real-Time Guidance

Each of the available approaches to improve the real-time performance of Linux also bring various trade-offs related to programming API, kernel porting, divergence from standard Linux source tree, modification of device drivers, and IO/throughput. Because of these issues:

There is no one-size-fits-all solution for improving Linux real-time response
Some of these approaches can be challenging from a general-support and kernel synchronization perspective

Though we don't actively support many approaches below, we do have Zynq customers who are using such approaches.

Approaches to Real-Time on Linux

The content below provides some insight and useful information related to various options for the enhancement of real-time performance on a Zynq-7000 AP SoC based system. To the best of our knowledge, each of the solutions described below are technically valid for ARM Cortex A9-based systems such as Zynq. We have not however, vetted each and every solution from a technical perspective.Some implementations may require a different degree of effort for implementation and support than other implementations.

Dr. Jeremy Brown and Brad Martin provide a nice comparison in their paper: "How fast is fast enough? Choosing between Xenomai and Linux for real-time applications" which compares the performance of unmodified Linux, preempt_rt patched Linux and Xenomai on a ARM Cortex platform running at 720 MHz.

Although focusing on Enea's specific solutions for real-time, Enea's white paperprovides rather good background and summary for each of the approaches to enhanced real-time with Linux.

Native Linux-API Based Solutions

These approaches improve the real-time performance of Linux by either utilizing features that are already built-into Linux, or by modifying the Linux kernel with patches that address low latency and pre-emptibility of the Linux kernel.

Xilinx partners who include Native-Linux Real-time capabilities in their Linux distributions include:

Denx
Enea
Mentor
MontaVista
SYSGO
Wind River

These solutions utilize the native Linux API for the creation of real-time tasks. Although each supports the native Linux programming API for applications, the level of modification to the Linux kernel, device drivers, or other systems software does vary across such solutions.

SMP Affinity (Open Source Solution)

This approach uses existing features and capabilities of the Xilinx Git kernel in order to improve system-level responsiveness.

Processor Affinity

The Slackware Documentation project provides some very good information on hardware processor affinity. Processor affinity can be set in two ways:

From within the application code, it can be defined by setting a CPU bit mask for each thread via the calls: sched_setaffinity() (use sched_getaffinity()to read the value), or pthread_setaffinity_np()(use pthread_attr_getaffinity_np() to read the value).
From the command line, if you have the package util-linux (or here) installed, you can use the command "taskset" which allows tasks to be assigned to a specific CPU according to their PID.

IRQ Affinity

IRQ Affinity complements Processor affinity by providing the ability to directly route an interrupt to a specific processor.
You'll find a helpful blog post here, and the kernel documentation at IRQ-affinity.txt

Preempt_RT (Open Source Solution)

The intent of the preempt_rt patch is to improve the latency and response time of Linux. It does this by touching hundreds of locations in the Linux kernel and thousands of lines of code. Unfortunately, regardless of the platform which runs preempt_rt, these changes often require modification to device drivers in order to be real-time "safe", and it can also come at the expense of overall system throughput. As a result of these less than ideal trade-offs, the Linux kernel maintainers have not yet adopted preempt_rt into the "vanilla" Linux kernel sources.

The OSADL provides a running test bench showing the performance of various platforms Xilinx and others, with respect to latency benchmarks. Refer to the OSADL Zed board latency page for further details.

To enable Preempt_RT support preempt_rt patches can be applied to Xilinx Linux. Note that this feature is in the experimental stage and not officially supported

Example documentation is provided only for educational purposes.

Steps to integrate RT support in Vanilla AMD-Xilinx linux kernel

git clone https://github.com/Xilinx/linux-xlnx.git
cd linux-xlnx
git remote add rt_origin https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git
git remote update
git checkout xilinx-v2023.1
git merge v6.1-rc7-rt5 
git checkout --ours drivers/net/ethernet/xilinx/xilinx_axienet_main.c
git add drivers/net/ethernet/xilinx/xilinx_axienet_main.c
git merge --continue (Commit and exit)

Run kernel menuconfig and select fully preemptible kernel (General Setup → Preemption Model → Fully Preemptible Kernel) and build the kernel. Please note that this option may not show up until you have run the defconfig for your target of choice - Eg. make xilinx_defconfig

When Linux boots up we should see SMP PREEMPT_RT in the kernel boot log and legacy/RT applications can be run on this kernel.

Preempt RT is also supported by a number of our commercial Linux partners including Wind River, ENEA, and SYSGO.
The best information can be found at the OSADL's page for preempt_rt support for Linux, with additional technical details available here: https://rt.wiki.kernel.org/index.php/Main_Page

Enea Light Weight Run Time - LWRT (Commercial Solution)

Enea LWRT is a commercial solution offering user-space implementations of scheduling, message passing, and resource management, and unlike functionality provided by the kernel, the user-space implementation can be used without the overhead and indeterminism caused by context switches to supervisor mode. Though LWRT uses a user-space implementation of Enea's OSE RTOS, LWRT exposes a Linux programming API to the developer for support of real-time tasks. As a result, applications can be migrated between Linux (standard) and Real-Time, easily.

Enea LWRT solution - source http://www.enea.com/software/resources/whitepapers/Whitepapers/OSE/Enabling-Linux-for-Real-Time-on-Embedded-Multicore-Devices/

Enea LWRT solution - source http://www.enea.com/software/resources/whitepapers/Whitepapers/OSE/Enabling-Linux-for-Real-Time-on-Embedded-Multicore-Devices/

This approach requires fewer kernel patches than does preempt_rt, and should provide low latencies consistent with the preempt_rt patch while simultaneously offering higher throughput for both the Linux and real-time sides.
For more information see Enea's "Enabling Linux for Rea-Time on Embedded Multicore Devices" and "Alternative Design Approaches Adding Real-Time Capabilities to Linux" white papers.

Asymmetric Multi Processing (AMP) Real-Time Solutions

Linux can be configured in what is essentially an AMP configuration where a separate RTOS can run dedicated on one of Zynq's processors, or it can pre-empt a running Linux (to share a single processor), or - in the case of TrustZone-based solutions, Linux can run in SMP mode, and the RTOS uses TrustZone functionality to dynamically redirect one of the two CPU cores for the execution of real-time tasks.

Xilinx partners who provide Dual-kernel (AMP) Real-time capabilities in their Linux distributions include:

Denx
Sierraware
SYSGO
Xilinx PetaLinux

Linux / Bare-metal AMP (Open Source Solution)

Xilinx provides XAPP1079 which includes both software and hardware designs needed to run Linux on one processor core of the Zynq-7000 AP SoC, and bare-metal applications on the second core.

Linux / Bare-Metal AMP

PetaLinux (Commercial/Open Source Solution)

Xilinx PetaLinux includes a board support package for the Xilinx ZC-702 Evaluation Kit which supports an AMP configuration consisting of Linux & FreeRTOS.

Linux / FreeRTOS AMP

Xenomai (Open Source Solution)

Xenomai is a dual-kernel, open source real time solution for Linux. Though it uses a separate kernel (and separate API) for real-time tasks, real-time tasks can be deployed within Linux user-space or within a dedicated real-time kernel that exists outside of the memory context of Linux. Xenomai includes RTOS "skins" which are recreations of APIs for some traditional commercial RTOS offerings. Note that such RTOS skins are not supported by the original RTOS vendor, and they may be incomplete implementations of the original RTOS API.

Linux community developers have been successful in building and applying Xenomai patches to Linux kernels for Zynq provided from Xilinx GIT. Please find those instructions at our Xenomai for Zynq page.

Xenomai is also supported by Xilinx development partners including:

Denx 5.4 release of ELDK includes support for Zynq with included Xenomai capabilities.
SYSGO also includes Xenomai support within their ElinOS product

Xenomai

Related Links

Multi-OS Solutions for Zynq - This wiki page describes ways that separate software stacks can co-exist with Linux in order to provide enhanced real-time support at the system level.
Real-Time Systems Presentation - 2010 presentation offering additional technical information related to implementation of preempt_rt in Linux-based systems.
Understanding and Using SMP - 2008 presentation but with some useful information
Preempt_rt