Porting embeddedsw components to system device tree (SDT) based flow

Introduction 

The legacy methodology of building AMD embeddedsw components takes .xsa as a handoff file from hardware persona and uses mdd, mld and mss files for different software configurations. This makes the legacy approach dependent on AMD proprietary tools such as the software command-line tool (XSCT) and hardware software interface (HSI). Without these proprietary tools, the embeddedsw components can't be compiled with the legacy approach. 

The proposed system device tree based flow aims to avoid such dependencies and embrace the open-source industry standards without compromising the performance and the functionality. It uses the system device tree, CMAKE and a Python-based open source tool named Lopper to organize the required Build System.

The system device tree is a super set of regular Linux device tree. Unlike regular Linux device tree which represents hardware information that is only needed for Linux/APU, system device tree represents complete hardware information in device tree format. It contains information about all processors (ex: PMC, PSM, RPU, APU) and all peripherals in the system. Details of steps to generate a system device tree can be found under Appendix A.

The System Device Tree based flow uses Lopper to extract the required hardware metadata from the system device tree. The software configurations earlier done via mld/mss files in the legacy approach is now replaced with the CMAKE-based infrastructure where the CMAKE variables are set to generate the required headers. All the component specific info which were kept earlier in mdd/mld/mss files in the legacy flow are now kept inside a YAML file. There are python scripts that read these YAML files, configures the component's sources using Lopper and CMAKE commands and populates the standalone BSP specific data accordingly. This way, the whole build system replaces the usage of proprietary AMD files and tools with the available open-source infrastructure.

Porting existing embeddedsw bare-metal and RTOS components is necessary to ensure legacy software components work seamlessly in this System Device Tree based flow. The following sections explain the steps and changes required to port a legacy bare-metal driver/library/application to the System Device Tree based flow.

Assumptions and Prerequisites

  • This document makes the following assumptions from the users:

    • Familiarity with AMD embeddedsw drivers and libraries.

    • Familiarity with CMAKE and yaml.

    • Python basics.

What’s New in EmbeddedSW for this Flow

 

  • The Open-Source System Device Tree specifications borrows heavily from Linux device tree specifications. The specifications do not have concepts of Device IDs to distinguish between similar peripherals.

As an example, in the legacy flow, if there are two Ethernet MACs in a system from the same vendor, each of the MACs would be assigned a separate Device ID. The Device IDs are positive integers starting from 0. The Driver Config structure always has a field "u16 DeviceId" that represents the assigned Device ID.

The System Device Tree based flow gets rid of Device IDs and hence Driver Config structures do not have an entry for Device ID. Instead, Driver Config structures have an entry "char *Name" that stores the Compatible String(s)" for the corresponding device. This Compatible property along with the base address of the device are used the uniquely identify a device in a system.

  • The System Device Tree based flow abstracts out interrupt handling through a wrapper. In the legacy flow the driver users had the responsibility to figure out the interrupt controller to be initialized (GIC or Soft AXI INTC). The new flow provides generic APIs that are to be called with the generated interrupt ID available in the Driver Config structures. The new flow introduces a new entry in the Driver Config structure with the name IntrParent that identifies whether the peripheral device is connected to GIC or AXI INTC for interrupt management For more details refer this section.

  • The System Device Tree based flow makes the xiltimer library inclusion mandatory. The xiltimer library abstracts out the timer and sleep handling in baremetal environments. A typical system can have multiple timer devices which can be used as timers or to implement sleep functionality. The xiltimer provides uniform implementation where the users do not have to worry about the underlying hardware timer. For more details of xiltimer, please refer to UG-643 Chapter 13.

Updates in 2024.1

 

  • New support added for Microblaze RISC-V processor and 64 bit Soft Microblaze processor

  • Incremental Build support for BSP

  • Enhanced Hardware related error mechanism

  • Improvement in BSP creation and build timing over Windows with the usage of Ninja generator

  • Fixed known gaps from 2023.2:

    • The BSP workspace and the Application Workspaces are now relocatable.

    • Microblaze designs no longer need to have "microblaze" in their processor names.

    • Latest source versions are picked irrespective of alphabetical order.

    • STDIN and STDOUT configuration selection are available for FreeRTOS BSP.

    • Addition of Linear flash devices as an available memory section in the linker script.

    • Addition of LwIP library to a FreeRTOS platform BSP settings

  • Addition of new lopper assists:

    • gen_domain_dts : to generate the domain specific dts (i.e. cortexa53.dts, cortexr5.dts, linux.dts etc.). Earlier it was done using lops.

    • baremetal_validate_comp_xlnx : to facilitate the hardware related error mechanism.

  • Updates in YAML spec:

    • device_type : Helps in giving meaningful error when there is a dependency of one kind of IP over a template application, a library or a driver. Available values are ethernet, serial, interrupt-controller, ipi and timer.

    • xparam_prefix : To specify a name different than that of ip-name in canonical macros

  • Updates in xparameters.h :

    • Addition of XPAR_CPU_ID macro

    • Addition of macros for generic interrupt IDs as available in legacy flow (in addition to the available macros used by interrupt wrapper)

    • Addition of macros for all the available addresses (from reg property) and interrupts (from interrupt property) in the device node.

    • Updated NOC related macros to follow the same names as that in legacy flow

    • Removal of the Interrupt ID macro for PS peripherals from xparameters.h (except for IPI). These macros have to be included from xparameters_ps.h

  • If driver is not compatible with SDT or the driver YAML doesn't have the "required" section, config file will not be generated. Earlier empty config file used to be generated.

  • A new property "is-hierarchy" is added to Hierarchical IP nodes. This is used to distinguish the IP sub-cores which are part of a subsystem and not accessible from AXI bus. Such IP sub-cores will not be treated as system level peripheral (like timer, gpio etc)

  • The Interrupt Controller example is made the first example in the peripheral tests sequence.

  • Updated logic to preserve the system device tree node order while generating metadata in xparameters.h and config files.

  • Updated generate_config_object assist to facilitate user-driven customization of xilpm library options.

Porting to the System Device Tree based flow

The following sections provide instructions to port the existing bare-metal drivers, libraries, and applications to work with the System Device Tree based flow.

Porting a driver

Every driver in AMD embeddedsw contains 4 folders as mentioned below

  • data

  • src

  • examples (optional)

  • doc (Not Relevant for this Porting Guide)

Changes in data folder

  • Addition of a new YAML file:

A YAML file named <driver_name>.yaml must be added inside data folder. This YAML file is a replacement of the older AMD proprietary metadata files (*.mdd, *.tcl). For example, data folder under CSUDMA driver must contain a file named csudma.yaml.

The adopted yaml specification borrows heavily from open source Linux yaml device tree bindings with some customizations to work with AMD embeddedsw baremetal stack. The yaml file contains the following sections:

  • Headers, copyrights, and generic information such as maintainer and type.

    # SPDX-License-Identifier: MIT %YAML 1.2 --- title: <Description of the yaml file> maintainers:   - Name(s) of the maintainer(s)  type: <Describes the type of baremetal module the yaml file describes, e.g, driver> device_type: <Describes the type of devices that the driver supports> # Helps in giving meaningful error when there is a dependency of one kind of IP over # a template application, a library or a driver. # Available values are ethernet, serial, interrupt-controller, ipi and timer.

     

    • Specification(s) that help(s) picking up the corresponding driver for an IP. The corresponding driver would get picked up into BSP if the IP is present in the system device tree and its compatible property in the system device tree node matches with this "compatible" string specified in the driver yaml. It replaces the mdd file-based approach and instead relies on Linux-like compatible property. Detailed usage of compatible property can be found under Appendix B.

      properties:   compatible:      items:        - const: <Typically the name of the HW IP as seen in xsa file, e.g. xlnx,zynqmp-csudma-1.0>   reg:      description: <Physical base address and size of the controller register map>   other_properties_as_needed:     description: <Description of the property>

       

    • Specifications that help in generation of driver config file (*_g.c file) containing the driver config structure alongside the required xparameters.h. More Data on the usage of below section can be found in below snippet and under Appendix C, Appendix D.

    •  

      config:     - <The name of the driver config structure, e.g. XCsuDma_Config> xparam_prefix: xcsu # Specifies a custom name different than that of ip-name in canonical macros # General canonical macros follow the XPAR_X<driver_name>_<instance_number>_<PROPERTY> syntax # X<driver_name> will be replaced with the value of xparam_prefix in the above defined syntax required: # Describes the elements that will show up in the driver config structure # As an example, csudma driver *_g.c file will contain the XCsuDma_Config structure with following elements.     #- compatible     #- reg     #- xlnx,dma-type     #- interrupts     #- interrupt-parent # The order mentioned under the "required" section must be same as that the driver config structure. optional: # Entries that are needed in the config structure but not in xparameters.h will be generated using this option. # Same keys and rules as that under the required section additionalProperties: # Entries that are needed only in xparameters.h and not in the config structure will be generated using this option. # Same keys and rules as that under required section.

       

    • Specifications that describe driver example and their dependencies. Examples can depend on other supporting .h or .c files. Examples can have dependency on HW IP properties (like interrupts). Few examples can be valid only for few platforms. Please refer to the sample csudma.yaml below and in Appendix D for more info. Examples listed here will be listed under the import example section in the Vitis v2023.2 GUI.

       

    • Specifications that help in defining dependencies. A driver can be dependent on other drivers or a library (e.g. libmetal).

       

    • Specifications that help in generating peripheral tests. Examples mentioned here will be included in peripheral tests. For more insights, please refer Appendix E.

       

    • Below is a sample yaml file of an existing csudma driver. The yaml file name is csudma.yaml. More info on how the yaml takes the system device tree data and convert them into xparameters and Config structure can be found in Appendix D.

       

Changes in src Folder

  • Introduction of a new CMakeLists.txt File: 

    • Instead of makefile, every driver should have a  CMakeLists.txt file. Almost all driver CMakeLists.txt look similar. Below is a typical driver CMakeLists.txt.

       

    • Sample CMakeLists.txt file for CSUDMA driver:

       

  • Changes in existing source files (.c and .h):

    • AMD drivers have to be generic and the driver sources should not include platform specific configuration details through xparameters.h.

    • The existing embeddedsw driver uses a AMD proprietary parameter "DeviceId" in the driver config structure. Since the System Device Tree based flow "truly" aspires to be open source, it no more uses "DeviceId" and instead uses "BaseAddress". If a new driver is being created then developers need to use "BaseAddress" and not "DeviceId". If an old driver is being ported to this System Device Tree based flow, changes are needed to support both legacy ("DeviceId" based) and System Device Tree based flow ("BaseAddress" based).

    • A char * (for Name) in the driver *_Config structure has to be used to support this System Device Tree based flow. This can also be used later for differentiating between different versions of IP  in the driver.

These lead to the below changes in source files: 

 

 

  • Sample changes in CSUDMA driver:

     

Changes in examples Folder

  • LookUp_Config API has to be updated in all the driver examples to use base address instead of Device ID. 

  • In the System Device Tree based flow, interrupt registration has been simplified and simpler APIs are provided to offload interrupt registration process.

    • All references to Interrupt handling in the example must be removed. Use XSetupInterruptSystem() for registering the driver/application handler and use XDisconnectInterruptCntrl for disable in the interrupts 

    • "interrupts" and "interrupt-parent" fields must be added to the Driver Config structure.

    • Please refer the changes in xcsudma_intr_example.c file below to get more detail.

  • In the xparameters.h file, two definitions for each interrupt are generated

    • A definition ending with _INTR, which is used in the legacy code. This is the raw interrupt number or ID that is used with the old interrupt handling functions. For PS peripherals except IPI, this macro is available in the xparameters_ps.h file, it won’t be generated in xparameters.h file.

    • A definition ending with _INTERRUPTS, which is used with the new interrupt wrapper API. This is a u16 variable which contains more information about the interrupt, such as the interrupt number, the priority, and the interrupt sensitivity information.

    • For interrupt wrapper definition refer here

    • Note: FreeRTOS port Interrupt calls like vPortEnableInterrupt and vPortEnableInterrupt are using Interrupt wrapper API calls, It expects interrupt ID's ending with _INTERRUPTS macro.

    • More Data on the usage can be found in below snippet and under Appendix C, Appendix D.

  • Sample changes for porting CSUDMA driver examples: 

 

 

 

 

Porting a Library 

Every library in AMD embeddedsw contains below 4 folders 

  • data

  • src

  • examples (Optional)

  • doc (Not Relevant for this Porting Guide)

Changes in data Folder

  • Addition of a new YAML file:

A YAML file named <library_name>.yaml must be added inside data folder. This YAML file is a replacement of the older AMD proprietary metadata files (*.mld, *.tcl). For example, data folder under XILFPGA library must contain a file named xilfpga.yaml.

The yaml file contains the following sections:

  • Headers, Copyrights, Versions, Generic information (maintainer, type etc)

     

    • Specification(s) that help(s) picking up the library properly for a given platform (processor/OS configuration).

       

    • Specification(s) that describe(s) the dependencies of the library. A library can depend on certain driver/IP configurations which must be present in the system device tree for the library to be used. In addition, a library can also depend upon some other libraries. 

       

    • Specifications that describe library example and their dependencies. Examples can depend on other supporting .h or .c files. There can be few examples only valid for few platforms. Examples listed here will be listed under the import example section in the Vitis 2023.2 GUI. Please refer sample xilfpga.yaml below for more info.

       

    • A sample library yaml (xilfpga.yaml):

       

       

Changes in src Folder

  • Introduction of a new CMakeLists.txt File:  

  • Instead of makefile, every library should have a  CMakeLists.txt file. Below is a typical library CMakeLists.txt.

     

     

    • As an example, the already available xilfpga library has a file CMakeLists.txt that has the following content. More details on Usage of CMAKE_MACHINE and CMAKE_SYSTEM_PROCESSOR can be found under Appendix F.

       

       

  • If the library has software configuration it is facilitated by addition of two more files (<library_name>.cmake and x<library_name>_config.h.in):

    • It is typical for a library to have software configurations for users to make use of.

    • The System Device Tree based flow makes use of open source CMake based flow and replaces the existing mld/tcl based flow to configure the library. 

    • Combination of <library_name>.cmake and x<library_name>_config.h.in generates the required software configuration header file <library_name>_config.h based on the user configuration.

    • x<library_name>_config.h.in:

  • <library_name>.cmake :

  • Once the cmake options and variables are set, configure_file() command has to be called. It calls the earlier described config.h.in file and generates the <libray name>_config.h

    • Below is a sample xilfpga.cmake (More details on Usage of CMAKE_MACHINE and CMAKE_SYSTEM_PROCESSOR can be found under Appendix F)

       

  • Introduction of a new header file:

    • As seen in the last sub-section, a new file named <libray name>_config.h is originated by the configure_file() API when the <library_name>.cmake and x<library_name>_config.h.in files are processed together.

    • This separates out the library specific configurations into <libray name>_config.h file which earlier used to reside within xparameters.h in the existing flow.

    • This file will be auto-generated as mentioned before.

    • Sample xfpga_config.h :

       

  • Changes in the existing source files (.c and .h):

    • Changes mentioned under the Driver Section's "Changes in existing source files (.c and .h)" must be complied to in library sources as well.

      • All the calls of LookupConfig() API of every driver must be updated to use BaseAddress instead of DeviceId

      • All the references to XPAR_<drv_name>_DEVICE_ID macro must be replaced with the macro defined under x<library_name>_config.h .

    • Include the newly generated x<library_name>_config.h in the .c files wherever needed

    • Below is a sample change:

       

       

Changes in examples Folder

Changes needed in library examples are similar to that needed for driver examples. Please refer to the "Changes in examples Folder" under "Porting a driver" section for more details. Below is a sample change:

 

Porting a Template application:

 Every Template application contains below folders

  • data

  • src

Changes in data Folder

  • Addition of a new YAML file:

A YAML file named <app_name>.yaml must be added inside data folder. This YAML file is a replacement of the older AMD proprietary metadata files (*.mss, *.tcl). For example, data folder under hello_world template folder must contain a file named hello_world.yaml.

The yaml file contains the following sections:

  • Headers, Copyrights, Versions, Generic information (maintainer, type etc)

     

    • Specification(s) that help(s) picking up the application properly for a given platform (processor/OS configuration).

       

    • Specification(s) that describes the dependencies of the Application. An application can depend on certain driver/IP configurations which must be present in the system device tree for the application to be compiled. In addition, an application can also depend on the libraries and can by default need specific library configurations depending upon the use case.

       

    • Specification(s) that describes the os-level and linker related requirements of the Application. These will be used when the application requires some change in the os level parameter or the linker heap and stack size. For more details on linker generation, please refer Appendix G.

       

    • A sample Application yaml (freertos_lwip_udp_perf_client.yaml):

       

Changes in src Folder

  • Introduction of a new file CMakeLists.txt:

    • Instead of Makefile, every application should have a CMakeLists.txt file. Below is a sample CMakeLists.txt for hello_world template.

       

  • Changes in the existing source files (.c and .h):

    • Changes mentioned under the Driver Section's "Changes in existing source files (.c and .h)" must be complied to in application sources as well.

      • All the calls of LookupConfig() API of every driver must be updated to use BaseAddress instead of DeviceId

    • Sample Change:

       

FAQs

  • Why is AMD shifting to this System Device Tree based flow?

Earlier approach of using mdd, mld and mss files along with the .xsa as a handoff file involved a lot of AMD proprietary tools to build our software stack. This approach uses all the open source file formats and tools once the handoff file is generated by the Hardware Persona.

  • How is the System Device Tree different than the Linux Device tree?

Linux Device Tree contains only the Linux/APU (or Microblaze) related peripheral info at a time. System Device Tree contains the whole system info (includes info related to everything i.e. APU, RPU, Microblaze, PMC etc.)

  • How to differentiate your component whether it is a driver, library or an application?

 "type" key in every YAML determines this. Available options for this key are: driver, library, apps, os

  • How is the driver probed/picked while building libxil?

Lopper backend checks for a given node compatible against the compatible specified in the driver yaml file under the properties section. if there is a match it picks the driver and generates _g.c file in the subsequent process.

  • How are the _g.c entries and the xparameters generated for drivers?

Lopper backend generates the _g.c entries based on the properties specified in yaml file "required" section. Please make sure these entries are following the driver config structure order. Refer to Appendix C and Appendix D for more details.

  • If the config structure entry has dependency on the other node base address, then how to handle it? 

The meta-data representation for the same is <property-name>: phandle (if the property is dependent on child node is available, it will read the reg property of that node and generates it's value in this entry otherwise generates zero). Refer to Appendix C and Appendix D for more details.

  • If the config structure has more than one base address in sequence, then how to add that info in meta-data to generate proper addresses in config structure?

The meta-data representation for the above requirement is reg:<number of entries>. Refer to Appendix C for more details.

  • If the property mentioned in the required section is not present in the device-tree node, then what value will get generated in the _g.c file?

Zero (0)

  • How to add multiple configurations for a library?

Addition of <library_name>.cmake and x<library_name>_config.h.in facilitates the library software configuration. Please refer "Changes in src folder" under "Porting a Library" Section for more info.

  • How to use platform or a processor specific piece of code?

CMAKE_MACHINE and CMAKE_SYSTEM_PROCESSOR variables inside cmake can be used to differentiate a platform or a processor specific piece of code. Please refer Appendix E to know the available values for this cmake variable.

  • Where are the Library related configuration specific Macros located? It is missing in xparameters.h

Those have been relocated to the <library>_config.h file. Please refer "Changes in src folder" under "Porting a Library" Section for more info.

  • How to define dependencies of a library or an application?

"depends" and "depends_libs" keys under a library YAML describes the Hardware and Software dependency of the library respectively. Please refer "Changes in data folder" under "Porting a Library" (or under "Porting a template Application") Section in conjunction with Appendix F for more details.

  • How to define the archiver dependencies of the application?

collect(PROJECT_LIB_DEPS xil) kind of statements in Application CMakeLists defines this dependency. This statement means that the application depends on libxil.a.

  • How is the linker script generated?

Intermediary file lscript.ld.in kept within the embeddedsw repository is copied into the application source folder which in conjunction with the <Application>Example.cmake generates the final lscript.ld. Refer to  Appendix G for more details.

  • How to set a different stack and heap size for the application that doesn't fit in the default size?

"linker_constraints" key in the application YAMLs is used to modify the default stack and heap sizes. Please refer  Appendix G for more details.

 

Appendix A: System Device Tree (SDT) Generation at XSCT

Refer this README for more details on SDTGen.

System Device Tree:

  • Represents complete hardware information in the form of device trees.

  • All peripheral information and its properties, memories, clusters, soc peripheral information, soft IP information etc. available in the hardware design are represented in Linux-like device tree structure.

XSCT:

  • XSCT (Xilinx Software Command-Line) is a tool that allows us to create complete Xilinx SDK workspaces using the batch mode, investigate the hardware and software, debug and run the project, all from the command line.

  • More details on this tool can be accessed at UG-1208. System Device Tree Generation has been a recent addition to this tool. 

SDTGEN (Also known as DTG++):

  • An XSCT package that uses tcls and Hardware HSI APIs to read the hardware information from XSA and put it in device tree format.

  • The TCL source files for this package can be found under <Installed Vitis Path>/2023.2/data/system-device-tree-xlnx

  • The package sources the device_tree.tcl file from <above_path>/device_tree/data/device_tree.tcl and exports the following three procs as commands:

    • set_dt_param 

    • get_dt_param

    • generate_sdt

Generation steps:

  • Launch the installed xsct

  • set_dt_param:

    • SDTGEN command that takes the user inputs like the Vivado XSA file and the system device tree output directory

    • It can also be used to set the system device tree parameters such as the board file, custom dts file etc.

    • Usage:

       

       

  • get_dt_param:

    • SDTGEN command that can return the value set for a given parameter.

    • Usage:

       

  • generate_sdt:

    • SDTGEN command that generates the system device tree with the set parameters

  • Overall Usage Example:

     

System device tree output directory:

The generated dts directory can contain following files:

  1. soc.dtsi, which is static soc specific file (e.g.: for versal: versal.dtsi, for zynqmp: zynqmp.dtsi, for zynq: zynq-7000.dtsi, for microblaze: there is no such soc specific file)

  2. pl.dtsi which contains soft IPs information

  3. board.dtsi, which is board file (Is pulled in only when -board_dts option is set)
    Sample e.g.: For versal: versal-vck190-rev1.1.dtsi / For zynqmp: zcu102-rev1.0.dtsi / For zynq: zc702.dtsi / For microblaze: kc705-lite.dtsi

  4. clk.dtsi, which is clock information
    Ex: For versal: versal-clk.dtsi/ For zynqmp: zynqmp-clk-ccf.dtsi / For zynq, microblaze: There is no clock framework file 

  5. system-top.dts, which is top level system information which contains memory, clusters, aliases etc.

  6. pcw.dtsi, which contains peripheral configuration wizard information of the peripherals

  7. psu_init* files for Zynq and ZU+, pdi file for Versal

  8. For Versal designs, a folder with design name that contains the pdi_files contents

Appendix B: Usage of "compatible" under "properties"

Different ways to show compatible string under properties key of driver yamls:

  • When there is one IP and one driver :

 

  • When multiple IPs are having the same driver:

 

  • When same IP is having different version:

Appendix C: Available options under "required" section

 

Key

Notes

Config structure

XPARAMETERS