Introduction 

The legacy methodology of building AMD embeddedsw components takes .xsa as a handoff file from hardware persona and uses mdd, mld and mss files for different software configurations. This makes the legacy approach dependent on AMD proprietary tools such as the software command-line tool (XSCT) and hardware software interface (HSI). Without these proprietary tools, the embeddedsw components can't be compiled with the legacy approach. 

The proposed system device tree based flow aims to avoid such dependencies and embrace the open-source industry standards without compromising the performance and the functionality. It uses the system device tree, CMAKE and a Python-based open source tool named Lopper to organize the required Build System.

The system device tree is a super set of regular Linux device tree. Unlike regular Linux device tree which represents hardware information that is only needed for Linux/APU, system device tree represents complete hardware information in device tree format. It contains information about all processors (ex: PMC, PSM, RPU, APU) and all peripherals in the system. Details of steps to generate a system device tree can be found under Appendix A.

The System Device Tree based flow uses Lopper to extract the required hardware metadata from the system device tree. The software configurations earlier done via mld/mss files in the legacy approach is now replaced with the CMAKE-based infrastructure where the CMAKE variables are set to generate the required headers. All the component specific info which were kept earlier in mdd/mld/mss files in the legacy flow are now kept inside a YAML file. There are python scripts that read these YAML files, configures the component's sources using Lopper and CMAKE commands and populates the standalone BSP specific data accordingly. This way, the whole build system replaces the usage of proprietary AMD files and tools with the available open-source infrastructure.

Porting existing embeddedsw bare-metal and RTOS components is necessary to ensure legacy software components work seamlessly in this System Device Tree based flow. The following sections explain the steps and changes required to port a legacy bare-metal driver/library/application to the System Device Tree based flow.

Assumptions and Prerequisites

• This document makes the following assumptions from the users:

  • Familiarity with AMD embeddedsw drivers and libraries.

  • Familiarity with CMAKE and yaml.

  • Python basics.

What’s New

• The Open-Source System Device Tree specifications borrows heavily from Linux device tree specifications. The specifications do not have concepts of Device IDs to distinguish between similar peripherals.

• The System Device Tree based flow abstracts out interrupt handling through a wrapper. In the legacy flow the driver users had the responsibility to figure out the interrupt controller to be initialized (GIC or Soft AXI INTC). The new flow provides generic APIs that are to be called with the generated interrupt ID available in the Driver Config structures. The new flow introduces a new entry in the Driver Config structure with the name IntrParent that identifies whether the peripheral device is connected to GIC or AXI INTC for interrupt management.

• The System Device Tree based flow makes the xiltimer library inclusion mandatory. The xiltimer library abstracts out the timer and sleep handling in baremetal environments. A typical system can have multiple timer devices which can be used as timers or to implement sleep functionality. The xiltimer provides uniform implementation where the users do not have to worry about the underlying hardware timer. For more details of xiltimer, please refer to UG-643 Chapter 13.

Porting to the System Device Tree based flow

The following sections provide instructions to port the existing bare-metal drivers, libraries, and applications to work with the System Device Tree based flow.

Porting a driver

Every driver in AMD embeddedsw contains 4 folders as mentioned below

• data

• src

• examples (optional)

• doc (Not Relevant for this Porting Guide)

Changes in data folder

• Addition of a new YAML file:

A YAML file named <driver_name>.yaml must be added inside data folder. This YAML file is a replacement of the older AMD proprietary metadata files (*.mdd, *.tcl). For example, data folder under CSUDMA driver must contain a file named csudma.yaml.

The adopted yaml specification borrows heavily from open source Linux yaml device tree bindings with some customizations to work with AMD embeddedsw baremetal stack. The yaml file contains the following sections:

• Headers, copyrights, and generic information such as maintainer and type.

  # SPDX-License-Identifier: MIT %YAML 1.2 --- title: <Description of the yaml file> maintainers:   - Name(s) of the maintainer(s)  type: <Describes the type of baremetal module the yaml file describes, e.g, driver>

   

  • Specification(s) that help(s) picking up the corresponding driver for an IP. The corresponding driver would get picked up into BSP if the IP is present in the system device tree and its compatible property in the system device tree node matches with this "compatible" string specified in the driver yaml. It replaces the mdd file-based approach and instead relies on Linux-like compatible property. Detailed usage of compatible property can be found under Appendix B.

    properties:   compatible:      items:        - const: <Typically the name of the HW IP as seen in xsa file, e.g. xlnx,zynqmp-csudma-1.0>   reg:      description: <Physical base address and size of the controller register map>   other_properties_as_needed:     description: <Description of the property>

     

  • Specifications that help in generation of driver config file (*_g.c file) containing the driver config structure alongside the required xparameters.h. More Data on the usage of below section can be found in below snippet and under Appendix C, Appendix D.

  •  

    config:     - <The name of the driver config structure, e.g. XCsuDma_Config> required: # Describes the elements that will show up in the driver config structure # As an example, csudma driver *_g.c file will contain the XCsuDma_Config structure with following elements.     #- compatible     #- reg     #- xlnx,dma-type     #- interrupts     #- interrupt-parent # The order mentioned under the "required" section must be same as that the driver config structure. optional: # Entries that are needed in the config structure but not in xparameters.h will be generated using this option. # Same keys and rules as that under the required section additionalProperties: # Entries that are needed only in xparameters.h and not in the config structure will be generated using this option. # Same keys and rules as that under required section.

     

  • Specifications that describe driver example and their dependencies. Examples can depend on other supporting .h or .c files. Examples can have dependency on HW IP properties (like interrupts). Few examples can be valid only for few platforms. Please refer to the sample csudma.yaml below and in Appendix D for more info. Examples listed here will be listed under the import example section in the Vitis v2023.2 GUI.

     

  • Specifications that help in defining dependencies. A driver can be dependent on other drivers or a library (e.g. libmetal).

     

  • Specifications that help in generating peripheral tests. Examples mentioned here will be included in peripheral tests. For more insights, please refer Appendix E.

     

  • Below is a sample yaml file of an existing csudma driver. The yaml file name is csudma.yaml. More info on how the yaml takes the system device tree data and convert them into xparameters and Config structure can be found in Appendix D.

     

Changes in src Folder

• Introduction of a new CMakeLists.txt File: 

  • Instead of makefile, every driver should have a  CMakeLists.txt file. Almost all driver CMakeLists.txt look similar. Below is a typical driver CMakeLists.txt.

     

  • Sample CMakeLists.txt file for CSUDMA driver:

     

• Changes in existing source files (.c and .h):

  • AMD drivers have to be generic and the driver sources should not include platform specific configuration details through xparameters.h.

  • The existing embeddedsw driver uses a AMD proprietary parameter "DeviceId" in the driver config structure. Since the System Device Tree based flow "truly" aspires to be open source, it no more uses "DeviceId" and instead uses "BaseAddress". If a new driver is being created then developers need to use "BaseAddress" and not "DeviceId". If an old driver is being ported to this System Device Tree based flow, changes are needed to support both legacy ("DeviceId" based) and System Device Tree based flow ("BaseAddress" based).

  • A char * (for Name) in the driver *_Config structure has to be used to support this System Device Tree based flow. This can also be used later for differentiating between different versions of IP  in the driver.

These lead to the below changes in source files: 

• Sample changes in CSUDMA driver:

   

Changes in examples Folder

• LookUp_Config API has to be updated in all the driver examples to use base address instead of Device ID. 

• In the System Device Tree based flow, interrupt registration has been simplified and simpler APIs are provided to offload interrupt registration process.

  • All references to Interrupt handling in the example must be removed. Use XSetupInterruptSystem() for registering the driver/application handler and use XDisconnectInterruptCntrl for disable in the interrupts 

  • "interrupts" and "interrupt-parent" fields must be added to the Driver Config structure.

  • Please refer the changes in xcsudma_intr_example.c file below to get more detail.

• Sample changes for porting CSUDMA driver examples: 

Porting a Library 

Every library in AMD embeddedsw contains below 4 folders 

• data

• src

• examples (Optional)

• doc (Not Relevant for this Porting Guide)

Changes in data Folder

• Addition of a new YAML file:

A YAML file named <library_name>.yaml must be added inside data folder. This YAML file is a replacement of the older AMD proprietary metadata files (*.mld, *.tcl). For example, data folder under XILFPGA library must contain a file named xilfpga.yaml.

The yaml file contains the following sections:

• Headers, Copyrights, Versions, Generic information (maintainer, type etc)

   

  • Specification(s) that help(s) picking up the library properly for a given platform (processor/OS configuration).

     

  • Specification(s) that describe(s) the dependencies of the library. A library can depend on certain driver/IP configurations which must be present in the system device tree for the library to be used. In addition, a library can also depend upon some other libraries. 

     

  • Specifications that describe library example and their dependencies. Examples can depend on other supporting .h or .c files. There can be few examples only valid for few platforms. Examples listed here will be listed under the import example section in the Vitis 2023.2 GUI. Please refer sample xilfpga.yaml below for more info.

     

  • A sample library yaml (xilfpga.yaml):

     

     

Changes in src Folder

• Introduction of a new CMakeLists.txt File:  

• Instead of makefile, every library should have a  CMakeLists.txt file. Below is a typical library CMakeLists.txt.
  
  

   

   

  • As an example, the already available xilfpga library has a file CMakeLists.txt that has the following content. More details on Usage of CMAKE_MACHINE and CMAKE_SYSTEM_PROCESSOR can be found under Appendix F.

     

     

• If the library has software configuration it is facilitated by addition of two more files (<library_name>.cmake and x<library_name>_config.h.in):

  • It is typical for a library to have software configurations for users to make use of.

  • The System Device Tree based flow makes use of open source CMake based flow and replaces the existing mld/tcl based flow to configure the library. 

  • Combination of <library_name>.cmake and x<library_name>_config.h.in generates the required software configuration header file <library_name>_config.h based on the user configuration.

  • x<library_name>_config.h.in:

    • Using cmakedefine macro in config.h.in file, a series of define statements (macros) can be generated.

    • This config file when processed in conjunction with <library name>.cmake results into a .h (header) file which contains all the generated #define statements in C syntax

    • Sample xfpga_config_h_in file

       

       

    • Please refer to https://gitlab.kitware.com/cmake/community/wikis/doc/tutorials/How-To-Write-Platform-Checks for more details on how to use the configuration files with CMAKE.

• <library_name>.cmake :

  • The Software configuration options can be of  variety of data types. For example, these parameters can be a boolean, a hex value, an entry among a list of possible options or a number among a large range of numbers. All these software configurations tend to have a default value which can later be changed.

  • All the possible choices defined above can be described using below CMAKE commands:

    • For configuring parameters that have boolean properties

      • option(<option_variable> "help string describing option" [initial value])

      • For more info on options, please refer:  https://cmake.org/cmake/help/v3.13/command/option.html  

    • For configuring parameters whose values could be one in a list of values or a range of values
      
      

      • set(<variable> <value> [[CACHE <type> <docstring> [FORCE]] | PARENT_SCOPE])

      • For more info on set command, please refer: https://cmake.org/cmake/help/v3.0/command/set.html

• Once the cmake options and variables are set, configure_file() command has to be called. It calls the earlier described config.h.in file and generates the <libray name>_config.h

  • Below is a sample xilfpga.cmake (More details on Usage of CMAKE_MACHINE and CMAKE_SYSTEM_PROCESSOR can be found under Appendix F)

     

• Introduction of a new header file:

  • As seen in the last sub-section, a new file named <libray name>_config.h is originated by the configure_file() API when the <library_name>.cmake and x<library_name>_config.h.in files are processed together.

  • This separates out the library specific configurations into <libray name>_config.h file which earlier used to reside within xparameters.h in the existing flow.

  • This file will be auto-generated as mentioned before.

  • Sample xfpga_config.h :

     

• Changes in the existing source files (.c and .h):

  • Changes mentioned under the Driver Section's "Changes in existing source files (.c and .h)" must be complied to in library sources as well.

    • All the calls of LookupConfig() API of every driver must be updated to use BaseAddress instead of DeviceId

    • All the references to XPAR_<drv_name>_DEVICE_ID macro must be replaced with the macro defined under x<library_name>_config.h .

  • Include the newly generated x<library_name>_config.h in the .c files wherever needed

  • Below is a sample change:

     

     

Changes in examples Folder

Changes needed in library examples are similar to that needed for driver examples. Please refer to the "Changes in examples Folder" under "Porting a driver" section for more details. Below is a sample change:

Porting a Template application:

 Every Template application contains below folders

• data

• src

Changes in data Folder

• Addition of a new YAML file:

A YAML file named <app_name>.yaml must be added inside data folder. This YAML file is a replacement of the older AMD proprietary metadata files (*.mss, *.tcl). For example, data folder under hello_world template folder must contain a file named hello_world.yaml.

The yaml file contains the following sections:

• Headers, Copyrights, Versions, Generic information (maintainer, type etc)

   

  • Specification(s) that help(s) picking up the application properly for a given platform (processor/OS configuration).

     

  • Specification(s) that describes the dependencies of the Application. An application can depend on certain driver/IP configurations which must be present in the system device tree for the application to be compiled. In addition, an application can also depend on the libraries and can by default need specific library configurations depending upon the use case.

     

  • Specification(s) that describes the os-level and linker related requirements of the Application. These will be used when the application requires some change in the os level parameter or the linker heap and stack size. For more details on linker generation, please refer Appendix G.

     

  • A sample Application yaml (freertos_lwip_udp_perf_client.yaml):

     

Changes in src Folder

• Introduction of a new file CMakeLists.txt:

  • Instead of Makefile, every application should have a CMakeLists.txt file. Below is a sample CMakeLists.txt for hello_world template.

     

• Changes in the existing source files (.c and .h):

  • Changes mentioned under the Driver Section's "Changes in existing source files (.c and .h)" must be complied to in application sources as well.

    • All the calls of LookupConfig() API of every driver must be updated to use BaseAddress instead of DeviceId

  • Sample Change:

     

FAQs

• Why is AMD shifting to this System Device Tree based flow?

Earlier approach of using mdd, mld and mss files along with the .xsa as a handoff file involved a lot of AMD proprietary tools to build our software stack. This approach uses all the open source file formats and tools once the handoff file is generated by the Hardware Persona.

• How is the System Device Tree different than the Linux Device tree?

Linux Device Tree contains only the Linux/APU (or Microblaze) related peripheral info at a time. System Device Tree contains the whole system info (includes info related to everything i.e. APU, RPU, Microblaze, PMC etc.)

• How to differentiate your component whether it is a driver, library or an application?

 "type" key in every YAML determines this. Available options for this key are: driver, library, apps, os

• How is the driver probed/picked while building libxil?

Lopper backend checks for a given node compatible against the compatible specified in the driver yaml file under the properties section. if there is a match it picks the driver and generates _g.c file in the subsequent process.

• How are the _g.c entries and the xparameters generated for drivers?

Lopper backend generates the _g.c entries based on the properties specified in yaml file "required" section. Please make sure these entries are following the driver config structure order. Refer to Appendix C and Appendix D for more details.

• If the config structure entry has dependency on the other node base address, then how to handle it? 

The meta-data representation for the same is <property-name>: phandle (if the property is dependent on child node is available, it will read the reg property of that node and generates it's value in this entry otherwise generates zero). Refer to Appendix C and Appendix D for more details.

• If the config structure has more than one base address in sequence, then how to add that info in meta-data to generate proper addresses in config structure?

The meta-data representation for the above requirement is reg:<number of entries>. Refer to Appendix C for more details.

• If the property mentioned in the required section is not present in the device-tree node, then what value will get generated in the _g.c file?

Zero (0)

• How to add multiple configurations for a library?

Addition of <library_name>.cmake and x<library_name>_config.h.in facilitates the library software configuration. Please refer "Changes in src folder" under "Porting a Library" Section for more info.

• How to use platform or a processor specific piece of code?

CMAKE_MACHINE and CMAKE_SYSTEM_PROCESSOR variables inside cmake can be used to differentiate a platform or a processor specific piece of code. Please refer Appendix E to know the available values for this cmake variable.

• Where are the Library related configuration specific Macros located? It is missing in xparameters.h

Those have been relocated to the <library>_config.h file. Please refer "Changes in src folder" under "Porting a Library" Section for more info.

• How to define dependencies of a library or an application?

"depends" and "depends_libs" keys under a library YAML describes the Hardware and Software dependency of the library respectively. Please refer "Changes in data folder" under "Porting a Library" (or under "Porting a template Application") Section in conjunction with Appendix F for more details.

• How to define the archiver dependencies of the application?

collect(PROJECT_LIB_DEPS xil) kind of statements in Application CMakeLists defines this dependency. This statement means that the application depends on libxil.a.

• How is the linker script generated?

Intermediary file lscript.ld.in kept within the embeddedsw repository is copied into the application source folder which in conjunction with the <Application>Example.cmake generates the final lscript.ld. Refer to  Appendix G for more details.

• How to set a different stack and heap size for the application that doesn't fit in the default size?

"linker_constraints" key in the application YAMLs is used to modify the default stack and heap sizes. Please refer  Appendix G for more details.

Appendix A: System Device Tree (SDT) Generation at XSCT

System Device Tree:

• Represents complete hardware information in the form of device trees.

• All peripheral information and its properties, memories, clusters, soc peripheral information, soft IP information etc. available in the hardware design are represented in Linux-like device tree structure.

XSCT:

• XSCT (Xilinx Software Command-Line) is a tool that allows us to create complete Xilinx SDK workspaces using the batch mode, investigate the hardware and software, debug and run the project, all from the command line.

• More details on this tool can be accessed at UG-1208. System Device Tree Generation has been a recent addition to this tool. 

SDTGEN (Also known as DTG++):

• An XSCT package that uses tcls and Hardware HSI APIs to read the hardware information from XSA and put it in device tree format.

• The TCL source files for this package can be found under <Installed Vitis Path>/2023.2/data/system-device-tree-xlnx

• The package sources the device_tree.tcl file from <above_path>/device_tree/data/device_tree.tcl and exports the following three procs as commands:

  • set_dt_param 

  • get_dt_param

  • generate_sdt

Generation steps:

• Launch the installed xsct

• set_dt_param:

  • SDTGEN command that takes the user inputs like the Vivado XSA file and the system device tree output directory

  • It can also be used to set the system device tree parameters such as the board file, custom dts file etc.

  • Usage:

     

     

• get_dt_param:

  • SDTGEN command that can return the value set for a given parameter.

  • Usage:

     

• generate_sdt:

  • SDTGEN command that generates the system device tree with the set parameters

• Overall Usage Example:

   

System device tree output directory:

The generated dts directory can contain following files:

1. soc.dtsi, which is static soc specific file (e.g.: for versal: versal.dtsi, for zynqmp: zynqmp.dtsi, for zynq: zynq-7000.dtsi, for microblaze: there is no such soc specific file)

2. pl.dtsi which contains soft IPs information

3. board.dtsi, which is board file (Is pulled in only when -board_dts option is set)
   Sample e.g.: For versal: versal-vck190-rev1.1.dtsi / For zynqmp: zcu102-rev1.0.dtsi / For zynq: zc702.dtsi / For microblaze: kc705-lite.dtsi

4. clk.dtsi, which is clock information
   Ex: For versal: versal-clk.dtsi/ For zynqmp: zynqmp-clk-ccf.dtsi / For zynq, microblaze: There is no clock framework file 

5. system-top.dts, which is top level system information which contains memory, clusters, aliases etc.

6. pcw.dtsi, which contains peripheral configuration wizard information of the peripherals

7. psu_init* files for Zynq and ZU+, pdi file for Versal

8. For Versal designs, a folder with design name that contains the pdi_files contents

Appendix B: Usage of "compatible" under "properties"

Different ways to show compatible string under properties key of driver yamls:

• When there is one IP and one driver :

• When multiple IPs are having the same driver:

• When same IP is having different version:

Appendix C: Available options under "required" section

Note: Do check the canonical macros coming in the xparameters.h. In the legacy flow, few drivers had defined the canonical macros in their own way. This has now been made uniform across the driver. it now follows XPAR_X<driver_name>_<instance_number>_<PROPERTY> syntax. Thus, the corresponding changes are now needed in the source files.

Appendix D: A Comprehensive sample YAML for drivers

• A comprehensive Sample YAML file for drivers (Data points are only for syntax related understanding and is not related to any IP in real):

• A Sample Device Tree data for IP:

• Generated Config structure File:

• Generated xparameters.h for this driver:

Note:  The Canonical entries follow XPAR_X<driver_name>_<instance_number>_<PROPERTY> syntax.

Appendix E: A Driver example in the Peripheral Test App

How to include a Driver example in the Peripheral Test App:

Changes in data folder of the driver:

• Add a tapp section in the <driver>.yaml as mentioned earlier under "Porting a driver" Section. In existing flow, <driver>_tapp.tcl used to facilitate the same.

  • Sample "tapp" section in a <driver>.yaml contains:

     

• All the function declarations mentioned in the above tapp section must be available in the <driver>_header.h file.

• For Self Tests, example function declaration contains only one argument i.e BaseAddress

• For other example functions, declaration contains two arguments i.e instance name and BASEADDRESS

• Sample changes:

Changes in example folder of the driver:

Please refer to the "Changes in examples Folder" Under "Porting a driver" section

Appendix F: CMAKE_MACHINE and CMAKE_SYSTEM_PROCESSOR

• To differentiate a platform or a processor specific piece of code in cmake, CMAKE_MACHINE and CMAKE_SYSTEM_PROCESSOR variable can be used.

• List of available CMAKE_MACHINEs:

  Platform NameCMAKE_MACHINEZYNQ ZynqZynq Ultrscale+ZynqMPVersalVersal

  Kintex or Microblaze boards

  Name depends on family of the design.

  It can be kintex7, kintexu, virtex etc

   

• List of available CMAKE_SYSTEM_PROCESSORs:

   

  Processor NameCMAKE_SYSTEM_PROCESSOR (value)Soft MicroblazemicroblazePMU Microblazepmu_microblazePSM MicroblazemicroblazePLM Microblazeplm_microblazeCortexA53

  cortexa53

  CortexR5cortexr5CortexA72cortexa72CortexA9cortexa9

   

Appendix G: Apps: Hardware Dependency and Linker Script Generation

Generation of <Component>Example.cmake file under src folder and the linker script

Significance:

• It is an auto generated file inside library and application's src folders. It contains the required peripheral info in cmake format.

• The Lopper backends deduce the required peripherals data mentioned under "depends" section from system device tree and keep them in these Example.cmake file.

• The CMakeLists.txt kept under src folder then include these data into the Build System and generates the required Software level configurations.

• For Applications:

  • This cmake also contains the available memory related info which will translate into the linker script.

  • This cmake in conjunction with lscript.ld.in results into the final application's lscript.ld

Sample depends and linker sections:

Sample Lwip_echo_serverExample.cmake:

Sample lscript.ld.in file:

Known Issues

1. Incremental Build for the BSP and the application is not yet available. In case of any source change in BSP, the whole BSP has to be compiled again and the same goes for the application workspace.

2. Hardware related errors for a template application or a library have to be explicitly added via cmake files. Earlier it was handled in tcl but the same is not translated to yaml.

3. The BSP workspace and the Application Workspaces are not completely relocatable. Build folders (for BSP, it is <bsp_ws>/libsrc/build_configs and for application, it is <app_ws>/build) have to be deleted before relocating.

4. All the microblaze designs are expected to have "microblaze" in their processor names. This issue will be resolved in the coming release.

5. In case of Embeddedsw repo having the multiple versions of the same component (say csudma_v3_5 and csudma_v3_6 as it used to be in the legacy flow), the build system will pick the first source sequenced alphabetically by default (csudma_v3_5 in this case). The source can later be changed (csudma_v_6 path can be chosen among the available options) once the BSP is created.

6. Not all drivers/libraries/apps/examples in embeddedsw are ported to this new flow. Incase if a driver/library/app isnt ported, it wouldn't be considered while creating BSP/application and the example wouldnt showup for selection in GUI.

7. IAR/armcc compilers are not supported

8. ZynqUS+ A53 32-bit is not supported

9. 64 bit Soft Microblaze is not supported

10. Few of the BSP/Library configuration settings have been relocated and some of them have been removed during this transition to new SDT based flow.

11. Libmetal and OpenAM FreeRTOS apps arent supported yet and support of OpenAMP for Zynq Cortex-A9 and Versal, ZU+ R5_1 processsors isnt present yet in this new flow

12. STDIN and STDOUT configuration selection for FreeRTOS BSP isnt present.

13. Memory region names are slightly modified in the linker script from the legacy flow. Linear_qspi/EMC memories are currently missing in the linker script.

14. LwIP library can't be added to a FreeRTOS platform BSP settings. Workaround is to create a FreeRTOS lwIP template application and place the files needed for the custom application in the source folder. Default lwIP FreeRTOS apps work as expected and are unaffected by this issue.