libdfx - Linux User Space Solution for FPGA Programming

 

Introduction

The library is a lightweight user-space library built on top of the Linux driver stack to support the FPGA device programming. This 'C' library can be built statically and needs to be integrated with user application. It provides different APIs that can address multiple use cases for DFX or PL configuration data programming. It also provides faster programming capability by avoiding multiple buffer copies that are involved in other methods.

Source Code

https://github.com/Xilinx/libdfx.git

API Details

 Pre-fetch

  • dfx_cfg_init (const char *dfx_package_path, const char *devpathu32 flags);

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 /* Provide a generic interface to the user to specify the required parameters for PR programming.  * The calling process must call this API before it performs -load/remove.  *  * char *dfx_package_path: The contents of the package folder should look something as below:  *                                -package: //-package1   *                                                                |--> Bit_file  *                                                                |--> DT_Overlayfile  *  *  char *devpath: Unused for now. The dev interface for now is always exposed at /dev/fpga0  *  * unsigned long flags: Flags to specify any special instructions for library to perform.  *                      Unused for now.  *  * Return: returns unique package_Id or Error code on failure.  */     Usage example: #include "libdfx.h"    package_id1, package_id2;   /* More code */ /* -store /Pre-fetch data */ package_id1 = dfx_cfg_init ("/path/package1/", "/dev/fpga0", flags);   /* More code */   /* -store /Pre-fetch data */ package_id2 = dfx_cfg_init ("/path/package2/", "/dev/fpga0", flags);   /* More code */



fpga-load

  • int dfx_cfg_load ( struct dfx_package_Id *package_Id)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 /* This API is Responsible for the following things.  *      -->Load  into the PL  *      -->Probe the Drivers which are relevant to the Bitstream as per DT overlay mentioned in dfx_package folder)  *  *  package_id: Unique package_id value which was returned by dfx_cfg_init.  *  * Return: returns zero on success or Error code on failure.  */     Usage example: #include "libfpga.h"   /* More code */   ret = dfx_cfg_load (package_id); if (ret)     return -1   /* More code */



Deferred-drivers-load

  • int dfx_cfg_drivers_load(struct dfx_package_Id *package_Id)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 /* This API is Responsible for the following things.  *      -->Probe the Drivers which are relevant to the Bitstream as per  *         DT overlay mentioned in dfx_package folder(With name: *_d.dtbo)  *  * package_id: Unique package_id value which was returned by dfx_cfg_init.  *  * Return: returns zero on success or Error code on failure.  */     Usage example: #include "libdfx.h"   /* More code */   ret = dfx_cfg_drivers_load(package_id); if (ret)         return -1   /* More code */

Remove

  • dfx_cfg_remove (package_Id)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 /* This API is Responsible for unloading the drivers corresponding to a package  *      *  package_id: Unique package_id value which was returned by dfx_cfg_init.  *  * Return: returns zero on success or Error code on failure.  */     Usage example: #include "libdfx.h"     /* More code */   ret = dfx_cfg_remove (package_id);  if (ret)     return -1; /* More code */





Destroy package

  • dfx_cfg_destroy (package_Id)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 /* This API frees the resources allocated during dfx_cfg_init.  *      *  package_id: Unique package_id value which was returned by dfx_cfg_init.  *  * Return: returns zero on success.  On error, -1 is returned.  */     Usage example: #include "libdfx.h"     /* More code */    ret = dfx_cfg_destroy (package_id); /* Returns zero on success.  On error, -1 is returned */  if (ret)     return -1   /* More code */ ................


Example Application flow

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 #include "libfpga.h"   int main() {  package_id, ret;         /* package initialization */     package_id = dfx_cfg_init(pck1/, /dev/fpga0, 0);      if (package_id < 0)          return -1;                        /* Package load */     ret = dfx_cfg_load(package_id);     if (ret)         return -1;       /* Remove package */     ret = dfx_cfg_remove(package_id);     if (ret)         return -1;       /* Destroy package */     ret = dfx_cfg_destroy(package_id);       return ret;   }



Build procedure

Build procedure for  compiling library from source

  1.  clone :  https://github.com/Xilinx/libdfx.git

  2. mkdir build

  3. cd build

  4. Ensure required tool chain added to your path.

  5. Execute cmake  -DCMAKE_TOOLCHAIN_FILE="cmake tool chain file(complete path)"  ..  

    • Example:  cmake -DCMAKE_TOOLCHAIN_FILE="../libdfx/cmake/toolchain.cmake" ..

  6. cd src

  7. make

  8. The required libdfx.a is now available in build/src folder.

  1. clone :  https://github.com/Xilinx/libdfx.git

  2. Replace the existing apps/libdfx_app.C contents with the user required application.

  3. mkdir build

  4. cd build

  5. Ensure required tool chain added to your path

  6. cmake  -DCMAKE_TOOLCHAIN_FILE="cmake tool chain file(complete path)"  ..  

    • Example:  cmake -DCMAKE_TOOLCHAIN_FILE="../libdfx/cmake/toolchain.cmake" ..

  7. make

  8. The final elf will be available at: apps/dfx_app

Reference DTBO file format

The Device Tree Overlay (DTO) is used to reprogram an FPGA while Linux is running. The DTO overlay will add the child node and the fragments from the .dtbo file to the base device tree,

The newly added device node/drivers will be probed after PDI /BIN/BIT programming.

Devicetree Overlay file contents example: For Only PDI/BIN/BIT configuration

1 2 3 4 5 6 7 8 9 10 11 /dts-v1/; /plugin/; / {         fragment@0 {                 target = <&fpga>;                 overlay0: __overlay__ {                         firmware-name = "partial_1.pdi";                         partial-fpga-config;                                        };         }; };



PL drivers probing ( For Deferred Probe)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 /dts-v1/; /plugin/; / {         fragment@0 {                 target = <&amba>;                 __overlay__ {                         axi_gpio_0: gpio@a0000000 {                                 #gpio-cells = <3>;                                 clock-names = "s_axi_aclk";                                 clocks = <&zynqmp_clk 71>;                                 compatible = "xlnx,axi-gpio-2.0", "xlnx,xps-gpio-1.00.a";                                 gpio-controller ;                                 reg = <0x0 0xa0000000 0x0 0x1000>;                                 xlnx,all-inputs = <0x0>;                                 xlnx,all-inputs-2 = <0x0>;                                 xlnx,all-outputs = <0x1>;                                 xlnx,all-outputs-2 = <0x0>;                                 xlnx,dout-default = <0x00000000>;                                 xlnx,dout-default-2 = <0x00000000>;                                 xlnx,gpio-width = <0x8>;                                 xlnx,gpio2-width = <0x20>;                                 xlnx,interrupt-present = <0x0>;                                 xlnx,is-dual = <0x0>;                                 xlnx,tri-default = <0xFFFFFFFF>;                                 xlnx,tri-default-2 = <0xFFFFFFFF>;                         };             }; };



Devicetree Overlay file contents example: For PDI configuration + PL drivers probing

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 /dts-v1/; /plugin/; / {         fragment@0 {                 target = <&fpga>;                 overlay0: __overlay__ {                         firmware-name = "partial_1.pdi";                         partial-fpga-config;                                        };         fragment@1 {                 target = <&amba>;                 overlay1: __overlay__ {                         axi_gpio_0: gpio@a0000000 {                                 #gpio-cells = <3>;                                 clock-names = "s_axi_aclk";                                 clocks = <&zynqmp_clk 71>;                                 compatible = "xlnx,axi-gpio-2.0", "xlnx,xps-gpio-1.00.a";                                 gpio-controller ;                                 reg = <0x0 0xa0000000 0x0 0x1000>;                                 xlnx,all-inputs = <0x0>;                                 xlnx,all-inputs-2 = <0x0>;                                 xlnx,all-outputs = <0x1>;                                 xlnx,all-outputs-2 = <0x0>;                                 xlnx,dout-default = <0x00000000>;                                 xlnx,dout-default-2 = <0x00000000>;                                 xlnx,gpio-width = <0x8>;                                 xlnx,gpio2-width = <0x20>;                                 xlnx,interrupt-present = <0x0>;                                 xlnx,is-dual = <0x0>;                                 xlnx,tri-default = <0xFFFFFFFF>;                                 xlnx,tri-default-2 = <0xFFFFFFFF>;                         };             }; };

Create Device Tree Overlay Blob (.dtbo) file from .dts file

  •   # dtc -O dtb -o partial.dtbo -b 0 -@ partial.dtsi

                     Ex: ./scripts/dtc/dtc -O dtb -o partial.dtbo -b 0 -@ partial.dts

Copy the generated PDI and dtbo files into the target package folder 

Limitations

  • The drivers dtbo file extension should be _d.dtbo (Ex: design_drivers_d.dtbo)

  • The current version supports only static builds and does not support dynamic linking

  • Input package/folder should contain only one bitstream/PDI image file and its relevant overlay file.

  • To use the deferred probe functionality both Image DTBO and relevant Drivers DTBO files are mandatory

Known issues

  • Upstream Linux kernel changes from v5.0 and above versions will affect the use of Device Tree (DT) overlays. These changes introduce restrictions that may require users to adjust their device tree overlays.   Fortunately the adjustments required are small. However, even with adjustments users will experience new warnings with no functional impact. With continued upstream improvements, the new warnings are expected to subside in the future. We can ignore these warnings for now.

  • Upstream overlay framework incorrectly updates the configfs ‘status’  to ‘applied’ when there is an error in create_overlay() and libdfx checks the content of the status file to decide if the overlay was a success or not. Hence libdfx dfx_cfg_load() API doesn’t return a proper status in certain use cases.

References

https://github.com/Xilinx/libdfx/tree/master/doc

https://rocketboards.org/foswiki/Documentation/UpstreamV50KernelDeviceTreeChanges