Versions Compared

Old Version 74

changes.mady.by.user harinik

Saved on

compared with

New Version 75

changes.mady.by.user harinik

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Axi Ethernet Linux driver for Microblaze, Zynq, Zynq Ultrascale+ MPSoC and Versal


Introduction

Table of Contents

Table of Contents
This page gives an overview of Axi Ethernet Linux driver which is available as part of the Linux distribution.

...

  • Hardened Ethernet IP block on Versal.
  • Multi rate Ethernet MAC supporting speeds from 10G to 100G.
    • The driver supports 25GE and 10GE with 1 to 4 lanes.
  • Hardened IP (to be used with Soft DMA and logic for driver subsystems)
  • High performance, low latency.
  • Low data path latency
  • User-side AXI4-Stream interface for data
  • AXI4-Lite register interface
  • Detailed statistics gathering
  • IEEE1588 support

...

          
  • The driver supports channel observer feature through sysfs. This custom feature is useful in multi-core (Observer) system where MCDMA is a shared resource for all cores. MCDMA IP supports a maximum of six cores and 16 Channels can be distributed across each core as a static configuration. The Channel Observer is available for each group and provides the status about the channels in a group being serviced.
  • The driver supports per channel weight configuration through sysfs. This custom feature specifies the channel weight i.e number of packets to be sent in one iteration.

  • The driver supports Linux multiqueue networking. It uses the alloc_etherdev_mq() function to allocate the subqueues for the device.

    The userspace command 'tc,' part of the iproute2 package, is used to configure qdiscs. To add the MULTIQ qdisc assuming the device is called eth0, run the following command: 

    # tc qdisc add dev eth0 root handle 1: multiq

    The qdisc will allocate the number of bands to equal the number of queues that the device reports, and bring the qdisc online.

    Assuming eth0 has 4 Tx queues, the band mapping would look like:

    band 0 => queue 0
    band 1 => queue 1
    band 2 => queue 2
    band 3 => queue 3

...

NOTE- There is ~10% drop (compared to 2019.2) in performance for 1500 MTU.
The drop is due to enable CONFIG_OPTIMIZE_INLINING forcibly” commit in linux kernel.

Kernel and networking stack is full of inline functions and it could be some unoptimized
inline function (could also be dependent on gcc version) leading to a performance drop.

The performance drop is observed on GEM and Xilinx Axi Ethernet MAC’s on Zynq

The plan is to document the performance drop on zynq and initiate the discussion with
the mainline community so that it is analyzed by respective kernel maintainers.



TCP (Mbps)UDP (Mbps)
MTUTXCPU(%)RXCPU(%)TXCPU(%)RXCPU(%)
1500

740

67.53

537

89.39

453
52.86

456

88.72

8192

977

60.69

732

50.26

743

36.10

643

50.32


...

Traditionally microblaze designs are not targeted for high performance applications so only functional sanity is done.

10G Ethernet with AXIMCDMA

Kernel version: 5.10

ZynqMP
Board: ZCU102 board (production silicon) + SFP Module


TCP (Gbps)UDP (Gbps)
MTUTXCPU(%)RXCPU(%)TXCPU(%)RXCPU(%)
15002.2950.911.7376064.98713.0399.931.6570.04
90005.052.353.5653.96.5165.094.6354.28
NOTE: In this design 1588 is not enabled.

Setup Details
Host setup: Dell System Precision Tower 7910 (0619)
Iperf: iperf 3-CURRENT (cJSON 1.5.2)
OS : Linux 3.13.0-147-generic #196-Ubuntu SMP Wed May 2 15:51:34 UTC 2018 x86_64
NIC (10G Solarflare's SFN6322F Dual-Port 10GbE SFP+ Adapter) : Default

Performance benchmarking

Pre-requisites:
  • Set Ethernet MCDMA TX interrupt affinity to core-1
root@10g-mcdma-no1588-build:~# echo 2 > /proc/irq/xx/smp_affinity
  • Run iperf servers on ZynqMP (core2 and core3)
root@10g-mcdma-no1588-build:~# taskset -c 2 iperf3 -s -p 5101 &
root@10g-mcdma-no1588-build:~# taskset -c 3 iperf3 -s -p 5102 &
  • CPU Utilization reporting
root@10g-mcdma-no1588-build:~# ./mpstat -P ALL 1 50
  • Run iperf servers on the remote host
server:~# iperf3 -s -p 5101 & ; iperf3 -s -p 5102 & ; iperf3 -s -p 5103 & ; iperf3 -s -p 5104 &

Steps:

...

  • Set the CROSS_COMPILE environment variable arm toolchain
  • Install the kernel headers

          https://www.kernel.org/doc/Documentation/kbuild/headers_install.txt

  • Include the headers path in makefile

          INC = -I/proj/epdsw/punnaiah/git/test/ethernet/1588/header/include
          CFLAGS = -Wall $(VER) $(incdefs) $(DEBUG) $(INC) $(EXTRA_CFLAGS)

  • run make

Execution steps

In order to perform master-slave sync, run the following:

Master (linux server) : ptp4l -i < interface name> -m
Slave (xilinx board) :   ptp4l -i <interface name> -m -s

NOTE: If intended before synchronization phc2sys -s <devicename> -w & can be run to synchronize the system clock to a PTP hardware clock.

Synchronization is stabilized in a few secs.


Mainline status

The current Axi Ethernet driver is currently in sync with mainline except for the following

...

33ebfdb net: xilinx: axiethernet: Fix crash in ifconfig down
f5b9e58 net: xilinx: axiethernet: Fix axiethernet register description
e491e78 net: xilinx: axiethernet: Check for queue full in transmit path
0ba2b93 net: xilinx: axiethernet: Fix code checker warnings
d4c6c09 net: xilinx: axiethernet: Use %pa format specifier for phys_addr_t type
270968c net: xilinx: axiethernet: Add 64-bit support
d139077 net: xilinx: axiethernet: Extend clocking support
fdce589 net: xilinx: axiethernet: Fix kernel crash on MII ioctl
3f2d6cd net: xilinx: axiethernet: use channel-id for mcdma interrupt names
aaad9c0 net: xilinx: axiethernet: Fix netconsole implementation

2018.3
  • Sync kconfig description.
  • In axienet_skb_tstsmp() failure return TX_BUSY.
  • Add error output on DMA allocation failed.
  • Fix memory leak in axienet bd_free().
  • Refactor and split axidma and mcdma programming in separate sources.
  • Fix dma name buffer size and skb_free in xmit.
  • Format XXV error output.
  • Fix compiler warnings.

...