Table of Contents

Introduction

Axi Ethernet Linux driver for Microblaze, Zynq, Zynq Ultrascale+ MPSoC and VersalIntroduction

Table of Contents

...

This page gives an overview of Axi Ethernet Linux driver which is available as part of the Linux distribution.

Paths, files, links and documentation on this page are given relative to the Linux kernel source tree.

HW IP features

AXI 1G/2.5G Ethernet Subsystem (PG138)

...

• Hardened Ethernet IP block on Versal.

• Multi rate Ethernet MAC supporting speeds from 10G to 100G.

  • The driver supports 25GE and 10GE with 1 to 4 lanes.

• Hardened IP (to be used with Soft DMA and logic for driver subsystems)

• High performance, low latency.

• Low data path latency

• User-side AXI4-Stream interface for data

• AXI4-Lite register interface

• Detailed statistics gathering

• IEEE1588 support

...

• Hardened Ethernet IP block on Versal Premium and Versal HBM (to be used with Soft DMA and logic for driver subsystems).

• 1 x 400GE, 3 x 200GE, 6 x 100GE, or combinations of 100 Gbps, 200 Gbps, and 400 Gbps totaling up to 600 Gbps support.

  • Driver supports 100G with single lane (1 x 100GE).

• Supports 100GBASE-R, 200GBASE-R and 400GBASE-R phy interfaces.

• 40-channel time-sliced MAC capable of 600 Gbps operation.

  • Channelized option for time-sliced applications.

  • Up to 40 channels supported.

  • User-defined bandwidth allocation granularity.

• Pause frame processing, including priority-based flow control.

• Optional built-in RS-FEC functionality.

• IEEE1588 support.

• User-side segmented AXI4-Stream interface.

• AXI4-Lite register interface.

• Detailed statistics gathering.
  

...

• Multiple TX and RX channel in MCDMA have common configuration and reset registers and hence cannot be used independently by multiple MACs. For ex., if XXV Ethernet instance 1 uses channel 0-4 of MCDMA and then XXV Ethernet instance 2 uses channels 5-15, then resets during driver initialization and error management effect all channels and both instance need to use common registers. Due to this limitation, multiple MACs cannot be used with a single MCDMA.

• Ethtool or link/speed information support is not available for MRMAC; this limitation is due to phylink framework and non-presence of an external PHY or PCS. However, there is a plan to add this support in an upcoming release.

• RSFEC configuration is not supported or tested in the driver.

• AXI Ethernet or any other high speed Ethernet does not have any module support with MCDMA. There are some dependencies.

• XXV Block lock register access in the IP fails (kernel crash with SError) when the reference GT clock is not stable (and link is not up). Since there is no indication to the SW on the clock stability, reads to this register cannot be avoided beforehand. If the clock is stable and link is up, there is no effect from this known issue. IP workaround with corresponding driver and DT changes are proposed and documented. Please refer “Important AR link” section below.

• DCMAC driver is validated on Versal premium platform.

• DCMAC driver supports only 1x100G configuration.

• IEEE 1588 support for DCMAC is not there. It will be added in future release.

• 2.5G SGMII phy mode support for AXI 1G/2.5G ethernet is not there.

• "max-speed" is required DT property for AXI 1G/2.5G as it is needed to distinguish 1G and 2.5G speed MAC types. Without this property, driver probe will fail.

Missing Features and Known Issues/Limitations in SDT

...

NOTE- There is ~10% drop (compared to 2019.2) in performance for 1500 MTU.
The drop is due to enable CONFIG_OPTIMIZE_INLINING forcibly” commit in linux kernel.

Kernel and networking stack is full of inline functions and it could be some unoptimized
inline function (could also be dependent on gcc version) leading to a performance drop.

The performance drop is observed on GEM and Xilinx Axi Ethernet MAC’s on Zynq

The plan is to document the performance drop on zynq and initiate the discussion with
the mainline community so that it is analyzed by respective kernel maintainers.

...

NOTE: In this design 1588 is not enabled.

Setup Details
Host setup: Dell System Precision Tower 7910 (0619)
Iperf: iperf 3-CURRENT (cJSON 1.5.2)
OS : Ubuntu 22.04.2 LTS (Linux kernel 5.15.0-87-generic)
NIC (10G Solarflare's SFN6322F Dual-Port 10GbE SFP+ Adapter) : Default

Performance benchmarking

NOTE: Better performance numbers are observed with linux v6.6 and flag -P 2 in iperf command.
Pre-requisites:

• Set Ethernet MCDMA TX interrupt affinity to core-1

root@10g-mcdma-no1588-build:~# echo 2 > /proc/irq/xx/smp_affinity

• Run iperf servers on ZynqMP (core2 and core3)

root@10g-mcdma-no1588-build:~# taskset -c 2 iperf3 -s -p 5101 &
root@10g-mcdma-no1588-build:~# taskset -c 3 iperf3 -s -p 5102 &

• CPU Utilization reporting

root@10g-mcdma-no1588-build:~# ./mpstat -P ALL 1 50

• Run iperf servers on the remote host

...

• Set the CROSS_COMPILE environment variable arm toolchain

• Install the kernel headers

          https://www.kernel.org/doc/Documentation/kbuild/headers_install.txt

• Include the headers path in makefile

          INC = -I/proj/epdsw/punnaiah/git/test/ethernet/1588/header/include
          CFLAGS = -Wall $(VER) $(incdefs) $(DEBUG) $(INC) $(EXTRA_CFLAGS)

• run make

Execution steps

In order to perform master-slave sync, run the following:

Master (linux server) : ptp4l -i < interface name> -m
Slave (xilinx board) :   ptp4l -i <interface name> -m -s

NOTE: If intended before synchronization phc2sys -s <devicename> -w & can be run to synchronize the system clock to a PTP hardware clock.

...

Synchronization is stabilized in a few secs.

...

Mainline status

The current Axi Ethernet driver is not in sync with mainline. The following components need to be upstreamed:

...

https://github.com/Xilinx/linux-xlnx/commits/xilinx-v2024.1/drivers/net/ethernet/xilinx

2023.2

• Acquire ptp device information dynamically

• Add switchable 1/10/25G MAC support

• Implement work queue to enable/disable link training.

Commits

https://github.com/Xilinx/linux-xlnx/commits/xilinx-v2023.2/drivers/net/ethernet/xilinx

...

33ebfdb net: xilinx: axiethernet: Fix crash in ifconfig down
f5b9e58 net: xilinx: axiethernet: Fix axiethernet register description
e491e78 net: xilinx: axiethernet: Check for queue full in transmit path
0ba2b93 net: xilinx: axiethernet: Fix code checker warnings
d4c6c09 net: xilinx: axiethernet: Use %pa format specifier for phys_addr_t type
270968c net: xilinx: axiethernet: Add 64-bit support
d139077 net: xilinx: axiethernet: Extend clocking support
fdce589 net: xilinx: axiethernet: Fix kernel crash on MII ioctl
3f2d6cd net: xilinx: axiethernet: use channel-id for mcdma interrupt names
aaad9c0 net: xilinx: axiethernet: Fix netconsole implementation

2018.3

• Sync kconfig description.

• In axienet_skb_tstsmp() failure return TX_BUSY.

• Add error output on DMA allocation failed.

• Fix memory leak in axienet bd_free().

• Refactor and split axidma and mcdma programming in separate sources.

• Fix dma name buffer size and skb_free in xmit.

• Format XXV error output.

• Fix compiler warnings.

...