Zynq Ultrascale MPSOC Linux USB device driver
Introduction
USB 3.0 is the second major revision of the Universal Serial Bus (USB) standard for computer connectivity. USB 3.0 implements a 5.0 Gbit/s raw transfer rate using 8b/10b encoding. USB 2.0 implements the Hi-Speed mode (HS – 480 Mbit/s), while USB 1.1 implements Low Speed (LS) – 1.5 Mbit/s and Full Speed (FS) – 12 Mbit/s). The USB 3.0 Controller used shall provide one 5.0Gbit/s USB channel using the PS internal GT as PHY.Table of Contents
HW/IP Features
The ZynqMP USB 3.0 Controller shall provide one 5.0Gbit/s USB channel using the PS internal GT as PHY.- Two USB 2.0/3.0 controllers
- Compliant with USB 3.0 specs
- Compliant with xHCI standard
- Supports 5.0 Gbps data rate
- Support host, device and OTG modes
- Support On The Go (OTG) 2.0 host/device selection
- Provide simultaneous operation of the USB2.0 and USB3.0 interfaces where applicable
- In host mode, as required by the standard for speed negotiation and switching
- In device mode, statically configured as USB2.0 or USB3.0
- In OTG mode, dynamically switch between host role and device role
- HNP, SRP supported at USB 2.0 speeds in OTG
- DRD supported in USB 3.0 speeds
- 64-bit AXI master port with built-in DMA
- Register programming via AXI and/or APB slave ports
- Power management features: hibernation mode
- Support 44-bit address space
- Supports Link Power Management (LPM) transfers to save power when bus is idle
Features supported in driver
- All the HW/IP features are supported by driver
Missing Features, Known Issues and Limitations
- ZynqMP USB 3.0 controller doesn't support USB 3.0 OTG (On The Go) host/device selection
Important AR links
USB core reset in Linux can cause issues with USB device connected if it was previously powered in U-boot - AR-72376
- USB3.0 device mode does not work when booting through USB boot mode - AR-72409
USB 3.0 standalone device driver does not send STALL response for every unsupported command - AR-71925
- USB 3.0 port hangs after plug-in and plug-out cable in gadget mode - AR-76696
- USB DWC3-core getting the clock error message "Failed to get clk 'ref': -2“ - AR-72764
- USB Remote wake-up feature support (2020 release) - AR-76750
- USB Remote wake-up feature support (2021.1 and later release) - AR-76949
Host Mode
ZCU102 board jumper settings for Host modeJ7 - ON
J113 - 1-2
J110 - 2-3
Please refer the below image for jumper settings on ZCU102 board
USB 3.0 Host mode setup snapshot
Kernel Configuration
Device Drivers------> USB support <*> xHCI HCD (USB 3.0) support <*> USB Mass Storage support <*> DesignWare USB3 DRD Core Support DWC3 Mode Selection (Dual Role mode) ---> (X) Dual Role mode
By enabling the above we need to see the below mentioned Kconfig parameter enabled
CONFIG_USB_DWC3 = y
CONFIG_USB_DWC3_DUAL_ROLE = y
CONFIG_USB_SUPPORT=y
CONFIG_USB_COMMON=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB=y
CONFIG_USB_XHCI_HCD=y
CONFIG_USB_XHCI_PLATFORM=y
Devicetree
usb0: usb0@ff9d0000 { #address-cells = <2>; #size-cells = <2>; status = "disabled"; compatible = "xlnx,zynqmp-dwc3"; reg = <0x0 0xff9d0000 0x0 0x100>; clock-names = "bus_clk", "ref_clk"; power-domains = <&zynqmp_firmware PD_USB_0>; ranges; nvmem-cells = <&soc_revision>; nvmem-cell-names = "soc_revision"; dwc3_0: dwc3@fe200000 { compatible = "snps,dwc3"; status = "disabled"; reg = <0x0 0xfe200000 0x0 0x40000>; interrupt-parent = <&gic>; interrupt-names = "dwc_usb3", "otg", "hiber"; interrupts = <0 65 4>, <0 69 4>, <0 75 4>; #stream-id-cells = <1>; iommus = <&smmu 0x860>; snps,quirk-frame-length-adjustment = <0x20>; snps,refclk_fladj; snps,enable_guctl1_resume_quirk; snps,enable_guctl1_ipd_quirk; snps,xhci-stream-quirk; /* dma-coherent; */ /* snps,enable-hibernation; */ }; }; &usb0 { status = "okay"; }; &dwc3_0 { status = "okay"; dr_mode = "host"; };
Test Procedure
Connect the USB 3.0/2.0 pendrive to USB 3.0 capable board and see it getting detected in /dev/sd<x>.Expected Output
Once Linux boots the below highlighted prints should be visible when we connected the mass storage device84943.023896] usb 2-1: new SuperSpeed USB device number 4 using xhci-hcd [84943.044949] usb 2-1: New USB device found, idVendor=0781, idProduct=5581 [84943.051579] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [84943.058695] usb 2-1: Product: SanDisk Ultra [84943.062858] usb 2-1: Manufacturer: SanDisk [84943.066939] usb 2-1: SerialNumber: A20037A3AD063355 [84943.072653] usb-storage 2-1:1.0: USB Mass Storage device detected [84943.078836] scsi host1: usb-storage 2-1:1.0 [84944.080804] scsi 1:0:0:0: Direct-Access SanDisk SanDisk Ultra PMAP PQ: 0 ANSI: 6 [84944.089531] sd 1:0:0:0: [sda] 30283008 512-byte logical blocks: (15.5 GB/14.4 GiB) [84944.097708] sd 1:0:0:0: [sda] Write Protect is off [84944.103904] sd 1:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [84944.309725] sda: [84944.312888] sd 1:0:0:0: [sda] Attached SCSI removable disk
Performance
When Mass Storage device is connectedDevice used :
USB 3.0 : HP USB 3.0 16 GB pendrive (idVendor=03f0, idProduct=4840)
USB 3.0 - UASP capable drive - Transcend Storjet 128GB
USB 2.0 : Sandisk Cruzer Blade USB 2.0 8 GB Pendrive (idVendor=0781, idProduct=5567)
Using HDPARM tool
Mode | Speed |
USB 3.0 (Super Speed) | 128 MB/sec |
USB 3.0 (UASP capable) | 327.10 MB/sec |
USB 2.0 (High Speed) | 25.48 MB/sec |
Note : Above results may vary from device to device
When Ethernet device is connected
Device used :
USB 3.0/2.0 : Xilinx ZynqMP RNDIS/Ethernet Gadget
Using Iperf tool (TCP)
Mode | Speed |
USB 3.0 (Super Speed) | 761.6 Mbits/sec |
USB 2.0 (High Speed) | 467.2 Mbits/sec |
Using Iperf tool (UDP)
Mode | Speed |
USB 3.0 (Super Speed) | 761.6 Mbits/sec |
USB 2.0 (High Speed) | 467.2 Mbits/sec |
Note : The above results are taken with LPM mode disabled
Peripheral Mode
ZCU102 jumper settings for Peripheral modeJ7 - OFF
J113 - 1-2
J110 - 1-2
J109- No jumper
Please refer the below image for jumper settings required for peripheral mode on ZCU102 board
USB 3.0 peripheral mode setup snapshot
This document only explains the USB 3.0 peripheral mode configurations for MASS STORAGE gadget or ETHERNET Gadget profiles.
Mass Storage
Kernel Configuration
Device Drivers--------> USB support <*> USB Gadget Support <M> USB Gadget Drivers <M> Gadget Filesystem <M> Mass Storage Gadget File systems ---> Pseudo filesystems ---> {M} Userspace-driven configuration filesystem
By enabling the above we need to see the below mentioned Kconfig parameter enabled
CONFIG_USB_DWC3 = y
CONFIG_USB_DWC3_DUAL_ROLE = y
CONFIG_USB_SUPPORT=y
CONFIG_USB_COMMON=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB=y
CONFIG_USB_XHCI_HCD=y
CONFIG_USB_XHCI_PLATFORM=y
CONFIG_CONFIGFS_FS=m
CONFIG_USB_GADGET=y
CONFIG_USB_LIBCOMPOSITE=m
CONFIG_USB_F_MASS_STORAGE=m
CONFIG_USB_CONFIGFS=m
CONFIG_USB_CONFIGFS_MASS_STORAGE=y
Please find the required .ko files in the below mentioned paths:
linux-xlnx/fs/configfs/configfs.ko
linux-xlnx/drivers/usb/gadget/libcomposite.ko
linux-xlnx/drivers/usb/gadget/legacy/gadgetfs.ko
linux-xlnx/drivers/usb/gadget/function/usb_f_mass_storage.ko
linux-xlnx/drivers/usb/gadget/legacy/g_mass_storage.ko
Devicetree
usb0: usb0@ff9d0000 { #address-cells = <2>; #size-cells = <2>; status = "disabled"; compatible = "xlnx,zynqmp-dwc3"; reg = <0x0 0xff9d0000 0x0 0x100>; clock-names = "bus_clk", "ref_clk"; power-domains = <&zynqmp_firmware PD_USB_0>; ranges; nvmem-cells = <&soc_revision>; nvmem-cell-names = "soc_revision"; dwc3_0: dwc3@fe200000 { compatible = "snps,dwc3"; status = "disabled"; reg = <0x0 0xfe200000 0x0 0x40000>; interrupt-parent = <&gic>; interrupt-names = "dwc_usb3", "otg", "hiber"; interrupts = <0 65 4>, <0 69 4>, <0 75 4>; #stream-id-cells = <1>; iommus = <&smmu 0x860>; snps,quirk-frame-length-adjustment = <0x20>; snps,refclk_fladj; snps,enable_guctl1_resume_quirk; snps,enable_guctl1_ipd_quirk; snps,xhci-stream-quirk; /* dma-coherent; */ /* snps,enable-hibernation; */ }; }; &usb0 { status = "okay"; }; &dwc3_0 { status = "okay"; dr_mode = "peripheral"; };
Performance
Using HDPARM tool (Linux Host)Link Power Management (LPM) mode Enable
Mode | Speed |
USB 3.0 (Super Speed) | 127.18 MB/sec |
USB 2.0 (High Speed) | 38.82 MB/sec |
Link Power Management (LPM) mode Disable
Mode | Speed |
USB 3.0 (Super Speed) | 237 MB/sec |
USB 2.0 (High Speed) | 38.47 MB/sec |
Testing Procedure
Please use the below settings for configuring USB as MASS STORAGE profile in device mode:insmod configfs.ko insmod libcomposite.ko insmod usb_f_mass_storage.ko insmod gadgetfs.ko dd if=/dev/zero of=/tmp/mydev count=256 bs=1M insmod g_mass_storage.ko file=/tmp/mydev removable=1 stall=1 iSerialNumber=7ABC7ABC7ABC7ABC7ABC7ABC
Testing the mass storage functionality by connecting the board to host using the below steps:
- Connect the cable from board to windows host machine
- Format the mass storage device that got detected in windows
- The windows screen shot when we the device gets detected should be as below
- The board will be detected as mass storage drive with size 256 MB.
- Copy some files into the mass storage drive , remove the cable and connect it again. We should be able to see the files that we copied into the drive.
Ethernet Gadget
Kernel Configuration
For the xilinx-v2016.2 and prior tags, apply the following patch.USB support-----> <*> USB Gadget Support <M> USB Gadget Drivers <M> Gadget Filesystem <M> Ethernet Gadget (with CDC Ethernet support) [*] RNDIS support File systems ---> Pseudo filesystems ---> {M} Userspace-driven configuration filesystem
CONFIG_USB_DWC3 = y
CONFIG_USB_DWC3_DUAL_ROLE = y
CONFIG_USB_SUPPORT=y
CONFIG_USB_COMMON=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB=y
CONFIG_USB_XHCI_HCD=y
CONFIG_USB_XHCI_PLATFORM=y
CONFIG_CONFIGFS_FS=m
CONFIG_USB_GADGET=y
CONFIG_USB_LIBCOMPOSITE=m
CONFIG_USB_ETH=m
CONFIG_USB_ETH_RNDIS=y
Please find the required .ko files in the below mentioned paths:
linux-xlnx/fs/configfs/configfs.ko
linux-xlnx/drivers/usb/gadget/libcomposite.ko
linux-xlnx/drivers/usb/gadget/legacy/gadgetfs.ko
linux-xlnx/drivers/usb/gadget/function/u_ether.ko
linux-xlnx/drivers/usb/gadget/function/usb_f_rndis.ko
Devicetree
usb0: usb0@ff9d0000 { #address-cells = <2>; #size-cells = <2>; status = "disabled"; compatible = "xlnx,zynqmp-dwc3"; reg = <0x0 0xff9d0000 0x0 0x100>; clock-names = "bus_clk", "ref_clk"; power-domains = <&zynqmp_firmware PD_USB_0>; ranges; nvmem-cells = <&soc_revision>; nvmem-cell-names = "soc_revision"; dwc3_0: dwc3@fe200000 { compatible = "snps,dwc3"; status = "disabled"; reg = <0x0 0xfe200000 0x0 0x40000>; interrupt-parent = <&gic>; interrupt-names = "dwc_usb3", "otg", "hiber"; interrupts = <0 65 4>, <0 69 4>, <0 75 4>; #stream-id-cells = <1>; iommus = <&smmu 0x860>; snps,quirk-frame-length-adjustment = <0x20>; snps,refclk_fladj; snps,enable_guctl1_resume_quirk; snps,enable_guctl1_ipd_quirk; snps,xhci-stream-quirk; /* dma-coherent; */ /* snps,enable-hibernation; */ }; }; &usb0 { status = "okay"; }; &dwc3_0 { status = "okay"; dr_mode = "peripheral"; };
Performance
Iperf tool (TCP)Mode | Speed |
USB 3.0 (Super Speed) | 761.6 Mbits/sec |
USB 2.0 (High Speed) | 467.2 Mbits/sec |
Iperf tool (UDP)
Mode | Speed |
USB 3.0 (Super Speed) | 761.6 Mbits/sec |
USB 2.0 (High Speed) | 467.2 Mbits/sec |
Testing Procedure
Please use the below settings for configuring USB as ETHERNET gadget profile in device mode:insmod configfs.ko insmod libcomposite.ko insmod u_ether.ko insmod usb_f_rndis.ko mount -t configfs none /sys/kernel/config cd /sys/kernel/config/usb_gadget mkdir g1 cd g1 echo "64" > bMaxPacketSize0 echo "0x200" > bcdUSB echo "0x100" > bcdDevice echo "0x03FD" > idVendor echo "0x0502" > idProduct mkdir configs/c1.1 mkdir functions/rndis.rn0 ln -s functions/rndis.rn0/ configs/c1.1/ echo "fe200000.dwc3" > UDC ifconfig usb0 10.10.70.1 ifconfig usb0 up
Testing the ethernet gadget by connecting to linux/windows machine:
- Connect the cable from board to windows/linux host machine
- Set the ipaddress on device side
systest# ifconfig usb0 192.168.1.200 - Set the ipaddress on host side
- ping host address from device side and the result should be ass below mentioned
ping <host address>
Expected Output:
USB Attached SCSI Protocol(UASP)
LimitationTCM MODULE USED FOR UASP does not support ATA_12 commands. so user needs to change VID and PID in file drivers/usb/gadget/legacy/tcm_usb_gadget.c as below
#define UAS_VENDOR_ID 0x0bc2
#define UAS_PRODUCT_ID 0xa013
Kernel Configuration
Device Drivers ---> <*> Generic Target Core Mod (TCM) and ConfigFS Infrastructure ---> --- Generic Target Core Mod (TCM) and ConfigFS Infrastructure <*> TCM/IBLOCK Subsystem Plugin for Linux/BLOCK <*> TCM/FILEIO Subsystem Plugin for Linux/VFS <*> TCM/pSCSI Subsystem Plugin for Linux/SCSI <*> TCM/USER Subsystem Plugin for Linux <*> TCM Virtual SAS target and Linux/SCSI LDD fabric loopback module <*> Linux-iSCSI.org iSCSI Target Mode Stack [*] USB support ---> <*> USB Gadget Support <*> USB functions configurable through configfs [*] USB Gadget Target Fabric <M> USB Gadget Target Fabric Module
KCONFIG options for UASP profile
CONFIG_USB_DWC3 = y
CONFIG_USB_DWC3_DUAL_ROLE = y
CONFIG_USB_SUPPORT=y
CONFIG_USB_COMMON=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB=y
CONFIG_USB_XHCI_HCD=y
CONFIG_USB_XHCI_PLATFORM=y
CONFIG_CONFIGFS_FS=m
CONFIG_USB_GADGET=y
CONFIG_USB_LIBCOMPOSITE=m
CONFIG_USB_CONFIGFS=m
CONFIG_BLK_DEV_INTEGRITY=y
CONFIG_TARGET_CORE=y
CONFIG_TCM_IBLOCK=y
CONFIG_TCM_FILEIO=y
CONFIG_TCM_PSCSI=y
CONFIG_TCM_USER2=y
CONFIG_LOOPBACK_TARGET=y
CONFIG_ISCSI_TARGET=y
CONFIG_USB_F_TCM=y
CONFIG_USB_CONFIGFS_F_TCM=y
CONFIG_USB_GADGET_TARGET=m
CONFIG_CRC_T10DIF=y
Devicetree
Example Device tree node:&dwc3_0 { status = "okay"; dr_mode = "peripheral"; }
linux-xlnx/fs/configfs/configfs.ko
linux-xlnx/drivers/usb/gadget/libcomposite.ko
linux-xlnx/drivers/usb/gadget/legacy/gadgetfs.ko
linux-xlnx/drivers/usb/gadget/legacy/tcm_usb_gadget.ko
Performance
HDPARM Tool (UBUNTU Host)Link Power Management (LPM) mode | Result |
Enabled | 367.77 MB/sec |
Disabled | 381.42 MB/sec |
Test procedure
Connect the USB 3.0 cable to linux PC and perform the steps given below on the device sideInstall module
modprobe tcm_usb_gadget
Mount the configfs
mount a -t configfs /sys/kernel/config
Create a memory that will be assigned to gadget (here creating memory of 750MB)
mkdir /root dd if=/dev/zero of=/root/file.bin bs=10M count=75
Format the memory that just has been created
mkdosfs -v /root/file.bin -n ramfs
Assign memory to gadget module
mkdir -p /sys/kernel/config/target/core/fileio_0/fileio echo "fd_dev_name=/root/file.bin,fd_dev_size=734003200" > /sys/kernel/config/target/core/fileio_0/fileio/control echo 1 > /sys/kernel/config/target/core/fileio_0/fileio/enable mkdir -p /sys/kernel/config/target/usb_gadget/naa.6001405c3214b06a/tpgt_1 mkdir /sys/kernel/config/target/usb_gadget/naa.6001405c3214b06a/tpgt_1/lun/lun_0 echo naa.6001405c3214b06a > /sys/kernel/config/target/usb_gadget/naa.6001405c3214b06a/tpgt_1/nexus ln -s /sys/kernel/config/target/core/fileio_0/fileio /sys/kernel/config/target/usb_gadget/naa.6001405c3214b06a/tpgt_1/lun/lun_0/virtual_scsi_port
Change max busrt to get maximum performance
echo 15 > /sys/kernel/config/target/usb_gadget/naa.6001405c3214b06a/tpgt_1/maxburst
Connect device to Host
echo 1 > /sys/kernel/config/target/usb_gadget/naa.6001405c3214b06a/tpgt_1/enable
Ubuntu
On performing the steps given above, below prints should be shown on host side on typing "dmesg"
Do data transfer on HOST(UBUNTU) side
Windows
Note: UASP is supported from windows 8
On windows need to follow below steps in order to mount device
1. open manage by right click on Desktop (screen shot as below)
2. open disk management and format the device (screen shot as below)
3. It will mount the device as drive
4. Do data transfer
OTG Mode
Using the correct cables is the key to OTG operation. Testing was done using two cables joined together to create an OTG cable. An OTG cable has a micro A connector on one end and a micro B connector on the other end. The micro A connector is the host side of the cable and the micro B connector is the device side by default.Hardware Setup
Boards: ZynqMP ZCU102 (2 boards back to back connected)
Host Machine: Linux Machine with USB 3.0 ports
Kernel Configuration
Linux Kernel image with xHCI Host enabled (static) and Mass Storage Gadget created as dynamic module- Boot image for initial FSBL sequence
- Device tree with DWC USB OTG mode enabled
- CONFIG_PM=y
- CONFIG_USB_OTG=y
- CONFIG_USB_OTG_FSM=y
- CONFIG_USB_DWC3_OTG=y
Test Procedure
- When the Linux kernel boots up, give the below commands:
$ modprobe g_mass_storage file=/tmp/mydev removable=1 stall=1 iSerialNumber=7ABC7ABC7ABC7ABC7ABC7ABC
Peripheral Mode
- Connect board to any Host PC via USB Std-A-Male to Micro-B-Male cable.
- It should be detected as a Mass Storage Device
- Do some data transfers
Host Mode
- Connect any pen-drive with the board via USB Micro-A-Male to Std-A-Female type connector
- Check that the pen-drive is detected properly
- Do some data transfers.
HNP & SRP
- When two boards (OTG enabled) are connected back to back, OTG 2.0 supports HNP and SRP
- To start HNP from the Device, issue below command
- $ echo 1 > /sys/devices/platform/amba/ff9d0000.usb0/fe200000.dwc3/hnp
- Please check the role reversal on both the boards
- To start SRP from the Device, issue below command
- $ echo 1 > /sys/devices/platform/amba/ff9d0000.usb0/fe200000.dwc3/srp
Note : Both Host and Peripheral mode should work without resetting/rebooting the board.
Mainline Status
The current driver available in the Xilinx Linux git is in sync with the open-source kernel driver except for the following:- USB Hibernation feature,
- USB Remote wakeup feature,
- USB OTG feature,
- Suspend-Resume feature.
configured to anyone among 26MHz, 52 MHz, and 100MHz. On the ZCU102 board, this is factory programmed to 26MHz.
Phy Settings
This USB controller on Zynq UltraScale+ MPSoC is connected to High Speed GTR through PIPE3 interface. This phy requires a reference clock to operate with USB 3.0, which can beconfigured to anyone among 26MHz, 52 MHz, and 100MHz. On the ZCU102 board, this is factory programmed to 26MHz.
Change log
2024.1
- 6.6 kernel upgrade
- a66247a1aa39 dwc3: core: For ZU+ enable GUCTL1 bit 9 for correcting Inter Packet Delay(IPD)
- d21f9d46eaba Revert "dwc3: core: Enable GUCTL1 chicken bit 9 for correcting Inter Packet Delay(IPD)"
- e06cd16aaa5e usb: dwc3: define dummy dwc3_gadget_*_hibernation APIs for DWC3_HOST mode
- 1b9e0fd55158 usb: dwc3: define dummy dwc3_host_wakeup_* APIs for DWC3_GADGET mode
- c4c8377c917f usb: dwc3: gadget: switch to 'wakeup' interrupt name for hibernation support
- f462b9499016 usb: dwc3: Remove not required version.h header
2023.2
- None
2023.1
- kernel upgrade to 6.1
- Added reset-controller support for versal platform
- Added power-domains property support
2022.2
- None
2022.1
- kernel upgrade to 5.15
2021.2
- None
2021.1
- Created a new Xilinx DWC3 glue driver for better management of Xilinx specific changes
- Kernel upgrade to 5.10
2020.2
- None
2020.1
- kernel upgrade to 5.4
2019.2
- Add support for disabling U1 and U2 entries
- Add support for masking phy reset signal
- Add wakeup capability support for versal usb controller
- Add D0/D3 entry/exit support for versal
2019.1
- Start timer per request instead of per endpoint
- Correct the logic for checking TRB full in __dwc3_prepare_one_trb()
- Check for IOC/LST bit in both event->status and TRB->ctrl fields
- Check MISSED ISOC bit only for ISOC endpoints
- Don't kick transfer if LST or SHORT bits are set
- make otg driver work along with drd driver
- mask host/device soft reset from affecting the phy
- Add user event for ending hnp session
- Add DRD support for versal
- route lpd usb traffic through CCI path for making SMMU work
Fixed BUS Expiries for Iscoronous transfers when using usb video class
Related Links
© Copyright 2019 - 2022 Xilinx Inc. Privacy Policy