Zynq-7000 AP SoC - 32 Bit DDR Access with ECC Tech Tip
|Description of Revisions
As technology is evolving, dynamic random-access memory (DRAM) device size increases and components on chips get smaller, due to that DRAM chips becoming more affected by electrical or magnetic interference. Lower energy particles are able to change memory cell’s state. These kinds of interferences can cause a single bit of DRAM to spontaneously flip to the opposite state. It can lead system to either crash or to corruption of data.
Several approaches have been developed to deal with unwanted bit-flips. One of the approaches is to calculate an error-correcting code (ECC) for data and store it in DRAM along with data. The most common ECC, a SECDED Hamming code, allow a single bit error to be corrected and double bit errors to be detected.
Zynq-7000 SoC only offers 16 bit ECC on DRAM DDR devices which is only the half-bus (16-bit) data width DDR configuration. Several customers prefer to use a single DDR RAM at 32-bits width for data stored in DRAM, but require ECC for software storage in DRAM.
This Tech-tip intention is provide Zynq based system which supports 32 bit ECC access on DDR devices. It provides hardware IP which can be integrated in Zynq Programmable logic (PL) to support 32 bit ECC for DDR. It also provides detail about porting of Linux software on top it. User can use hardware IP integration and software porting details to get 32 bit ECC support for their system.
|PS + PL
|2 ARM Cortex-A9 667MHZ
- DDR3 533 MHZ
|Xilinx Tools Version
|ZC702 Rev 1.0
|See Appendix A for the descriptions of the files
Refer Appendix A, for more details about the design files structure.
- The ZC702 Evaluation Kit ships with the Xilinx Vivado™ Design Suite Device-locked to the Zynq-7000 XC7Z020 CLG484-1 device.
- A Linux development PC with the ARM GNU tools installed.
- A Linux development PC with the distributed version control system Git installed. For more information, refer to the Xilinx Git wiki and to UG821: Xilinx Zynq-7000 SoC Software Developers Guide.
- A Linux development PC with PetaLinux SDK installed. For more information , refer to Xilinx PetaLinux user manuals.
- Download design files (ZYNQ7000AP_Soc_32bitECCProxy.zip) from link provided above under Implementation details. Extract its content to Proxy home directory referred to as ZYNQ_PROXY_HOME in this document.
For Zynq to support 32 bit ECC on DDR device, a solution called “ECC Proxy” has been developed which adds ECC to selected address range in DDR. ECC Proxy block adds byte level ECC to each AXI transaction. With Byte level ECC, 1 ECC byte is used for each data byte which doubles the amount of read or writes data. Transactions protected by the ECC proxy will have increased latency because effective memory bandwidth is reduced. System level performance will be less affected since the primary usage is to load DDR data to L2 cache. Outside of the increased memory latency to protected regions of DDR, the use of ECC protection should be transparent to the system.
Application Processer unit (APU) of Zynq-7000 SoC normally accesses external DDR memory via L2 cache. All coherent access from APU goes through Snoop control unit (SCU) which in turn connected to L2-cache controller. There is dedicated port from L2- controller to DDR controller for reducing latency in accessing external DDR memory.
When the ECC proxy is added to the Programmable Logic (PL) as shown below, APU transactions to General purpose Master AXI interface (M_AXI_GP1) are redirected to DDR through the ECC proxy, which adds ECC protection bytes to each data bytes. Data movement to/from DDR will be through High Performance Slave (HP) port.
|Figure 1: 32 bit ECC Block Diagram|
For system level details, refer to ZYNQ7000AP_SoC_ProxyDesign.pdf which is provided along with design files under ZYNQ_PROXY_HOME/doc folder. Appendix A provides details about design files.
This tech tip provides a use-case in which DDR memory region is divided in following region
Below is PS address space for above mentioned use-case
- Non ECC region: It is of size 256 MB. Address range from 0x0000_0000 to 0x0FFF_FFFF. User can use this area for Non-ECC data of data/instruction transaction.
- ECC region: This region is of size 384 MB. Address range is 0x8800_0000 to 0x9FFF_FFFF. This region is capable of doing ECC for any kind of data/instruction transaction.
Above figure depicts M_AXI_GP1 address space from 0x8800000 to 0x9FFFFFFF maps to DDR memory space starts from 0x10000000 to 0x3FFFFFF. This ECC protected region is defined with in the physical system address space with a 1MB granularity.
DDR ECC protected region addresses are expanded by 2x of M_AXI_GP1 ECC protected area. That is for each byte written through M_AXI_GP1 interface is expanded because of additional ECC byte written along with the data byte. So half of the complete ECC protected region in DDR is used for ECC protected data and half for byte level ECC storage.
For any Non-ECC access, software has to use address range starts from 0x0000_0000 to 0x0FFF_FFFF. For ECC access, software has to use address ranges starts from 0x88000000 to 0x9FFFFFFF. Software also marks DDR address range from 0x10000000 to 0x3FFFFFFF as non-addressable range.
|Figure 2: PS Address space|
Building FPGA bitstream
This section explains how to generate the FPGA hardware bitstream using the Xilinx Vivado tool (version: 2014.1) and how to export the hardware platform to Xilinx Software Development Kit (SDK) for software application development.
Steps for building the FPGA hardware bitstream
The above step creates a project for Zynq 32 bit DDR proxy.
In the Flow Navigator pane on the left-hand side under Program and Debug, click Generate Bitstream. The bitstream will be generated at $ZYNQ_PROXY_HOME/hardware/vivado/project/zynq_proxy_2014.1.runs/impl_1/ design_1_wrapper.bit
- Launch Vivado tool.
- cd $ZYNQ_PROXY_HOME/hardware/Vivado
- source scripts/project.tcl
In the Export Hardware for SDK window press OK. The SDK hardware platform will be exported to $ZYNQ_PROXY _HOME/hardware/vivado/project/zynq_proxy_2014.1.sdk/SDK/SDK_Export
Note: If the Launch SDK option is checked in the Export Hardware window, SDK will be launched immediately after SDK export has completed. This is not recommended at this point.
|Figure 4: Vivado IPI Block diagram|
Building the Frist Stage Bootloader (FSBL)
This section explains how to import and build the First Stage Boot Loader (FSBL) and the standalone OS based Board Support Package (BSP) from the provided SDK projects.
Note: The provided FSBL project is a customized version of the FSBL SDK project template. The following features have been edited to support Zynq 32 bit Proxy design:
Steps for building the FSBL
- LoadBootImage() function under image_mover.c is modified to support application download to 32 bit proxy address space. In this function partition load address range checking is disabled.
- PcapLoadParition() functionunder pcap.c is modifiedto enable level shifters and deasserting FCLK_RESET0_N reset from PS to PL logic. It is done by calling ps7_post_config() function inside PcapLoadParition() function.
- General Debug prints are enabled by defining FSBL_DEBUG under fsbl_debug.h
- ddr_clean() function is added in main.c. It is used to fill complete DDR memory space with zeros. This function is getting executed after ps7 initialization is done.
- Launch SDK tool.
- In the Workspace Launcher window, click Browse and navigate to $ZYNQ_PROXY_HOME/software/workspace, then click OK. Close the welcome screen.
- Select File > Import, to import the hardware platform (hw_platform) , FSBL (fsbl) and FSBL BSP (fsbl_bsp) into the SDK workspace.It opens Import wizard
|Figure 5: SDK Import Wizard|
- In the Import wizard, expand the General folder, select Existing Projects into Workspace, and click Next.
|Figure 6: SDK Import Wizard Settings> |
- Select “Select archive file” option and Click Browse to navigate to $ZYNQ_PROXY_HOME/software/workspace/workspace.zip.
- Press Finish. Note: Make sure the hw_platform_0, fsbl and fsbl_bsp projects are checked. Press Finish.
The build process will start automatically and builds the BSP first and then the FSBL. The generated Zynq FSBL executable can be found at $ZYNQ_PROXY_HOME/software/workspace/fsbl/Debug/fsbl.elf.
|Figure 7: SDK Import Wizard Settings|
Porting and building the PetaLinux
Installation of PetaLinux SDK
- 2GB RAM (recommended minimum for Xilinx tools)
- Pentium 4 2GHz CPU clock or equivalent.
- 5 GB free HDD space.
- Supported OS:
- RHEL 5 (32-bit or 64-bit)
- RHEL 6 (32-bit or 64-bit)\
- SUSE Enterprise 11 (32-bit or 64-bit)
Download Petalinux 2013.10 SDK software from Xilinx website download section
Assuming all the prerequisites described in the last subsection are satisfied, PetaLinux installation is very straight forward. Without any options, the installer will install as a subdirectory of the current directory. Alternatively, an installation path may be specific. Run the downloaded PetaLinux installer.
PetaLinux will be installed in the petalinux-v2013.10-final directory, directly underneath the working directory of this command. So, if you install the installer into your home directory /home/user, PetaLinux will be installed in
/home/user/petalinux-v2013.10-final. You may move the resulting petalinux-v2013.10-final directory to a preferred location before continuing.
PetaLinux licenses are managed using the same system as all other Xilinx Design Tools. For more details on licensing and setup of license please refer to the "Xilinx Design Tools: Installation and Licensing Guide (UG798)" section
"Obtaining and Managing a License".
- PetaLinux release package downloaded.
- Valid PetaLinux license.
- PetaLinux 2013.10 Board Support Package for Xilinx Zc702 evaluation Kit. Download PetaLinux 2013.10 BSP for ZC702 board from Xilinx website download section.
- Common system packages and libraries are installed on your workstation. The installation process will check for these. See the section Required Tools and Libraries for more details. For detailed information refer PetaLinux installation guide UG976 .
Setup PetaLinux Working Environment
After extracting the package, the remainder of the setup is completed automatically.
bash> source settings.sh
The first time the setup script is sourced, it will perform some post installation tasks to check system dependencies and initialize the Linux kernel source tree.
Below is an example of the output from sourcing the setup script for the first time:
PetaLinux environment set to ’/home/user/petalinux-v2013.10-final
INFO: Finalizing PetaLinux installation
INFO: Checking free disk space
INFO: Checking installed tools
INFO: Checking installed development libraries
INFO: Checking network and other services
The post-install step only occurs once. Subsequent runs of the settings script should be much quicker, and simply output a confirmation message such as that shown below:
PetaLinux environment set to ’/home/user/petalinux-v2013.10-final'
- Go to the PetaLinux root directory by running this command on the command console: cd <path-to-installed-PetaLinux> e.g.: bash> cd /home/user/petalinux-v2013.10-final
- Source the appropriate PetaLinux setup script by running this command on the command console:
Verify PetaLinux Installation
Verify that the PetaLinux working environment has been set:
bash> echo $PETALINUX
Environment variable "$PETALINUX" should point to the path to the installed PetaLinux. Your echo output may be different from this example, depending upon where you installed PetaLinux.
ZC702 Board support Package (BSP) installation
Download PetaLinux 2013.10 BSP for ZC702 board described in subsection 220.127.116.11 prerequisites. It includes installable BSP files, and all necessary design and configuration files, including pre-built and tested hardware and software
images for ZC702 board.
Run petalinux-create command on the command console:
petalinux-create -t project -s <path-to-bsp>
bash> cd $PETALINUX
bash> petalinux-create -t project -s /home/user/petalinux/bsp/Xilinx-ZC702-v2013.10-final.bsp
INFO: Create project:
INFO: * Xilinx-ZC702-14.7
INFO: * Xilinx-ZC702-2013.3
Linux Kernel and u-boot configuration:
bash> cd $PETALINUX/Xilinx-ZC702-2013.3
- Steps for Linux kernel configuration modification for Zynq 32 bit ECC proxy
Select Kernel Base address in Linux Configuration. It opens Kernel base address sub menu
|Figure 8: Kernel Configuration|
|Figure 9 : Kernel configuration|
Note: Linux kernel gets updated with new base address.
- Change Kernel Base address from 0x0 to 0x88000000 and Select OK.
- Select Exit from Linux Configuration.
- Select Yes for “Do you wish to save your new configuration?”
- Open xparameters.h bash> $PETALINUX/Xilinx-ZC702-2013.3/subsystems/linux/hw-description/xparameters.h
Change following definition and save the file.
- #define XILINX_RAM_START 0x88000000
- #define XILINX_RAM_SIZE 0x18000000
- Open system.dts
- Comment out leds_4bits definition.
Note: As bitstream provided with ZC702 BSP includes GPIOs in PL which is connected to onboard LEDs, so devicetree by default provides leds_4bits definition.
In Zynq 32 bit Proxy design case, we don’t include this in PL, so it is mandatory to remove it from device tree. Otherwise Linux gets hangs while booting.
- Change the memory definition with the below contents.
device_type = “memory”;
reg = <0x88000000 0x18000000>;
- Save system.dts file.
- Open Config.mk
Change following definition and save the file
- TEXT_BASE= 0x9f000000
- CONFIG_SYS_TEXT_BASE= 0x9f000000 Note: This change is required to execute u-boot from the 32 bit proxy address space.
Run ’petalinux-build’ in the PetaLinux SDK project directory to build the PetaLinux system image:
bash> cd $PETALINUX/Xilinx-ZC702-2013.3
Note: For more verbose build message use PetaLinux-build -v.
The console shows the compilation progress. e.g.:
INFO: Checking component...
INFO: Generating make files and build Linux
INFO: Generating make files for the subcomponents of Linux
INFO: Building Linux
And the compilation log are stored in build.log in the $(PETALINUX)/build directory.
Generate BOOT image for Zynq
Follow the steps below to generate the SD boot image (BOOT.BIN).
petalinux-package --boot --fsbl <Path to FSBL image> --fpga <Path to FPGA bitstream> --uboot=<Path to uboot image> -o <output file>
petalinux-package command requires bootgen utility to be present in $PATH. Refer Vivado tools installation section for further information.
bash> cd $PETALINUX/Xilinx-ZC702-2013.3/images/linux
bash> petalinux-package --boot --fsbl $ZYNQ_PROXY_HOME/software/workspace/fsbl/Debug/fsbl.elf --fpga $ZYNQ_PROXY_HOME/hardware/vivado/project/zynq_proxy_2014.1.runs/impl_1/design_1_wrapper.bit --uboot
Boot.bin boot image and image.ub kernel image for Zynq get generated at
bash> cd $PETALINUX/Xilinx-ZC702-2013.3/images/linux
Copy following files to SD card.
ZC702 Board Setup:
This section describes about the ZC702 board setup and PetaLinux execution on 32 bit ECC proxy system
- Make sure that the hardware board is set up is done properly. Note:RefertoUG873, Zynq-7000 SoC: Concepts, Tools, and Techniques section 2.1.3 TakeTestDrive!Runningthe“HelloWorld” Applicationfor information about setting up the board.
- For SD card booting mode, Change the SW16 switch setting as shown below
|Figure 10: Switch setting for SD boot mode|
- Go to following folderbash> cd $PETALINUX/Xilinx-ZC702-2013.3/images/linux
- Copy following files to SD card
Follow the following procedure for Application execution
Note: By doing so FSBL start executing and download u-boot at defined location. After downloading, it handoff execution to u-boot.
Prints get display on the serial console.
- Make sure that the hardware board is set up is done properly as discussed in above section.
- Insert a SD-MMC memory card, which contains BOOT.bin and image.ub, into the SD slot on the ZC702 board
- Switch ON ZC702 board.
Hit any key to stop autoboot: 4
- During u-boot execution, the auto boot countdown message appears on serial consol.
Automatic booting from U-Boot stops and au-boot-PetaLinux>prompt appears on the serial terminal.
- Press Enter on serial console
u-boot-PetaLinux> setenv fdt_high 0x90000000
u-boot-PetaLinux> fatload mmc 0 0x93000000 image.ub
u-boot-PetaLinux> bootm 0x93000000
- On serial console u-boot-PetaLinux> prompt do following commands,
Note: After executing u-boot commands, Kernel starts booting from 32 bit Proxy memory region.
root@Xilinx-ZC702-2013.3~# cd /proc
- Waits for Kernel to boot. Provide login and password as (root/root). Once Kernel boots,**root@Xilinx-ZC702-2013.3~#**prompts appears on serial console.
- To check memory map, type following commands
88000000-9fffffff : System RAM
88008000-884f0fd3 : Kernel code
88bc6000-88c0f66f : Kernel data
e0001000-e0001ffe : xuartps
e0002000-e0002fff : /amba@0/ps7-usb@e0002000
e0002000-e0002fff : e0002000.ps7-usb
e000a000-e000afff : e000a000.ps7-gpio
e000d000-e000dfff : e000d000.ps7-qspi
e0100000-e0100fff : mmc0
f8003000-f8003fff : /amba@0/ps7-dma@f8003000
f8007000-f80070ff : xdevcfg
f8007100-f800711f : f8007100.ps7-xadc
- It shows following memory map.
Kernel code section mapped from 0x88008000 to 0x884f0fd3.
Kernel data section mapped from 0x88bc6000 to 0x88c0f66f.
System Memory mapped from 0x88000000 to 0x9fffffff.
Appendix A: File Descriptions in the design directory
Directory Structure is
- ZYNQ7000AP_SoC_ProxyDesign.pdf: It provides detail about system.
- constrs: It contains constraint file for the design.
- scripts: It contains TCL files to create project and system block diagram.
- ip: It contains custom IP used by design.
- hdl: Top level Verilog file for design
- workspace: It contains workspace.zip archive project file for SDK.
- BOOT.bin: Booting image.
- Image.ub: Kernel image.