Documentation Criteria

Document History

Summary

Implementation

1. Introduction to SPEC CPU 2000

1.1 Objectives and Workloads

Objectives:

Workloads:

Workloads Description:

2. Building and Compilation of Benchmark with SPEC tools

2.1 Runspec and its Build Environment

3. Setting up the Zynq-7000 Board and Configuration Disclosure

3.1 Build Procedure:

4. Installation Procedure:

4.1 Verification of Successful Build/Installation:

5. Running SPEC CPU 2000 on Zynq-7000

6. Obtaining the SPEC CPU2000 Results

7. Interpreting the SPEC CPU2000 Results

8. Conclusion

9. FAQ

Here are the criteria as a reminder. When publishing this can be removed.

Assumptions

1. It Is Assumed that the user of tech tip knows and understands terminologies associated with SPEC benchmarking.

2. No explicit Information regarding compilation process of SPEC process is provided. It is assumed that user is aware of cross compilation, building of SPEC binaries.

3. SPEC sources are not provided with this Tech Tip, Its assumed that user has the SPEC sources or is going to purchase it from SPEC consortium

4. User is requested to get any of the standard SPEC CPU 2000 related details from SPEC CPU 2000 Official website

The main purpose of this tech tip is to provide a brief introduction to SPEC CPU 2000 and update user with the process followed to build SPEC CPU 2000, benchmark it, and finally report the results generated for Zynq 7000 AP SOC. Some of the extracts are directly taken from the SPEC website as the process to be followed is standardized across platforms.

Files Provided
Config file	SPEC CPU 2000 Config file used for Zynq
Makefile Patch Open	Patch for Makefile used to compile the SPEC binaries
Ram disk Image Open	Ram disk images to resolve library dependencies

SPEC CPU 2000 is an industry standard benchmark which provides a comparative measure of integer and/or floating point compute intensive performance. SPEC CPU 2000 focuses on compute intensive performance, which means these benchmarks measure the performance of the

1. Processor (CPU)
2. Memory architecture (DDR)
3. The compilers.

The system performance of a device is not only limited to CPU performance but also the memory architecture of the device and the compilers are equally significant while evaluating the system performance.

Best results require steady performance across all programs. In order to arrive at predictable and useful conclusions on system performance, SPEC CPU2000 provides benchmarks in the form of source code, which are compiled in accordance to pre-defined rules. It is expected that a user obtains a copy of these suites from SPEC’s website, install the hardware, compilers, and other software described and reproduce the claimed performance with a small variation of 1 to 2% on each run. Thus serving as common reference and most importantly, can be considered as part of an evaluation process of the system under consideration.

Benchmarks are provided in two suites: An integer suite, known as CINT2000, and a floating point suite, known as CFP2000. There are several different ways to measure system performance. One way is to measure how fast the device completes a single task; this is called speed measure. Another way is to measure how many tasks a device can accomplish in a certain amount of time; this is called throughput, capacity or rate measure.

The SPEC speed metrics (e.g., SPECint2000) are used for comparing the ability of a device to complete a single task. Following are the well-known ways to evaluate Speed (1 copy) metrics
1) SPECint2000: The geometric mean of twelve normalized ratios (one for each integer benchmark) when compiled with aggressive optimization for each benchmark.
2) SPECint_base2000: The geometric mean of twelve normalized ratios when compiled with conservative optimization for each benchmark

The SPEC rate metrics (e.g., SPECint_rate2000) measure the throughput of a machine carrying out a number of tasks. Following are the well-known ways to evaluate Rate (2 copy) metrics
1) SPECint_rate2000: The geometric mean of twelve normalized throughput ratios when compiled with aggressive optimization for each benchmark.
2) SPECint_rate_base2000: The geometric mean of twelve normalized throughput ratios when compiled with conservative optimization for each benchmark.

The SPEC CPU2000 workload run times are normalized against the run times of a reference machine.
*Please note that only “conservative optimizations” have been evaluated for the current system under consideration.

CINT2000 contains 12 applications written in C and 1 in C++ (252.eon) that are used as benchmarks. CFP2000 contains 14 applications (6 Fortran-77, 4 Fortran-90 and 4 C) that are used as benchmarks.
Following is a brief description of the workloads.

In order to build, compile and run the SPEC CPU 2000, SPEC proposes rules and regulations that need to be followed. They can be classified in to following categories.
a) System Requirements: The basic system requirements that SPEC imposes are

1. System running UNIX

2. 256MB of RAM

3. 800MB to 1GB of disk space

4. A set of compilers such as C and C++ compiler
   
   

Further details on system requirements can be obtained from SPEC CPU 2000 System Requirements

b) Procedure to build the tools:
Cross compilation and individual builds of CPU2000 is allowed by SPEC however SPEC encourages use of its tools such as runspec, specperl etc. to build the SPEC CPU 2000. Further details on the tools build process can be found at Building SPEC tools

c) Generating the Config File:
A Config file is a file in which user specifies compiler flags and other system-dependent information. It is recommended to go through the following link to understand more about the Config file Understanding SPEC CPU 2000 Config File

d) Additional process of using utilities:
Few generic utilities are provided by SPEC to cross verify the Config file, understand the time spent on each invocation etc. Details of these utilities are found at Details about SPEC CPU 2000 Utilities

Everyone who uses SPEC CPU 2000 needs runspec. It is the primary tool in the suite. It is used to build the benchmarks, run them, and report on their results. Runspec makes sure that the set of tools based on GNU Make and Perl5 are used to build and run the benchmarks. The use of runspec helps produce publication quality results and also ensures that results are reproducible and optimizations used are available in the configuration file. Before using runspec one needs to install CPU2000. Further details on runspec can be found at Using runspec for SPEC CPU 2000

This section describes the setup, building and installing SPEC CPU 2000 for Zynq-7000 including the setting of Zynq-7000 board as an evaluation target for SPEC CPU 2000.

The SPEC CPU 2000 workloads were initially built on a TI Panda board, which has an ARM Cortex-A9 dual core CPU sub-system similar to Zynq-7000. Later the same process was followed to build the binaries on Zynq-7000 based ZC702 board. The binaries compiled and generated are portable across platforms and no optimization specific to any architectural features has been done. Thus it is safer to use portable binaries compiled on panda board to evaluate the device performance.
If the user prefers using a panda board, the detailed procedure on how to boot Ubuntu on Panda board is available from Ubuntu on Panda-board. upon bringing up Ubuntu on Panda board user is requested to install necessary tools such as
gfortran (v4.7.2) and Gcc if they are not available using apt-get utility and follow the build process stated in SPEC website for building/cross compiling SPEC binaries.

Below we provide an overview of the process followed to build SPEC binaries on ZC702 Evaluation board.

1. Bring up Ubuntu Linux on Zynq-7000 ZC702 evaluation board. The Detailed Procedure to run Ubuntu on Zynq-7000 is available at Ubuntu On Zynq .Please note that the System details of Zynq-7000 can be found at Zynq 7000 System details

2. If the GCC tool chains are not available by default, they can be easily installed from Ubuntu software repositories using apt-get utility. GCC v4.7.2 (the latest compiler available on ARM Ubuntu repositories) was used to build the workloads.

3. Apart from GCC tool chain, a FORTRAN compiler tool chain is needed, since nearly half of the workloads in SPEC CPU2000 suite are written in FORTRAN. gfortran (v4.7.2) tool chain was downloaded using similar methodology as followed for GCC.

4. Download the SPEC CPU2000 sources and tools from the official website of SPEC.

5. Since the SPEC CPU2000 distribution does not support ARM architecture, one needs to modify the makefile and create a Config file. The simple.cfg file from the SPEC website acts a template for creating an initial Config file. For your reference the Config file and the makefile patch that were used for building SPEC are provided as attachments with this tech tip

6. Run the specmake SPEC CPU2000 tool to build (compile) the benchmark source. Some warnings would be reported which can be safely ignored.i,,e specmake build 2 > make.err | tee make.out.

7. For ease of use on other boards, Create a tar ball of the binaries and SPEC CPU2000 tools created.

Few of important the compile flags used are stated for ready reference
hw_model = Zynq
hw_cpu = Cortex-A9
hw_cpu_mhz =
hw_disk = Flash Drive, 8 GB
hw_fpu = Integrated
hw_memory = 1 GB
hw_ncpu = 2
hw_ncpuorder = 2
tune = base
CROSS_COMPILE=arm-xilinx-linux-gnueabi-
COPTIMIZE = -O3 -fno-common
CXXOPTIMIZE = -O3 -fno-common

The tests were carried out on a Zynq-7000 ZC702 Evaluation board with Xilinx’s standard images released as a part of 14.7 . The procedure to install the SPEC tools and the binaries is described below.

1. Copy the standard Xilinx linux binaries downloaded from Zynq 14.7-2013.3 Release to the SD card.

2. Copy the generated build data for SPEC CPU2000 on the SD card.

3. Power on the board and Boot Zynq-7000 device using SD boot mode.

4. After booting, check if the SD card is mounted automatically, in some cases it may have failed so mount the SD card if not mounted automatically.

5. Make sure correct date is set on the system.

6. Go to the SPEC CPU 2000 Installation folder (I,,e path where SPEC binaries are placed), and Install the SPEC tools by executing install.sh script.i,,e by running ./install.sh from the command line.

7. The install.sh will prompt for few inputs from the user. Fill in the appropriate inputs to finish the installation.

8. Import the environment variables by executing shrc script. . ./shrc.

9. Make sure that no changes in the set up are further made.

10. Details of the installation can be found in the installation Log created in the SPEC Installation folder.

11. The Zynq-7000 devices ready for execution of workloads using runspec command.
    
    * Please note that if there are errors while installing the SPEC binaries, it could be because of missing libraries. Since this issue was frequently observed, We have provided ramdisk image as attachments along with this tech tip.