This page includes information on how to build and deploy Xen on Xilinx boards, including both Xilinx Ultrascale+ and Versal boards.
Table of Contents
...
The guide below shows you how to build Xen, boot Xen and then run some example configurations on ZU+. The steps below use PetaLinux and assume you have some knowledge of using PetaLinux. Before starting you need to create a PetaLinux project. It is assumed that a default PetaLinux reference design is used unchanged in these instructions. The default PetaLinux configuration has images ready to do boot Xen, these are the pre-built images. You can use those or you can manually edit receipes recipes and build Xen yourself. The pre-built images can be found in this directory (inside a PetaLinux project) pre-built/linux/images/ and prefixed with "xen-". You can either use the pre-builts or follow the next section to configure and build Xen yourself. If you are using the pre-builts you can skip to the booting Xen section for your release version.
...
SD Booting Xen and Dom0
To boot Xen from an SD card you need to copy the following files to the boot partition of the SD card:
- BOOT.bin
- xen-Image
- the compiled device tree file renamed to system.dtb (xen.dtb or xen-qemu.dtb for QEMU from the pre-built images, system.dtb from a Petalinux build)
- xen
- xen-rootfs.cpio.gz)
When using the pre-built images from the BSP, copy these files from <project-dir>/pre-built/linux/
images. The prebuilt images are built to support both Linux without Xen and with Xen such that some of the Xen based image file names are different than in a normal Petalinux build. The prebuilt Linux kernel image includes an initramfs rootfs. Petalinux builds (rather than prebuilt images) require an initrd rootfs such that another file for the rootfs must also be used as described below.Booting with ImageBulider
ImageBuilder's script uboot-script-gen can be used to generate a uboot script that loads all the binaries automatically from MMC. First, convert the xen and xen-rootfs binaries into raw binaries and write an appropriate config file, see TFTPing Xen using ImageBuilder above. Then, call ubootxen.Booting with ImageBulider
ImageBuilder's script uboot-script-gen can be used to generate a uboot script that loads all the binaries automatically from MMC. Call uboot-script-gen with the following command, assuming that $sdbootdev is 0 and $partid is 1:...
| Code Block |
|---|
|
$ mmc dev $sdbootdev &&&& mmcinfo; load mmc $sdbootdev:$partid 0xC00000 boot.scr; source 0xC00000 |
Graphical Desktop in Dom0
ZU+ only.
To get a graphic desktop, e.g. matchbox, working in dom0 on ZCU102, it is necessary to add two SMIDs to device tree: the SMID 0xce3 for zynqmp-display@fd4a0000 and SMID 0xce4 for dma@fd4c0000. The attached DTB comes with the necessary modifications.
Starting simple additional guests
If running on QEMU, we'll need to setup a port mapping for port 22 (SSH) in our VM.
In this example, we forward the hosts port 2222 to the VM's port 22.
| Code Block |
|---|
|
$ petalinux-boot --qemu --u-boot --qemu-args "-net nic -net nic -net nic -net nic -net user,tftp=pre-built/linux/xen,hostfwd=tcp:127.0.0.1:2222-10.0.2.15:22" |
Once you hit the u-boot prompt, follow the steps in the earlier section on how to run Xen dom0.
When dom0 has finished booting, we'll need to copy a guest Image into dom0's filesystem.
We'll use the base prebuilt PetaLinux Image as our domU guest.
If running on QEMU, we use scp's -P option to connect to our hosts port 2222 where QEMU will forward the connection to the guests port 22:
To target QEMU run the following on the host:
| Code Block |
|---|
|
scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -P 2222 pre-built/linux/xen/xen-Image root@localhost:/boot/ |
If running on hardware run the following on the host:
| Code Block |
|---|
|
scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no pre-built/linux/xen/xen-Image root@<board-ip>:/boot/ |
If you would prefer to load DomU's kernel to the guest via SD card, you can follow the instructions in the "Starting Linux guests with Pass-through networking" section.
The xen-image-minimal rootFS includes some prepared configurations that you can use. These are located in '/etc/xen/'
To start a simple guest run the following from the dom0 prompt
| Code Block |
|---|
|
# xl create -c example-simple.cfg |
You'll see another instance of Linux booting up.
At any time you can leave the console of the guest and get back to dom0 by pressing
ctrl+].
Once at the dom0 prompt you can list the guests from dom0:
To get back to the guests console:
| Code Block |
|---|
|
# xl console guest0 |
You can create further guests by for example running:
| Code Block |
|---|
|
# xl create example-simple.cfg name=\"guest1\"
# xl create example-simple.cfg name=\"guest2\"
root@plnx_aarch64:/etc/xen# xl list
Name ID Mem VCPUs State Time(s)
Domain-0 0 512 1 r----- 79.8
Domain-0 0 512 1 r----- 79.8
guest0 1 256 2 ------ 93.7
guest1 2 256 2 ------ 26.6
guest2 3 256 2 ------ 1.8 |
To destroy a guest:
| Code Block |
|---|
|
# xl destroy guest0 |
CPU Pinning
The following will only work on QEMU with multi-core enabled or on real HW.
When running multiple guests with multiple Virtual CPUs, Xen will schedule the various vCPUs onto real physical CPUs.
The rules and considerations taken in scheduling decisions depend on the chosen scheduler and the configuration.
To avoid having multiple vCPUs share a single pCPU, it is possible to pin a vCPU onto a pCPU and to give it exclusive access.
To create a simple guest with one Virtual CPU pinned to Physical CPU #3, you can do the following:
| Code Block |
|---|
|
xl create example-simple.cfg 'name="g0"' 'vcpus="1"' 'cpus="3"' |
Another way to pin virtual CPUs on to Physical CPUs is to create dedicated cpu-pools.
This has the advantage of isolating the scheduling instances.
By default a single cpu-pool named Pool-0 exists. It contains all the physical cpus.
We'll now create our pool named rt using the credit2 scheduler.
| Code Block |
|---|
|
xl cpupool-create 'name="rt"' 'sched="credit"'
xl cpupool-cpu-remove Pool-0 3
xl cpupool-cpu-add rt 3 |
Now we are ready to create a guest with a single vcpu pinned to physical CPU #3.
| Code Block |
|---|
|
xl create /etc/xen/example-simple.cfg 'vcpus="1"' 'pool="rt"' 'cpus="3"' 'name="g0"' |
Starting Linux guests with Para-Virtual networking (PV network)
...
This time we will run QEMU slightly different. We'll create two port mappings. One for dom0's SSH port and another for the Para-Virtual domU.
The default IP addresses assigned by QEMUs builtin DHCP server start from 10.0.2.15 and count upwards.
Dom0 will be assigned 10.0.2.15, the next guest 10.0.2.16 and so on.
So here's the command line that maps host port 2222 to dom0 port 22 and 2322 to domUs port 22.
| Code Block |
|---|
|
$ petalinux-boot --qemu --u-boot --qemu-args "-net nic -net nic -net nic -net nic -net user,tftp=pre-built/linux/xen,hostfwd=tcp:127.0.0.1:2222-10.0.2.15:22,hostfwd=tcp:127.0.0.1:2322-10.0.2.16:22" |
Now, follow the instructions from section 1 on how to boot Xen dom0.
Once you are at the dom0 prompt and have copied a domU image (see earlier steps) we'll need to setup the networking.
In this example, we will configure the guests to directly join the external network by means of a bridge.
First of all, we need to de-configure the default setup.
Kill the dhcp client for eth0:
| Code Block |
|---|
|
# killall -9 udhcpc |
List and remove existing addresses from eth0:
| Code Block |
|---|
|
# ip addr show dev eth0 |
In our example the address is 10.0.2.15/24:
| Code Block |
|---|
|
# ip addr del 10.0.2.15/24 dev eth0 |
Then, create the bridge and start DHCP on it for dom0:
| Code Block |
|---|
|
# brctl addbr xenbr0
# brctl addif xenbr0 eth0
# /sbin/udhcpc -i xenbr0 -b |
You should see something like the following:
| Code Block |
|---|
|
udhcpc (v1.24.1) started
[ 165.460858] xenbr0: port 1(eth0) entered blocking state
[ 165.461819] xenbr0: port 1(eth0) entered forwarding state
Sending discover...
Sending select for 10.0.2.15...
Lease of 10.0.2.15 obtained, lease time 86400
/etc/udhcpc.d/50default: Adding DNS 10.0.2.3 |
Similar to before we will use the pre-defined examples in '/etc/xen/'
The start DomU
| Code Block |
|---|
|
# xl create -c example-pvnet.cfg |
You should see a new linux instance boot up.
Now we'll ssh into the domU from the host running Para-Virtual networking:
| Code Block |
|---|
|
$ ssh -p 2322 root@localhost |
Starting Linux guests with Pass-through networkingStarting Linux guests with Pass-through networking
It is possible to directly assign the network peripheral to a domU on both ZU+ and Versal. The following example is for ZU+.
Turn xen.dtb into xen.dts:
| Code Block |
|---|
|
dtc -I dtb -O dts xen.dtb > xen.dts |
The, edit xen.dts by adding xen,passthrough; under the node of the device to assign, in this case ethernet@ff0e0000:
...
Starting a Guest with a Passthrough SD Card
This section explains how to It is possible to directly assign the MMC controller to a Xen DomU on ZynqMP family boardsdomU on both ZU+ and Versal. The following example is for ZU+ only. It allows a Xen DomU full unmediated access to a SD card plugged into the board.
...
Starting a guest with a passthrough SATA disk
This section explains how to It is possible to directly assign the SATA controller to a Xen DomU on ZynqMP family boardsdomU on both ZU+ and Versal. The following example is for ZU+ only. It allows a Xen DomU full unmediated access to any SATA disks connected to it.
...
