Table of Contents
Table of Contents | ||
---|---|---|
|
Building the Xen Image
Main use of Yocto for Xen is to build custom images. if you want pre-built images please check this page for instruction.
Install the repo
Repo is a tool that enables the management of many git repositories given a single manifest file. The repo will fetch the git repositories specified in the manifest and, by doing so, sets up a Yocto Project build environment for you.
Code Block | ||||
---|---|---|---|---|
| ||||
# Download the Repo script. curl -k https://storage.googleapis.com/git-repo-downloads/repo > repo # Generally, curl is install at /usr/bin/curl. If it is installed at custom location please point it to correct location. # Make it executable. chmod a+x repo # If it is correctly installed, you should see a Usage message when invoked with the help flag. repo --help # If system complaints about repo not found. Please run with './repo --help' |
Fetch all sources
Code Block | ||||
---|---|---|---|---|
| ||||
# initialize the repo. repo init -u https://github.com/Xilinx/yocto-manifests.git -b <release-version> # repo sync to get all sources repo sync |
Example of release-version: rel-v2020.2, rel-v2020.1 and rel-v2019.2 etc.
Source environment
Code Block | ||||
---|---|---|---|---|
| ||||
# source the environment to build using bitbake. source setupsdk |
Add option to generate rootfs image with Yocto build
This option will generate the rootfs in cpio.gz format which will be used in creating boot-scripts step. Rootfs built will contain the useful Xen tools like xl.
Code Block | ||||
---|---|---|---|---|
| ||||
# Edit conf/local.conf and Add the below line
IMAGE_FSTYPES += "cpio.gz" |
Add option to generate xen.dtsi
For some Yocto versions, system device tree for xen has not included “xen.dtsi”. To enable this it’s needed to add a user configuration
Code Block | ||
---|---|---|
| ||
# Edit recipes-bsp/device-tree/device-tree.bbappend
ENABLE_XEN_DTSI="1" |
Build using bitbake
Code Block | ||
---|---|---|
| ||
# For building: $ MACHINE=vck190-versal bitbake xen-image-minimal # Please use appropriate yocto machine name to build for a target hardware. Examples: #MACHINE=vck190-versal #MACHINE=zcu102-zynqmp #MACHINE=zcu104-zynqmp #MACHINE=ultra96-zynqmp |
If user wants to compile Xen from external source i.e. wants to modify Xen source code and rebuild the images. Please do the following:
Code Block | ||
---|---|---|
| ||
# Before 2022.1 release, edit conf/local.conf as follows: INHERIT += "externalsrc" EXTERNALSRC_pn-xen = "/path_to_xen_source/" EXTERNALSRC_BUILD_pn-xen = "/path_to_xen_source/" EXTERNALSRC_pn-xen-tools = "/path_to_xen_source/" EXTERNALSRC_BUILD_pn-xen-tools = "/path_to_xen_source/" # After 2022.1 release, edit conf/local.conf as follows: INHERIT+="externalsrc" EXTERNALSRC:pn-xen = "/path_to_xen_source/" EXTERNALSRC_BUILD:pn-xen = "/path_to_xen_source/" EXTERNALSRC:pn-xen-tools = "/path_to_xen_source/" EXTERNALSRC_BUILD:pn-xen-tools = "/path_to_xen_source/" # After local.conf is updated, clean the Xen build MACHINE=vck190-versal bitbake xen-image-minimal -c cleanall xen-tools # Rebuild using: MACHINE=vck190-versal bitbake xen-image-minimal |
Warning |
---|
This step can take a lot of time depending upon host machine processing power. It will also required significant disk space, please see Yocto for more details. |
Diagnosing Xen Build issue:
If build complains about "
error: liblzma: Memory allocation failed",
change the memory settings on your machine:Code Block language bash sudo sysctl -w vm.overcommit_memory=0 sudo sysctl -w vm.overcommit_ratio=50
In case you get the following error
Code Block language bash ayankuma@xcbayankuma40x:/scratch$ repo sync File "/scratch/.repo/repo/main.py", line 79 file=sys.stderr) ^ SyntaxError: invalid syntax
You need to add python3 before 'repo'. The command will then be as follows :-
Code Block language bash # initialize the repo. python3 repo init -u https://github.com/Xilinx/yocto-manifests.git -b <release-version> # repo sync to get all sources python3 repo sync
If builds complains about "Checkpolicy" or flask, do the following:
Code Block language bash # edit yocto_xen/source/meta-virtualization/recipes-extended/xen/xen.inc # to the DEPENDS variable, add this line below gnu-efi checkpolicy-native \ # Save and bitbake again
If build fails due to "qemu-xen-dir-find failed" issue: Always clean the xen repo before invoking "MACHINE=zcu102-zynqmp bitbake xen-image-minimal" ie do the following
Code Block language bash cd /path_to_xen_source/ git clean -fdx git reset --hard
- If you see do_package_qa issues like below
Code Block language bash Initialising tasks: 100% |############################################################################################| Time: 0:00:03 Sstate summary: Wanted 7 Found 0 Missed 7 Current 2070 (0% match, 99% complete) NOTE: Executing Tasks ERROR: xen-tools-4.14+stable999-r0 do_package_qa: QA Issue: xen-tools-pygrub: /work/cortexa72-cortexa53-xilinx-linux/xen-tools/4.14+s table999-r0/packages-split/xen-tools-pygrub/usr/lib/xen/bin/pygrub maximum shebang size exceeded, the maximum size is 128. [shebang-s ize] ERROR: xen-tools-4.14+stable999-r0 do_package_qa: QA Issue: /usr/lib/xen/bin/pygrub contained in package xen-tools-pygrub requires /s cratch/ayankuma/xilinx_yocto/build/tmp/work/cortexa72-cortexa53-xilinx-linux/xen-tools/4.14+stable999-r0/recipe-sysroot-native/usr/bi n/python3-native/python3, but no providers found in RDEPENDS_xen-tools-pygrub? [file-rdeps] ERROR: xen-tools-4.14+stable999-r0 do_package_qa: QA run found fatal errors. Please consider fixing them. ERROR: Logfile of failure stored in: /scratch/ayankuma/xilinx_yocto/build/tmp/work/cortexa72-cortexa53-xilinx-linux/xen-tools/4.14+s$ able999-r0/temp/log.do_package_qa.11224 ERROR: Task (/scratch/ayankuma/xilinx_yocto/sources/core/../meta-virtualization/recipes-extended/xen/xen-tools_4.14.bb:do_package_qa) failed with exit code '1'
The solution would be to remove pygrub. Under xen-tools.inc: do_install()
Code Block language bash rm -r ${D}${bindir}/pygrub rm -r ${D}${libdir}/xen/bin/pygrub
- During the boot, if you see issues for launch-xenstore as below
Code Block language bash Public key portion is: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCGljmUgRv8Kri5kDW/iCBElkjPFNs8qUSqOPvxjxbKXR5ZFZg4TjFc3LbHUjZFuOIqCPqWv82qp77rdwnc1dGpKvD7F/FHt9oPJgsFukurhaXhs99P+Z5BLZInj7ygM8V0u6pkZ7u5QyaLDpff03T0yt1BwiLFFHAcWnspJCljzvOIXVGh7RdZ9ob39gA74VpxM+yAxfIafM2xBySYkK56BIL+Jzc810HTSKbP7h5jEYSDBkCa2Or3QEbXth+DAZpapV5no70FcgXWFarevCFIrLgw7T3z8eILg3YPyaSPuWSbLOzUYNGh/RYMMUgUjtAuS4+cm6u3N05O1xT2tJGf root@zcu102-zynqmp Fingerprint: sha1!! f3:af:73:ac:22:15:0b:b0:97:90:19:75:27:21:e0:ef:c5:cd:c8:99 dropbear. Starting internet superserver: inetd. Starting syslogd/klogd: done Starting /usr/sbin/xenstored.../etc/xen/scripts/launch-xenstore: line 86: prlimit: command not found .............................. Could not start /usr/sbin/xenstored Starting domain watchdog daemon: xenwatchdogd startup
- The solution would be to add util-linux-prlimit and util-linux to the list of installed images.
You can do either the below change
Code Block language bash diff --git a/recipes-extended/xen/xen-tools.inc b/recipes-extended/xen/xen-tools.inc index 722c503..791f6fb 100644 --- a/recipes-extended/xen/xen-tools.inc +++ b/recipes-extended/xen/xen-tools.inc @@ -30,7 +30,7 @@ GLIBC32_class-native = "" DEPENDS += "${GLIBC32}" RDEPENDS_${PN} = "\ - bash perl xz \ + bash perl xz util-linux-prlimit util-linux \ ${PN}-console \ ${PN}-libxenguest \ ${PN}-libxenlight \
- Or a proper solution would be to do the following
Code Block language bash diff --git a/recipes-extended/images/xen-image-minimal.bb b/recipes-extended/images/xen-image-minimal.bb index 6733801..ee5bf02 100644 --- a/recipes-extended/images/xen-image-minimal.bb +++ b/recipes-extended/images/xen-image-minimal.bb @@ -9,6 +9,8 @@ XEN_KERNEL_MODULES ?= "kernel-module-xen-blkback kernel-module-xen-gntalloc \ " IMAGE_INSTALL += " \ + util-linux \ + util-linux-prlimit \ packagegroup-core-boot \ packagegroup-core-ssh-openssh \ ${XEN_KERNEL_MODULES} \
- During boot, if you see this error
Code Block language bash Setting domain 0 name, domid and JSON config... FATAL: Failed to open connection to gnttab: No such file or directory cannot set domid for Dom0
- Solution :- The kernel image is missing some of the XEN configs. You may use the kernel configuration attached here 1697873938
- If you see errors like the the following
Code Block language bash | ERROR: pkg-config binary 'aarch64-xilinx-linux-pkg-config' not found | | make: Entering directory '/scratch/ayankuma/upstream_xen/xen/tools/qemu-xen-build'
The remedy is as follows :-
- In xen/tools/Makefile, disable building subdir-install-qemu-xen-dir as follows
Code Block language bash -subdir-install-qemu-xen-dir: subdir-all-qemu-xen-dir - cd emu-xen-build; \ +#subdir-install-qemu-xen-dir: subdir-all-qemu-xen-dir +# cd emu-xen-build; \ $(MAKE) install
- Or errors related to "unistd.h" not present or gcc related errors
- cd <to_dir>/xen
- git clean -fdx
- git clean -fdx
- git reset --hard HEAD
- cd <to_dir>/xen/tools/qemu-xen-dir-remote
- git clean -fdx
- And then in Yocto , do the following
- MACHINE=zcu102-zynqmp bitbake -c clean xen-image-minimal
- MACHINE=zcu102-zynqmp bitbake xen-image-minimal
- If you see errors like the following,
Code Block ERROR: xen-tools-4.14+stable999-r0 do_compile: oe_runmake failed ERROR: xen-tools-4.14+stable999-r0 do_compile: Execution of '/scratch/ayankuma/xilinx_yocto/build/tmp/work/cortexa72-cortexa53-xilinx-linux/xen-tools/4.14+stable999-r0/temp/run.do_compile.23363' failed with exit code 1: make -C tools/include install make[1]: Entering directory '/scratch/ayankuma/upstream_xen/xen/tools/include' /scratch/ayankuma/upstream_xen/xen/tools/include/../../tools/Rules.mk:174: *** You have to run ./configure before building or installing the tools. Stop.
rm -rf tmp
Repeat the steps 5.b
- Also, if you see errors related to "version-going-backwards"
- You need to remove "version-going-backwards" from ERROR_QA in core/meta/classes/insane.bbclass
Thus core/meta/classes/insane.bbclass will have the following for ERROR_QA
Code Block language bash ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la \ perms dep-cmp pkgvarcheck perm-config perm-line perm-link \ split-strip packages-list pkgv-undefined var-undefined \ expanded-d invalid-chars \ license-checksum dev-elf file-rdeps configure-unsafe \ configure-gettext perllocalpod shebang-size \ already-stripped installed-vs-shipped ldflags compile-host-path \ install-host-path pn-overrides unknown-configure-option \ useless-rpaths rpaths staticdev \ "
For release 2022.1 and 2022.2, there is an issue with task do_write_xen_qemuboot_dtb for xen-image-minimal build and this results in following error:
Code Block language bash | Multiarch not setup properly. | WARNING: /scratch/devops/yocto/yocto_test/build/tmp/work/vck190_versal-xilinx-linux/xen-image-minimal/1.0-r0/temp/run.do_write_xen_qemuboot_dtb.14323:165 exit 1 from 'qemu-system-aarch64-multiarch -device qemu-xhci -device usb-tablet -device usb-kbd -machine virt,gic-version=3 -machine virtualization=true ${QB_CPU} -m 512 -nographic -serial mon:stdio -machine "dumpdtb=/scratch/devops/yocto/yocto_test/build/tmp/work/vck190_versal-xilinx-linux/xen-image-minimal/1.0-r0/xen-image-minimal-1.0/qemu-dumped.dtb"' | WARNING: Backtrace (BB generated script): | #1: generate_xen_qemuboot_dtb, /scratch/devops/yocto/yocto_test/build/tmp/work/vck190_versal-xilinx-linux/xen-image-minimal/1.0-r0/temp/run.do_write_xen_qemuboot_dtb.14323, line 165 | #2: do_write_xen_qemuboot_dtb, /scratch/devops/yocto/yocto_test/build/tmp/work/vck190_versal-xilinx-linux/xen-image-minimal/1.0-r0/temp/run.do_write_xen_qemuboot_dtb.14323, line 154 | #3: main, /scratch/devops/yocto/yocto_test/build/tmp/work/vck190_versal-xilinx-linux/xen-image-minimal/1.0-r0/temp/run.do_write_xen_qemuboot_dtb.14323, line 325 ERROR: Task (/scratch/devops/yocto/yocto_test/sources/core/../meta-virtualization/recipes-extended/images/xen-image-minimal.bb:do_write_xen_qemuboot_dtb) failed with exit code '1'
Please run below steps and apply the attached 0001-Fix-for-xen-image-minimal.patch :
Code Block language bash # If git tool is installed, user can apply patch using git: cd {Yocto_dir}/sources/meta-virtualization git am 0001-Fix-for-xen-image-minimal.patch # If git tools isn't installed, user can edit following file: vim {Yocto_dir}/sources/meta-virtualization/recipes-extended/images/xen-image-minimal.bb and add below line: deltask do_write_xen_qemuboot_dtb # Finally, re-run the bitbake MACHINE=vck190-versal bitbake xen-image-minimal
If there are error about missing pmu-rom.elf like below:
Code Block language bash runqemu - ERROR - Failed to run qemu: Error: Missing PMU ROM: /scratch/devops/yocto/yocto_xen/build/tmp/deploy/images/zcu102-zynqmp/pmu-rom.elf See "meta-xilinx/README.qemu.md" for more information on accquiring the PMU ROM.
Please copy a pmu-rom.elf under pre-built/linux/images folder of a BSP(you can download from here) or can be build using yocto like below:
Code Block language bash MACHINE=zcu102-zynqmp bitbake petalinux-image-minimal
Creating boot scripts
To create boot scripts, we use Imagebuilder which generates a U-Boot script that loads all the necessary binaries and automatically adds the required entries to device tree at boot time. In order to use it, we will need to write a config file first.
First, cd to <current directory>/tmp/deploy/images/$(machine_name). Example: build/tmp/deploy/images/vck190-versal.
Next, create a file named "config" with the contents as illustrated below and save the file.
Code Block | ||
---|---|---|
| ||
MEMORY_START="0x0" MEMORY_END="0x80000000" DEVICE_TREE="system.dtb" XEN="xen" DOM0_KERNEL="Image" DOM0_RAMDISK="xen-image-minimal-vck190-versal.cpio.gz" NUM_DOMUS=0 UBOOT_SOURCE="boot_xen.source" UBOOT_SCRIPT="boot_xen.scr" |
...