Docker defines a container as a "standard unit of software". Container images package an application and all of its dependencies enabling it to run quickly and reliably across platforms. A container is isolated from processes and other containers running on the same platform unless interfaces are explicitly defined. Docker containers provide a standard, lightweight and secure virtualization solution when a full hypervisor is overkill. Unlike hypervisors that virtualize the hardware, containers are lightweight because only the OS is virtualized. Containers rely on kernel features such as namespaces, cgroups and unionfs.
Docker provides:
This wiki assumes you have a working knowledge of Yocto. It will walk you through how to build and deploy Docker on Zynq Ultrascale+ with a Yocto flow in the following four configurations:
This is an unsupported from Xilinx Technical Support Service Request(SR). Please do your due diligence when implementing Docker with Yocto and regression test against your system requirements. |
rel-v2018.3 (Rocko)
ZCU102
The Docker recipe is included in the meta-virtualization
layer. The Xilinx Yocto manifest instructs repo
to automatically clone meta-virtualization, so there is no need to clone it manually unless you are using an unsupported Yocto flow. You can verify that the virtualization layer is installed in the sources
directory and that it's included in the bblayers.conf
. From this layer we will be building and installing the docker
and docker-contrib
packages. We will add this to a Docker machine configuration later.
IMAGE_INSTALL_append = " docker docker-contrib" |
Throughout this wiki we will be populating a custom layer, meta-xilinx-docker
, that reflects the listing below. Creating a Custom Xilinx Yocto Layer shows how to create a base layer using the Yocto scripts.
meta-xilinx-docker ├── conf │ ├── distro │ │ └── petalinux-systemd.conf │ ├── layer.conf │ └── machine │ ├── docker-systemd-zcu102-zynqmp.conf │ └── docker-zcu102-zynqmp.conf ├── COPYING.MIT ├── README ├── recipes-bsp │ └── hdf │ └── external-hdf.bbappend ├── recipes-containers │ └── docker │ └── docker_git.bbappend └── recipes-kernel └── linux ├── cfg │ └── docker.cfg └── linux-xlnx_%.bbappend |
The default ZCU102 kernel configuration does not have all the required CONFIG
options for Docker, so we will need to turn them on through a configuration fragment. If you don't already have a kernel recipe directory in your layer, create the directory structure as shown below in your custom layer.
recipes-kernel/ └── linux ├── cfg │ └── docker.cfg └── linux-xlnx_%.bbappend |
Next edit the linux-xlnx_%.bbappend
file as shown below and add the docker.cfg
. Note the wildcard "%"
is used to match any kernel version.
FILESEXTRAPATHS_prepend := "${THISDIR}/cfg:" SRC_URI_append = " file://docker.cfg" |
Add a docker.cfg
file in the cfg directory and add the CONFIG
requirements shown in the listing below. The script from the Moby Project was used to determine these (more on this later).
CONFIG_NAMESPACES=y CONFIG_NET_NS=y CONFIG_PID_NS=y CONFIG_IPC_NS=y CONFIG_UTS_NS=y CONFIG_CGROUP_CPUACCT=y CONFIG_CGROUP_DEVICE=y CONFIG_CGROUP_FREEZER=y CONFIG_CGROUP_SCHED=y CONFIG_CPUSETS=y CONFIG_MEMCG=y CONFIG_VETH=y CONFIG_IP_NF_TARGET_MASQUERADE=y CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=y CONFIG_NETFILTER_XT_MATCH_IPVS=y CONFIG_IP_NF_NAT=y CONFIG_USER_NS=y CONFIG_SECCOMP=y CONFIG_CGROUP_PIDS=y CONFIG_MEMCG_SWAP=y CONFIG_MEMCG_SWAP_ENABLED=y CONFIG_BLK_CGROUP=y CONFIG_BLK_DEV_THROTTLING=y CONFIG_CFQ_GROUP_IOSCHED=y CONFIG_CGROUP_PERF=y CONFIG_CGROUP_HUGETLB=y CONFIG_NET_CLS_CGROUP=y CONFIG_CGROUP_NET_PRIO=y CONFIG_CFS_BANDWIDTH=y CONFIG_FAIR_GROUP_SCHED=y CONFIG_RT_GROUP_SCHED=y CONFIG_IP_NF_TARGET_REDIRECT=y CONFIG_IP_VS=y CONFIG_IP_VS_NFCT=y CONFIG_IP_VS_PROTO_TCP=y CONFIG_IP_VS_PROTO_UDP=y CONFIG_IP_VS_RR=y CONFIG_EXT3_FS_XATTR=y CONFIG_EXT3_FS_POSIX_ACL=y CONFIG_EXT3_FS_SECURITY=y CONFIG_VXLAN=y CONFIG_INET_ESP=y CONFIG_IPVLAN=y CONFIG_MACVLAN=y CONFIG_DUMMY=y CONFIG_NF_NAT_FTP=y CONFIG_NF_CONNTRACK_FTP=y CONFIG_NF_NAT_TFTP=y CONFIG_NF_CONNTRACK_TFTP=y CONFIG_AUFS_FS=y CONFIG_BTRFS_FS_POSIX_ACL=y CONFIG_BLK_DEV_DM=y CONFIG_DM_THIN_PROVISIONING=y CONFIG_OVERLAY_FS=y CONFIG_MD=y CONFIG_NET_SCHED=y CONFIG_NET_L3_MASTER_DEV=y CONFIG_BRIDGE=m # CONFIG_EXT2_FS is not set CONFIG_EXT4_USE_FOR_EXT2=y |
Next we will configure the machine for installation of the Docker package(s) in the image. Create the machine
directory under the conf
directory in your layer and add the docker-systemd-zcu102-zynqmp.conf
file. Note that we have defined separate machine configuration allows you to keep both sysvinit
and systemd
Docker builds in your deploy
directory after baking. The distro configuration will be covered in the next section.
conf/ └── machine ├── docker-systemd-zcu102-zynqmp.conf └── docker-zcu102-zynqmp.conf |
Populate the docker-systemd-zcu102-zynqmp.conf
file as shown below. The docker-contrib
is an optional supporting package which contains a script that we will use to verify the kernel configuration.
Next you need to set your distro policy to the new petalinux-systemd
policy. Set the DISTRO
variable as shown below in your local.conf
or your custom <machine>.conf
. We will set it in the docker-systemd-zcu102-zynqmp.conf
machine configuration file.
In order to simplify the network manager on an embedded system, connman
is recommended to manage the network adapter. This is completely optional. Without connman
, you will need to setup the networkd
service unit configuration files which is beyond the scope of this wiki. Last the loglevel
was downgraded to level 6 and audit
turned off from the kernel command line to reduce console chatter.
# Inherit all the properties from the zcu102-zynqmp machine require conf/machine/zcu102-zynqmp.conf # Reuse the fdt for the zcu102 bsp YAML_DT_BOARD_FLAGS_pn-device-tree = "{BOARD zcu102-rev1.0}" # Downgrade the loglevel to reduce console chatter and disable audit KERNEL_BOOTARGS_append_pn-u-boot-zynq-uenv = " loglevel=6 audit=0" # Include docker and docker-contrib in the image IMAGE_INSTALL_append = " docker docker-contrib" # Comment these out to use sysvinit DISTRO = "petalinux-systemd" IMAGE_INSTALL_append = " connman" |
If you want to boot with an initramfs
, add the variables in the listing below to your <machine>.conf
or local.conf
and observe the ramdisk info note below when running Docker.
# Remove wic because it causes circular dependencies with bundled initramfs IMAGE_FSTYPES_remove = "wic.qemu-sd" INITRAMFS_IMAGE = "petalinux-image-minimal" INITRAMFS_IMAGE_BUNDLE = "1" |
If you are using the default |
Systemd
is a replacement init system for sysvinit
which is used by many advanced Linux Distributions. While Docker works with sysvinit
, it works best with systemd
. When systemd
is enabled, systemd
will start the docker daemon during boot. To enable systemd
in your image, you need to patch an existing distro configuration or create a new one. We will create a new one here based on the Petalinux Distro. Create the directory structure shown below in your layer and add the petalinux-systemd.conf
conf/ └── distro └── petalinux-systemd.conf |
require conf/distro/petalinux.conf DISTRO = "petalinux-systemd" DISTRO_NAME = "PetaLinux with systemd" DISTRO_FEATURES_append = " systemd" # Uncomment this to remove legacy sysvinit scripts #DISTRO_FEATURES_BACKFILL_CONSIDERED += "sysvinit" VIRTUAL-RUNTIME_init_manager = "systemd" VIRTUAL-RUNTIME_initscripts = "systemd-compat-units" |
|
If you are using a custom HDF, you may skip this section. |
One last housekeeping thing is to add a bbappend
to reuse the default HDF from GitHub, so we don't need to add a custom HDF (you're welcome Mr. software developer). This script just creates a link to the base machine we want to leverage, e.g. zcu102-zynqmp
, in the external-hdf
working directory.
recipes-bsp └── hdf └── external-hdf.bbappend |
# attempt to determine a base machine assuming "<prefix>-<board>-<arch>" # example: "myproj-zcu102-zynqmp" could use "zcu102-zynqmp" HDF from GitHub MACHINE_BASE = "${@'-'.join(MACHINE.rsplit('-')[-2:])}" do_deploy_prepend() { if [ "${MACHINE_BASE}" != "${MACHINE}" ] && [ -d ${WORKDIR}/git/${MACHINE_BASE} ]; then ln -sf ${MACHINE_BASE} ${WORKDIR}/git/${MACHINE} fi } |
The bitbake
command below assumes you are building a docker-systemd-zcu102-zynqmp.conf
machine which includes the configurations for Docker on a ZCU102. If you are setting the configurations in your local.conf
, then you may target any machine such as zcu102-zynqmp
.
$ MACHINE=docker-systemd-zcu102-zynqmp bitbake petalinux-image-minimal |
If you want to build Docker with sysvinit
, then you may create another machine docker-zcu102-zynqmp.conf
and comment out the systemd
lines as noted in the docker-systemd-zcu102-zynqmp.conf
listing above.
$ MACHINE=docker-zcu102-zynqmp bitbake petalinux-image-minimal |
Docker expects to run from a non-RAM based root filesystem since it uses pivot_root
to jail the container. For that reason it's recommended that you setup an SD card with VFAT and Ext4 partitions. Once your SD card is partitioned, copy the boot images to the VFAT partition and extract the rootfs to the Ext4 partition. These images are available in the deploy/images
directory of the machine you built. Copy the images as shown below noting that the VFAT partition is mounted on boot
and the Ext4 partition is mounted on rootfs
. This example is using the dtb
that is built from the kernel tree. If you are using a custom HDF, you will want to use the dtb
compiled from the DTG so make sure that uEnv.txt
is using the correct dtb
.
$ cd tmp/deploy/images/docker-systemd-zcu102-zynqmp $ cp BOOT-docker-systemd-zcu102-zynqmp.bin /media/<user>/boot/boot.bin $ cp Image /media/<user>/boot/ $ cp uEnv.txt /media/<user>/boot/ $ cp Image-zynqmp-zcu102-rev1.0.dtb /media/<user>/boot/ $ sudo tar xzf petalinux-image-minimal-docker-systemd-zcu102-zynqmp.tar.gz -C /media/<user>/rootfs/ |
Insert the SD card into you ZCU102, connect the Ethernet to an internet connected router and boot the board. To verify that the kernel is configured properly, you can run the Moby script check-config.sh
. Your output should look similar to the listing below. Note that AUFS_FS
and zfs
are not enabled which is expected.
root@docker-systemd-zcu102-zynqmp:~# /usr/share/docker/check-config.sh info: reading kernel config from /proc/config.gz ... Generally Necessary: - cgroup hierarchy: properly mounted [/sys/fs/cgroup] - CONFIG_NAMESPACES: enabled - CONFIG_NET_NS: enabled - CONFIG_PID_NS: enabled - CONFIG_IPC_NS: enabled - CONFIG_UTS_NS: enabled - CONFIG_CGROUPS: enabled - CONFIG_CGROUP_CPUACCT: enabled - CONFIG_CGROUP_DEVICE: enabled - CONFIG_CGROUP_FREEZER: enabled - CONFIG_CGROUP_SCHED: enabled - CONFIG_CPUSETS: enabled - CONFIG_MEMCG: enabled - CONFIG_KEYS: enabled - CONFIG_VETH: enabled - CONFIG_BRIDGE: enabled (as module) - CONFIG_BRIDGE_NETFILTER: enabled (as module) - CONFIG_NF_NAT_IPV4: enabled (as module) - CONFIG_IP_NF_FILTER: enabled - CONFIG_IP_NF_TARGET_MASQUERADE: enabled (as module) - CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled - CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled (as module) - CONFIG_NETFILTER_XT_MATCH_IPVS: enabled (as module) - CONFIG_IP_NF_NAT: enabled (as module) - CONFIG_NF_NAT: enabled (as module) - CONFIG_NF_NAT_NEEDED: enabled - CONFIG_POSIX_MQUEUE: enabled Optional Features: - CONFIG_USER_NS: enabled - CONFIG_SECCOMP: enabled - CONFIG_CGROUP_PIDS: enabled - CONFIG_MEMCG_SWAP: enabled - CONFIG_MEMCG_SWAP_ENABLED: enabled (cgroup swap accounting is currently enabled) - CONFIG_BLK_CGROUP: enabled - CONFIG_BLK_DEV_THROTTLING: enabled - CONFIG_IOSCHED_CFQ: enabled - CONFIG_CFQ_GROUP_IOSCHED: enabled - CONFIG_CGROUP_PERF: enabled - CONFIG_CGROUP_HUGETLB: enabled - CONFIG_NET_CLS_CGROUP: enabled - CONFIG_CGROUP_NET_PRIO: enabled - CONFIG_CFS_BANDWIDTH: enabled - CONFIG_FAIR_GROUP_SCHED: enabled - CONFIG_RT_GROUP_SCHED: enabled - CONFIG_IP_VS: enabled (as module) - CONFIG_IP_VS_NFCT: enabled - CONFIG_IP_VS_RR: enabled (as module) - CONFIG_EXT4_FS: enabled - CONFIG_EXT4_FS_POSIX_ACL: enabled - CONFIG_EXT4_FS_SECURITY: enabled - Network Drivers: - "overlay": - CONFIG_VXLAN: enabled Optional (for encrypted networks): - CONFIG_CRYPTO: enabled - CONFIG_CRYPTO_AEAD: enabled - CONFIG_CRYPTO_GCM: enabled - CONFIG_CRYPTO_SEQIV: enabled - CONFIG_CRYPTO_GHASH: enabled - CONFIG_XFRM: enabled - CONFIG_XFRM_USER: enabled - CONFIG_XFRM_ALGO: enabled - CONFIG_INET_ESP: enabled - CONFIG_INET_XFRM_MODE_TRANSPORT: enabled - "ipvlan": - CONFIG_IPVLAN: enabled - "macvlan": - CONFIG_MACVLAN: enabled - CONFIG_DUMMY: enabled - "ftp,tftp client in container": - CONFIG_NF_NAT_FTP: enabled (as module) - CONFIG_NF_CONNTRACK_FTP: enabled (as module) - CONFIG_NF_NAT_TFTP: enabled (as module) - CONFIG_NF_CONNTRACK_TFTP: enabled (as module) - Storage Drivers: - "aufs": - CONFIG_AUFS_FS: missing - "btrfs": - CONFIG_BTRFS_FS: enabled - CONFIG_BTRFS_FS_POSIX_ACL: enabled - "devicemapper": - CONFIG_BLK_DEV_DM: enabled - CONFIG_DM_THIN_PROVISIONING: enabled - "overlay": - CONFIG_OVERLAY_FS: enabled - "zfs": - /dev/zfs: missing - zfs command: missing - zpool command: missing Limits: - /proc/sys/kernel/keys/root_maxkeys: 1000000 |
systemd
If you are running Docker with systemd
and connman
, then systemd
will start dockerd
and your network adapter will come up automatically during boot.
If you want to run Docker with |
PACKAGES =+ "${PN}-conf" FILES_${PN}-conf = "${sysconfdir}/systemd/system/docker.service.d/*" ALLOW_EMPTY_${PN}-conf = "1" RDEPENDS_${PN} += "${PN}-conf" do_install_append() { if ${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'true', 'false', d)}; then install -d ${D}${sysconfdir}/systemd/system/docker.service.d if [ ! -z "${INITRAMFS_IMAGE}" ]; then echo -e "[Service]\nEnvironment=\"DOCKER_RAMDISK=true\"" > ${D}${sysconfdir}/systemd/system/docker.service.d/docker-ramdisk.conf fi fi } |
sysvinit
If you want to continue running your system with sysvinit
, then you will need to manually start the dockerd
in the background.
# dockerd & |
If you want to run Docker with
|
You can verify that the Docker daemon is running by issuing the "ps"
command. Next, test that you can connect to it by issuing "docker info"
.
root@docker-systemd-zcu102-zynqmp:~# docker info WARN[0038] Could not get operating system name: Error opening /usr/lib/os-release: open /usr/lib/os-release: no such file or directory WARN[0038] failed to retrieve docker-init version: exec: "docker-init": executable file not found in $PATH Containers: 0 Running: 0 Paused: 0 Stopped: 0 Images: 0 Server Version: 17.06.0-dev Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: 3addd840653146c90a254301d6c3a663c7fd6429 runc version: 9d6821d1b53908e249487741eccd567249ca1d99-dirty (expected: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4) init version: N/A (expected: ) Kernel Version: 4.14.0-xilinx-v2018.3 Operating System: <unknown> OSType: linux Architecture: aarch64 CPUs: 4 Total Memory: 3.851GiB Name: docker-zcu102-zynqmp ID: 7BBA:EM3B:34TT:2NFJ:BA3G:TUEB:MPKA:JL5G:G3MJ:NEBV:2TNP:LWWC Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false |
With the ZCU102 booted and connected to the internet, run the standard hello-world
container image to test your embedded Docker installation. Your console should look similar to the hello-world
listing. Since this is the first time you are running the container, the image must be pulled from the Docker Hub repository. Any subsequent runs will pull the image from the local repository.
root@docker-systemd-zcu102-zynqmp:~# docker run --rm hello-world Unable to find image 'hello-world:latest' locally WARN[0277] Could not get operating system name: Error opening /usr/lib/os-release: open /usr/lib/os-release: no such file or directory WARN[0277] failed to retrieve docker-init version: exec: "docker-init": executable file not found in $PATH latest: Pulling from library/hello-world 3b4173355427: Pull complete Digest: sha256:2557e3c07ed1e38f26e389462d03ed943586f744621577a99efb77324b0fe535 Status: Downloaded newer image for hello-world:latest [ 354.686277] cgroup: docker-runc (2485) created nested cgroup for controller "memory" which has incomplete hierarchy support. Nested cgroups may change behavior in the future. [ 354.701848] cgroup: "memory" requires setting use_hierarchy to 1 on the root Hello from Docker! This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps: 1. The Docker client contacted the Docker daemon. 2. The Docker daemon pulled the "hello-world" image from the Docker Hub. (arm64v8) 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading. 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal. To try something more ambitious, you can run an Ubuntu container with: $ docker run -it ubuntu bash Share images, automate workflows, and more with a free Docker ID: https://hub.docker.com/ For more examples and ideas, visit: https://docs.docker.com/get-started/ |
Once you have verified that Docker is working with hello-world
, you can try running more advanced containers such as ubuntu
, centos
, gcc
, etc.
root@docker-systemd-zcu102-zynqmp:~# docker run --rm -it ubuntu bash |
Now you should be in a Ubuntu bash shell. You can now install applications with apt-get
. Type exit
to quit the container.
Creating a Custom Xilinx Yocto Layer
Adding an HDF to a Xilinx Yocto Layer
Customizing Device Trees in Xilinx Yocto
Xilinx Yocto Builds without an Internet Connection
Yocto Mega Manual (2.4.4 Rocko)