Docker on Zynq Ultrascale+ (Xilinx Yocto Flow)

Table of Contents

Introduction to Docker and Containers

Docker defines a container as a "standard unit of software".  Container images package an application and all of its dependencies enabling it to run quickly and reliably across platforms.  A container is isolated from processes and other containers running on the same platform unless interfaces are explicitly defined.  Docker containers provide a standard, lightweight and secure virtualization solution when a full hypervisor is overkill.  Unlike hypervisors that virtualize the hardware, containers are lightweight because only the OS is virtualized.  Containers rely on kernel features such as namespaces, cgroups and unionfs.

Docker provides:

  • Docker Engine for container runtime
  • Docker tools for creating, deploying and managing containers
  • Docker Hub repository for hosting container images

This wiki assumes you have a working knowledge of Yocto.  It will walk you through how to build and deploy Docker on Zynq Ultrascale+ with a Yocto flow in the following four configurations:

  • systemd on a flash filesystem
  • systemd on a ramdisk
  • sysvinit on a flash filesystem
  • sysvinit on a ramdisk

Requirements

  1. ZCU102 Eval Board
  2. Linux host machine or VM
    1. Install Yocto dependencies or CROPS/Poky-container
    2. Clone Xilinx Yocto and checkout release version
  3. Internet

Testing

Releases

rel-v2018.3 (Rocko)

Platforms

ZCU102

meta-virtualization Layer

The Docker recipe is included in the meta-virtualization layer.  The Xilinx Yocto manifest instructs repo to automatically clone meta-virtualization, so there is no need to clone it manually unless you are using an unsupported Yocto flow.  You can verify that the virtualization layer is installed in the sources directory and that it's included in the bblayers.conf.  From this layer we will be building and installing the docker and docker-contrib packages.  We will add this to a Docker machine configuration later.

meta-xilinx-docker Layer

Throughout this wiki we will be populating a custom layer, meta-xilinx-docker, that reflects the listing below.  Creating a Custom Xilinx Yocto Layer shows how to create a base layer using the Yocto scripts.

Linux Configuration

The default ZCU102 kernel configuration does not have all the required CONFIG options for Docker, so we will need to turn them on through a configuration fragment.  If you don't already have a kernel recipe directory in your layer, create the directory structure as shown below in your custom layer.

Next edit the linux-xlnx_%.bbappend file as shown below and add the docker.cfg.  Note the wildcard "%" is used to match any kernel version.

Add a docker.cfg file in the cfg directory and add the CONFIG requirements shown in the listing below.  The script from the Moby Project was used to determine these (more on this later).

Machine Configuration

Next we will configure the machine for installation of the Docker package(s) in the image.  Create the machine directory under the conf directory in your layer and add the docker-systemd-zcu102-zynqmp.conf file.  Note that we have defined separate machine configuration allows you to keep both sysvinit and systemd Docker builds in your deploy directory after baking.  The distro configuration will be covered in the next section.

Populate the docker-systemd-zcu102-zynqmp.conf file as shown below.  The docker-contrib is an optional supporting package which contains a script that we will use to verify the kernel configuration.  

Next you need to set your distro policy to the new petalinux-systemd policy.  Set the DISTRO variable as shown below in your local.conf or your custom <machine>.conf.  We will set it in the docker-systemd-zcu102-zynqmp.conf machine configuration file.

In order to simplify the network manager on an embedded system, connman is recommended to manage the network adapter.  This is completely optional.  Without connman, you will need to setup the networkd service unit configuration files which is beyond the scope of this wiki.  Last the loglevel was downgraded to level 6 and audit turned off from the kernel command line to reduce console chatter.

If you want to boot with an initramfs, add the variables in the listing below to your <machine>.conf or local.conf and observe the ramdisk info note below when running Docker.

Distro Configuration (systemd)

Systemd is a replacement init system for sysvinit which is used by many advanced Linux Distributions.  While Docker works with sysvinit, it works best with systemd.  When systemd is enabled, systemd will start the docker daemon during boot.  To enable systemd in your image, you need to patch an existing distro configuration or create a new one.  We will create a new one here based on the Petalinux Distro.  Create the directory structure shown below in your layer and add the petalinux-systemd.conf

HDF

One last housekeeping thing is to add a bbappend to reuse the default HDF from GitHub, so we don't need to add a custom HDF (you're welcome Mr. software developer).  This script just creates a link to the base machine we want to leverage, e.g. zcu102-zynqmp, in the external-hdf working directory.

Build Docker

The bitbake command below assumes you are building a docker-systemd-zcu102-zynqmp.conf machine which includes the configurations for Docker on a ZCU102.  If you are setting the configurations in your local.conf, then you may target any machine such as zcu102-zynqmp.

If you want to build Docker with sysvinit, then you may create another machine docker-zcu102-zynqmp.conf and comment out the systemd lines as noted in the docker-systemd-zcu102-zynqmp.conf listing above.

Install Docker

Docker expects to run from a non-RAM based root filesystem since it uses pivot_root to jail the container.  For that reason it's recommended that you setup an SD card with VFAT and Ext4 partitions.  Once your SD card is partitioned, copy the boot images to the VFAT partition and extract the rootfs to the Ext4 partition.  These images are available in the deploy/images directory of the machine you built.  Copy the images as shown below noting that the VFAT partition is mounted on boot and the Ext4 partition is mounted on rootfs.  This example is using the dtb that is built from the kernel tree.  If you are using a custom HDF, you will want to use the dtb compiled from the DTG so make sure that uEnv.txt is using the correct dtb.

Run Docker

Insert the SD card into you ZCU102, connect the Ethernet to an internet connected router and boot the board.  To verify that the kernel is configured properly, you can run the Moby script check-config.sh.  Your output should look similar to the listing below.  Note that AUFS_FS and zfs are not enabled which is expected.

systemd

If you are running Docker with systemd and connman, then systemd will start dockerd and your network adapter will come up automatically during boot.

sysvinit

If you want to continue running your system with sysvinit, then you will need to manually start the dockerd in the background.

You can verify that the Docker daemon is running by issuing the "ps" command.   Next, test that you can connect to it by issuing "docker info".

With the ZCU102 booted and connected to the internet, run the standard hello-world container image to test your embedded Docker installation.  Your console should look similar to the hello-world listing.  Since this is the first time you are running the container, the image must be pulled from the Docker Hub repository.  Any subsequent runs will pull the image from the local repository.

Once you have verified that Docker is working with hello-world, you can try running more advanced containers such as ubuntu, centosgcc, etc.

Now you should be in a Ubuntu bash shell.  You can now install applications with apt-get.  Type exit to quit the container.

Creating a Custom Xilinx Yocto Layer

Adding an HDF to a Xilinx Yocto Layer

Customizing Device Trees in Xilinx Yocto

Xilinx Yocto Builds without an Internet Connection

References

Yocto Mega Manual (2.4.4 Rocko)

BitBake User Manual

Docker

Moby Project

systemd - freedesktop.org