Accessing Storage Media in QEMU
In this chapter, we are going to use mainly SATA disks but the techniques covered here are directly applicable to SD, NAND and QSPI media.
SATA Disks
QEMU can emulate a SATA disk attached to the machine via the AHCI SATA controller. The SATA controller is always present in the model, but without extra command-line arguments, the SATA slots are modeled as empty.
FIrst, create a blank file 2GB in size (all zeros):
dd if=/dev/zero of=sata0.img bs=1M count=2048
Launch QEMU in PetaLinux with the following extra arguments to attach a SATA disk:
petalinux-boot --qemu --prebuilt 3 --qemu-args "-drive file=sata0.img,format=raw,id=sata-drive -device ide-drive,drive=sata-drive,bus=ahci@0xFD0C0000.0"
Breaking it down, the -drive argument is creating a QEMU storage drive for use by a device.
The file=sata0.bin argument means the drive will use our new file as data storage.
The format=raw argument tells QEMU that the file we pass is a raw file and the file contents are to be the disk contents "as-is".
The -device argument creates the actual SATA disk. Note the bus=ahci@0xFD0C0000.0 argument, which attaches the disk to the Zynq UltraScale+ MPSoC SATA controller (named by QEMU as ahci@0xFD0C0000.0).
Both arguments specify the same ID, which causes the SATA disk to use the drive specified in the first argument (in turn backing onto sata0.bin) for disk data.
The Kernel boot log should contain something like this:
[ 6.815251] ata2: SATA link down (SStatus 0 SControl 300) [ 6.818587] ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300) [ 6.823298] ata1.00: ATA-7: QEMU HARDDISK, 2.5+, max UDMA/100 [ 6.823960] ata1.00: 4194304 sectors, multi 16: LBA48 NCQ (depth 32) [ 6.825414] ata1.00: applying bridge limits [ 6.827816] ata1.00: configured for UDMA/100 [ 6.842715] scsi 0:0:0:0: Direct-Access ATA QEMU HARDDISK 2.5+ PQ: 0 ANSI: 5 [ 6.855480] sd 0:0:0:0: [sda] 4194304 512-byte logical blocks: (2.15 GB/2.00 GiB) [ 6.858989] sd 0:0:0:0: [sda] Write Protect is off [ 6.870276] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Note the third last line, which indicates we have found a 2GB SATA disk and enumerated as sda. This will be available for use in the logged in system as /dev/sda.
For those with experience using dev nodes, be aware that /dev/sda is not a partition, it is the raw disk image as full contents.
Later, when we come to work with partitions, we will have extra /dev notes for the partitions. We have no partitions or file-systems yet (as we simply have no meaningful data at all!).
The size of the sata0.bin file has been used to size the SATA disk itself. If we created our blank file as a different size, this size would change accordingly.
QEMU does not model any particular manufacturer of the disk, rather it models a generic SATA drive and hence it self-identifies to the system as "QEMU HARDDISK" (4th last line).
Log in to the system with root as username and password.
Low-Level Data Read and Write
In the QEMU system, inspect the first 4kB of the SATA disk contents using the hexdump utility. We could inspect more data by increasing the -n argument, but we will only operate of the first 4k for brevity in this exercise.
root@xilinx-zcu102-2020_2:~# hexdump -C /dev/sda -n 4096 | more
It should be blank, this is the expected output:
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00001000
We will use the dd utility to write some randomized data to the disk. Randomized dd'ing is a good low-level test of whether storage media works (without the complication of files and filesystems).
Write random data to the first 3kB of /dev/sda using this command.
root@xilinx-zcu102-2020_2:~# dd if=/dev/urandom of=/dev/sda bs=1k count=3
The dd command is a low-level data copying utility that is usable for copying data from one file to another. It allows the copying of a subsection of a source file to a subsection of the destination file.
The dev node /dev/sda is not a regular file, but behaves like a file and backs onto the SATA disk (via the kernel drivers and SATA hardware).
So dd'ing to /dev/sda will copy data to the SATA disk.
Don't try this at home! - if you do this on your Linux host you will destroy your OS installation (although you will usually have to use sudo to execute the command).
The arguments are as follows (check the man page by doing man dd on the host machine, outside of QEMU, for more explanation of each):
| Argument | Meaning |
|---|---|
if=/dev/urandom | Use the special file urandom for source data. This file when read just generates random data from the kernel's built-in random number generator |
of=dev/sda | Use the SATA disk as the destination |
bs=1k | Copy data in 1kB chunks |
count=3 | Copy 3 chunks of data total |
The expected output should be something like:
3+0 records in
3+0 records out
hexdump the data again, to see our random data:
root@xilinx-zcu102-2020_2:~# hexdump -C /dev/sda -n 4096 | more
The output should be something like this. Note that your actual data will be different from this one, as it is randomized:
00000000 5d a3 a1 75 74 08 b8 01 c3 70 0d 4c 0f d7 5c a0 |]..ut....p.L..\.|
...
...
00000be0 88 e9 04 34 80 6d 81 7e c4 ba 5e 68 03 9b a5 60 |...4.m.~..^h...`|
00000bf0 36 ec 9d 95 f3 b9 7b 49 5f ea da 76 7d bf db c3 |6.....{I_..v}...|
00000c00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00001000
Copy-paste the data (or just the start of it) to notepad or a similar utility for future reference. Exit QEMU.
Back on the host machine (not in QEMU) hexdump the sata0.bin file:
hexdump -C sata0.bin -n 4096 | more
You should observe identical contents, as the SATA disk file is just a raw file containing the disk data:
00000000 5d a3 a1 75 74 08 b8 01 c3 70 0d 4c 0f d7 5c a0 |]..ut....p.L..\.|
00000010 13 b7 cf 74 6c 7c e0 51 1d 6f c9 a0 01 b1 15 03 |...tl|.Q.o......|
00000020 ed e1 f4 27 b9 fd 1f d8 48 44 0e aa 27 81 5f 07 |...'....HD..'._.|
00000030 5e 5d 08 4e 3c 7a 73 70 39 ed 1c f8 ef e8 79 18 |^].N<zsp9.....y.|
00000040 28 6b e7 a7 c3 f5 27 7b e9 75 bf 3e 70 ec 16 19 |(k....'{.u.>p...|
00000050 f9 af 1b 72 a6 0e 25 e5 1f 0c f9 c9 b3 14 ae fa |...r..%.........|
...
00000be0 88 e9 04 34 80 6d 81 7e c4 ba 5e 68 03 9b a5 60 |...4.m.~..^h...`|
00000bf0 36 ec 9d 95 f3 b9 7b 49 5f ea da 76 7d bf db c3 |6.....{I_..v}...|
00000c00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00001000
Boot QEMU again and log in, and repeat the same hexdump command (first one, dumping /dev/sda).
You should observe the identical output. The same file contents have been populated from the first run.
Compressed Disk Images
2GB isn't very big and doesn't really test if our software can handle big disks. Creating large disks with the above method will use a lot of space. To resolve this problem, QEMU provides a compressed disk image format called QCOW2.
To create a QCOW2 image, run:
# qemu-img is located at build directory of QEMU. qemu-img create -f qcow sata0.qcow 8T
The qemu-img utility is also included as part of the PetaLinux tool suite. This is going to create an 8TB (don't panic) compressed image using the qcow2 image format. QCOW is an image format designed for use as virtual machine disk images. sata0.qcow is going to be the file holding the data. The expected output should be something like:
Formatting 'sata0.qcow', fmt=qcow size=8796093022208
Use ls to inspect the size of the newly created file, the expected output should look like:
ls -l sata0.qcow -rw-r--r-- 1 fnuv icdes 33554480 Feb 13 13:51 sata0.qcow
It only consumes 32MB so far, even though it is an 8TB image. This file size will grow as the disk accumulates meaningful data.
Start QEMU using this new disk image (instead of sata0.bin from before):
petalinux-boot --qemu --prebuilt 3 --qemu-args "-drive file=sata0.qcow,id=sata-drive -device ide-drive,drive=sata-drive,bus=ahci@0xFD0C0000.0"
The format=raw argument to the drive is dropped now. QEMU will auto-detect that we are using a QCOW image from the file header in sata0.qcow.
Check the boot log (same as before about halfway down) and see something like this:
[ 6.866559] ata1.00: configured for UDMA/100 [ 6.882732] scsi 0:0:0:0: Direct-Access ATA QEMU HARDDISK 2.5+ PQ: 0 ANSI: 5 [ 6.893114] sd 0:0:0:0: [sda] 17179869184 512-byte logical blocks: (8.80 TB/8.00 TiB) [ 6.896786] sd 0:0:0:0: [sda] Write Protect is off [ 6.899581] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
We can see the Linux kernel has found an 8TB SATA disk (third last line). Log in to the system using usual root/root credentials.
Like in the previous example, inspect the first 4kB of the SATA disk contents using the hexdump utility.
root@xilinx-zcu102-2020_2:~# hexdump -C /dev/sda -n 4096 | more # Again, it should be blank, this is the expected output: 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00001000
dd data, as before, but this time dd 1MB:
root@xilinx-zcu102-2020_2:~# dd if=/dev/urandom of=/dev/sda bs=1k count=1024
hexdump the first 4k (you could hexdump more, but 4k keeps the output brief):
root@xilinx-zcu102-2020_2:~# hexdump -C /dev/sda -n 4096 | more
The output should be something like this (your random data will differ):
00000f80 0d d8 8a 6f eb e7 e7 cf 87 d8 3e c1 6d 43 ff 2a |...o......>.mC.*| 00000f90 42 7a 77 85 d6 13 a3 10 4f 09 4a 0e e4 31 05 2d |Bzw.....O.J..1.-| 00000fa0 df 50 9d 98 26 5a 97 22 36 83 9d 20 ed 06 29 df |.P..&Z."6.. ..).| 00000fb0 1b 42 2c 5c ec 41 1a 2f 54 5c 27 f6 ab 71 1e 6f |.B,\.A./T\'..q.o| 00000fc0 0e 4b fd fc f9 d2 6d b9 f2 72 8f 56 5b ff ca f9 |.K....m..r.V[...| 00000fd0 a8 32 31 13 05 24 ad 5e d5 21 60 1a c0 81 e6 34 |.21..$.^.!`....4| 00000fe0 46 b2 b8 3b ac 75 5c 39 d7 50 7f 53 62 e2 fa 97 |F..;.u\9.P.Sb...| 00000ff0 73 de 07 93 08 d3 b1 ac e7 a1 94 cf ca 99 56 d7 |s.............V.| 00001000
Copy the output (or just a little bit of it) to notepad or similar for future reference. Exit QEMU. Inspect the size of the backing file:
ls -l sata0.qcow # The output should be something like: -rw-r--r-- 1 fnuv icdes 34611200 Feb 13 14:00 sata0.qcow
It's grown! - by about 1MB. This makes sense, as our DD command created 1MB of actual data that needs storage. Restart QEMU and log in.
Repeat the same hexdump command and compare it to the copied data from before. It should match. Exit QEMU.
We can use qemu-img to covert the compressed file back to a raw file, as 8TB is far too big to create as a raw file.
Multiple Disks
The SATA controller in QEMU has two ports. We can attach a data disk to each.
Let's create one more disk, called sata1.qcow. Note that this is just an arbitrary filename that is named to reduce confusion.
cp sata0.qcow sata1.qcow
Double up the command line arguments, with an extra -drive and -device for SATA disk 1:
petalinux-boot --qemu --prebuilt 3 --qemu-args "\
-drive file=sata0.bin,format=raw,id=sata-drive0 -device ide-drive,drive=sata-drive0,bus=ahci@0xFD0C0000.0 \
-drive file=sata1.qcow,id=sata-drive1 -device ide-drive,drive=sata-drive1,bus=ahci@0xFD0C0000.1"
Note that the id= arguments for each pair have been made unique. The second -device argument attaches to bus=ahci@0xFD0C0000.1 instead of .0, which will connect this SATA disk to the second port.
[ 6.921009] scsi 0:0:0:0: Direct-Access ATA QEMU HARDDISK 2.5+ PQ: 0 ANSI: 5 [ 6.931400] sd 0:0:0:0: [sda] 17179869184 512-byte logical blocks: (8.80 TB/8.00 TiB) [ 6.934314] sd 0:0:0:0: [sda] Write Protect is off [ 6.935904] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 6.953829] scsi 1:0:0:0: Direct-Access ATA QEMU HARDDISK 2.5+ PQ: 0 ANSI: 5 [ 6.960608] dwc3 fe200000.dwc3: Failed to get clk 'ref': -2 [ 6.962348] sd 1:0:0:0: [sdb] 17179869184 512-byte logical blocks: (8.80 TB/8.00 TiB) [ 6.962873] sd 1:0:0:0: [sdb] Write Protect is off [ 6.962968] dwc3 fe200000.dwc3: Configuration mismatch. dr_mode forced to gadget [ 6.964927] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
In the kernel boot log, we should see two SATA disks detected. The 2GB raw image is used for disk 0, enumerated as /dev/sda, and the 8TB QCOW image is used for disk 1, enumerated as /dev/sdb.
Log in to the system and hexdump /dev/sdb to see the data on the second SATA disk.
root@xilinx-zcu102-2020_2:~# hexdump -C /dev/sdb -n 4096 | more
Data should match the data from the QCOW disk image dd test (check your notepad pastes).
File Systems
Reboot the QEMU system and log in. Just boot with disk0 using the sata0.bin raw image:
petalinux-boot --qemu --prebuilt 3 --qemu-args "-drive file=sata0.bin,format=raw,id=sata-drive -device ide-drive,drive=sata-drive,bus=ahci@0xFD0C0000.0"
Log in as normal. From within the booted system, we are going to format the SATA disk with a partition table and single FAT partition. We use the fdisk utility on the /dev/sda node:
root@xilinx-zcu102-2020_2:~# fdisk /dev/sda Device contains neither a valid DOS partition table, nor Sun, SGI, OSF or GPT disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that the previous content won't be recoverable.
It created a new partition table for us. Print the partition table:
Command (m for help): p Disk /dev/sda: 2147 MB, 2147483648 bytes 255 heads, 63 sectors/track, 261 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System
The partition table contents are empty. Create a new partition, using the 'n' command. Select 'p' for primary, partition number 1, and use the defaults for the offset and sizes:
Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-261, default 1): 1 Last cylinder or +size or +sizeM or +sizeK (1-261, default 261): Using default value 261
Print the partition table again:
Command (m for help): p Disk /dev/sda: 2147 MB, 2147483648 bytes 255 heads, 63 sectors/track, 261 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 1 261 2096451 83 Linux
It now has a partition. Use the 't' command to set the partition type to FAT32. Use 'L' if you want to look at the options for partition types. Select partition #1 and set the Hex code to 'b'.
Command (m for help): t Selected partition 1 Hex code (type L to list codes): L 0 Empty 1b Hidden Win95 FAT32 9f BSD/OS 1 FAT12 1c Hidden W95 FAT32 (LBA) a0 Thinkpad hibernation 4 FAT16 <32M 1e Hidden W95 FAT16 (LBA) a5 FreeBSD 5 Extended 3c Part.Magic recovery a6 OpenBSD 6 FAT16 41 PPC PReP Boot a8 Darwin UFS 7 HPFS/NTFS 42 SFS a9 NetBSD a OS/2 Boot Manager 63 GNU HURD or SysV ab Darwin boot b Win95 FAT32 80 Old Minix b7 BSDI fs ... Hex code (type L to list codes): b Changed system type of partition 1 to b (Win95 FAT32)
Write the changes to disk using the w command:
Command (m for help): w The partition table has been altered. Calling ioctl() to re-read partition table [ 490.729264] sda: sda1
This will exit the fdisk program. We should now have a dev node for the partition:
root@xilinx-zcu102-2020_2:~# ls /dev/sda1 /dev/sda1
Make a FAT filesystem on our new partition.
root@xilinx-zcu102-2020_2:~# mkfs.vfat /dev/sda1
Mount the partition.
root@xilinx-zcu102-2020_2:~# mount /dev/sda1 /mnt
We can now copy files to and from /mnt directory. This will copy files to the emulated disk. Create a file in /mnt.
echo "Hello QEMU training world" >> /mnt/file
Sync filesystems:
root@xilinx-zcu102-2020_2:~# sync
Exit QEMU. hexdump the sata0.bin file with the following query (which will search for "FAT" string and some context around it). We should see the binary data for the partition table and the new FAT partition.
$ hexdump -C sata0.bin | grep FAT -A 100 -B 10 | more
Check the ASCII in the rightmost columns to see human-readable strings. We can see the "FAT32" magic string used to identify FAT partitions.
00000bd0 f4 62 02 18 94 aa 98 81 86 32 e6 84 74 f8 cb 31 |.b.......2..t..1|
00000be0 88 e9 04 34 80 6d 81 7e c4 ba 5e 68 03 9b a5 60 |...4.m.~..^h...`|
00000bf0 36 ec 9d 95 f3 b9 7b 49 5f ea da 76 7d bf db c3 |6.....{I_..v}...|
00000c00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00007e00 eb 58 90 6d 6b 64 6f 73 66 73 00 00 02 08 06 00 |.X.mkdosfs......|
00007e10 02 00 00 00 00 f8 00 00 3f 00 ff 00 00 00 00 00 |........?.......|
00007e20 86 fa 3f 00 f7 0f 00 00 00 00 00 00 02 00 00 00 |..?.............|
00007e30 01 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00007e40 00 01 29 c2 05 00 00 00 00 00 00 00 00 00 00 00 |..).............|
00007e50 00 00 46 41 54 33 32 20 20 20 0e 1f be 77 7c ac |..FAT32 ...w|.|
00007e60 22 c0 74 0b 56 b4 0e bb 07 00 cd 10 5e eb f0 32 |".t.V.......^..2|
00007e70 e4 cd 16 cd 19 eb fe 54 68 69 73 20 69 73 20 6e |.......This is n|
00007e80 6f 74 20 61 20 62 6f 6f 74 61 62 6c 65 20 64 69 |ot a bootable di|
00007e90 73 6b 0d 0a 00 00 00 00 00 00 00 00 00 00 00 00 |sk..............|
00007ea0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00406600 41 66 00 69 00 6c 00 65 00 00 00 0f 00 bc ff ff |Af.i.l.e........|
00406610 ff ff ff ff ff ff ff ff ff ff 00 00 ff ff ff ff |................|
00406620 46 49 4c 45 20 20 20 20 20 20 20 20 00 00 00 00 |FILE ....|
00406630 21 00 21 00 00 00 00 00 21 00 03 00 1a 00 00 00 |!.!.....!.......|
00406640 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00407600 48 65 6c 6c 6f 20 51 45 4d 55 20 74 72 61 69 6e |Hello QEMU train|
00407610 69 6e 67 20 77 6f 72 6c 64 0a 00 00 00 00 00 00 |ing world.......|
00407620 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
80000000
At the bottom here, we can even see the file data from our echo command as readable. Check the "Hello QEMU training world" string around address 407600 (your addresses may differ).
Restart QEMU and log in. Mount the /dev/sda1 partition again and cat the contents of the file we created in the previous boot to confirm our filesystem data was preserved across boots.
root@Xilinx-ZynqMP-EAApr2015:~# mount /dev/sda1 /mnt [ 43.396186] FAT-fs (sda1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck. root@Xilinx-ZynqMP-EAApr2015:~# cat /mnt/file Hello QEMU training world
You may get a warning about the volume not being unmounted from before.
© Copyright 2019 - 2022 Xilinx Inc. Privacy Policy