Containers and Virtualization

01 Lab overview

This lab contributes to U13.1 through U13.7, and to course learning outcomes CLO6, CLO7, and CLO8.

02 Learning objectives

03 Background

3.1 Virtual machines and hypervisors

A virtual machine (VM) is a software emulation of a complete computer, including a CPU, memory, disk, and network interfaces. The software that creates and runs VMs is the hypervisor (or virtual machine monitor, VMM). There are two types:

• Type 1 (bare-metal): runs directly on the hardware and hosts guest operating systems. Examples: KVM (with Linux as the host), Xen, VMware ESXi, Microsoft Hyper-V.
• Type 2 (hosted): runs as an application on top of a host operating system. Examples: VirtualBox, VMware Workstation, QEMU in user-mode emulation.

The key advantage of virtualisation is isolation: each guest OS runs in its own VM and cannot affect the others, even if it crashes or is compromised. The cost is overhead: every privileged instruction in the guest must be trapped and emulated by the hypervisor, and every memory access goes through an extra level of translation.

3.2 CPU virtualisation

Before hardware support, the hypervisor had to emulate privileged instructions in software, which was slow. Modern CPUs provide hardware-assisted virtualisation: Intel VT-x and AMD-V add a new privilege level (root mode) in which the hypervisor runs, and guest kernels run in non-root mode. Privileged instructions in the guest trap to the hypervisor, which handles them and resumes the guest. The guest executes most instructions natively, so the overhead is low — typically a few percent for CPU-bound workloads.

3.3 Memory virtualisation

A guest OS maintains its own page tables mapping guest-virtual to guest-physical addresses. The hypervisor maintains a second set of tables mapping guest-physical to host-physical addresses. Without hardware support, the hypervisor had to shadow the guest's page tables, updating them on every guest page-table change — expensive. Modern CPUs provide nested paging (Intel EPT, AMD NPT), which performs the two-level translation in hardware on every access. The hardware also provides a nested TLB to cache the combined translation.

3.4 I/O virtualisation

I/O is the hardest part of virtualisation because devices are numerous and diverse. Three approaches:

• Emulation: the hypervisor emulates a well-known device (for example, an Intel e1000 network card) in software. Portable but slow.
• Paravirtualisation: the guest is modified to use a hypervisor-aware device driver. The virtio framework is the standard in Linux: a virtio-net or virtio-blk device is much faster than an emulated device because the guest and hypervisor communicate through a shared ring buffer.
• Device passthrough: the hypervisor assigns a physical device directly to a guest (via VT-d or AMD-Vi IOMMU). The guest uses the device's real driver, achieving near-native performance, at the cost of dedicating the device to one guest. SR-IOV extends this by allowing a single physical device to present multiple virtual functions, each assignable to a different guest.

3.5 Containers

A container is not a virtual machine. It is a group of processes on the host, isolated by kernel features:

• Namespaces isolate the container's view of the system: mount, PID, network, UTS, IPC, user, and cgroup namespaces.
• cgroups limit the container's resource usage: CPU, memory, I/O, and device access.
• Capabilities restrict what the container's processes can do, even as root inside the container.
• LSM policies (SELinux, AppArmor) provide an additional layer of confinement.

Containers share the host kernel, so they start in milliseconds and have minimal overhead, but their isolation is weaker than a VM's: a kernel vulnerability can allow a container escape.

3.6 Container images and Docker

A container image is a layered file system: each layer is a set of file changes, and the layers are stacked to form the container's root file system. Images are built from a Dockerfile, which specifies the base image, the files to copy, the commands to run, and the entry point. Images are stored in a registry (Docker Hub, GitHub Container Registry, a private registry) and pulled on demand. At runtime, the container's file system is a union of the image's layers plus a writable layer for the container's own changes.

3.7 Orchestration

Container orchestration is the management of many containers across many hosts. It handles scheduling (deciding which host runs which container), service discovery, load balancing, rolling updates, health checks, and scaling. Docker Compose orchestrates containers on a single host; Kubernetes orchestrates containers across a cluster. The core concepts — declarative configuration, desired state, reconciliation — are shared. Kubernetes has become the de facto standard for production container orchestration and is the foundation of most modern cloud platforms.

3.8 Cloud operating systems

A cloud operating system is the software layer that turns a data centre's worth of physical machines into a pool of virtual resources. It provides multi-tenancy (many customers sharing the same hardware, isolated from each other), elasticity (resources scale up and down with demand), fault tolerance (failures are detected and recovered), and live migration (moving a running VM from one host to another without downtime). Examples include OpenStack, VMware vCloud, and the proprietary control planes of AWS, Azure, and Google Cloud. The challenge is to provide all of this while maintaining security, performance, and cost efficiency at data-centre scale.

04 Guided activities

Activity 4.1 — CPU virtualisation support

1. Create the lab directory:

   mkdir -p ~/comp3150/lab12 && cd ~/comp3150/lab12

2. Verify that your CPU supports hardware virtualisation:

   grep -E 'vmx|svm' /proc/cpuinfo | head -1

   The presence of vmx (Intel) or svm (AMD) indicates support. If neither appears, your VM may not have nested virtualisation enabled; check with your hypervisor's documentation.
3. Verify that the KVM kernel modules are loaded:

   lsmod | grep kvm
   ls -l /dev/kvm

   The /dev/kvm device should exist and be accessible. If not, load the modules with sudo modprobe kvm_intel (or kvm_amd).
4. Check the CPU's virtualisation capabilities with lscpu. Identify the flags related to virtualisation (nested paging, tagged TLB, etc.).

Activity 4.2 — Creating a virtual machine with QEMU/KVM

1. Install QEMU and the tools:

   sudo apt update && sudo apt install -y qemu-system-x86 qemu-utils cloud-image-utils

2. Download a cloud image (for example, Ubuntu 22.04 cloud image):

   wget https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img

3. Create a cloud-init seed ISO to configure the VM with a user and an SSH key:

   cat > user-data <<'EOF'
   #cloud-config
   users:
     - name: student
       sudo: ALL=(ALL) NOPASSWD:ALL
       ssh_authorized_keys:
         - $(cat ~/.ssh/id_rsa.pub)
   EOF
   cat > meta-data <<'EOF'
   instance-id: lab12-vm
   local-hostname: lab12
   EOF
   cloud-localds seed.img user-data meta-data

4. Resize the image to give the VM more disk space:

   qemu-img resize jammy-server-cloudimg-amd64.img +10G

5. Boot the VM with KVM acceleration:

   qemu-system-x86_64 \
     -enable-kvm \
     -m 2048 \
     -smp 2 \
     -cpu host \
     -drive file=jammy-server-cloudimg-amd64.img,if=virtio \
     -drive file=seed.img,if=virtio,format=raw \
     -netdev user,id=net0,hostfwd=tcp::2222-:22 \
     -device virtio-net-pci,netdev=net0 \
     -nographic

6. From another terminal, SSH into the VM:

   ssh -p 2222 student@localhost

   Once inside, verify the VM's kernel and the virtualised devices:

   uname -a
   lspci
   lscpu | grep -i hypervisor

7. Shut down the VM with sudo poweroff from inside, or by typing Ctrl-A X at the QEMU console.

Activity 4.3 — Building a container from scratch

1. Create a minimal root file system using debootstrap:

   sudo debootstrap --variant=minbase jammy rootfs http://archive.ubuntu.com/ubuntu/

   (If debootstrap is not available, use busybox or extract a Docker image's file system with docker export.)
2. Write a shell script mini-container.sh that launches a container using unshare and chroot:

   #!/bin/bash
   set -e
   ROOTFS="$1"
   shift
   exec unshare \
     --mount --uts --ipc --pid --fork \
     --mount-proc \
     chroot "$ROOTFS" "$@"

3. Make it executable and run a shell in the container:

   chmod +x mini-container.sh
   sudo ./mini-container.sh ./rootfs /bin/bash

4. Inside the container:
   • Run ps aux. Only the processes inside the container's PID namespace are visible. The container sees itself as PID 1.
   • Run hostname container. The hostname change affects only the container's UTS namespace.
   • Run mount. Only the container's mount namespace is visible.
   • Attempt to access the host's /home or /etc/shadow. The chroot confines the process to the container's root file system.
5. From another terminal on the host, list the namespaces of the container's process:

   ls -l /proc/<pid>/ns/

   Compare the namespace inode numbers to those of the host's init process. Each namespace (mnt, uts, ipc, pid, net, user, cgroup) has a different inode, showing that the container is in its own namespace.
6. Add a cgroup limit to the script: limit the container to 50% of one CPU and 256 MB of memory using systemd-run --scope -p CPUQuota=50% -p MemoryMax=256M or by writing to /sys/fs/cgroup/ directly. Verify that a CPU-bound or memory-hungry process inside the container is limited.

Activity 4.4 — Docker: building and running images

1. Install Docker:

   sudo apt install -y docker.io docker-compose-v2
   sudo usermod -aG docker $USER
   newgrp docker

2. Run a simple container:

   docker run --rm hello-world
   docker run --rm -it ubuntu:22.04 bash
   docker run --rm -d --name web -p 8080:80 nginx
   curl http://localhost:8080/

3. Write a Dockerfile for a simple Python web service:

   FROM python:3.11-slim
   WORKDIR /app
   COPY requirements.txt .
   RUN pip install --no-cache-dir -r requirements.txt
   COPY app.py .
   EXPOSE 5000
   CMD ["python", "app.py"]

   Write app.py (a minimal Flask app) and requirements.txt (with flask).
4. Build and run the image:

   docker build -t myapp:1.0 .
   docker run --rm -d --name myapp -p 5000:5000 myapp:1.0
   curl http://localhost:5000/

5. Inspect the image layers with docker history myapp:1.0. Explain how the layers are stacked and why the order of commands in a Dockerfile affects caching.
6. Measure the container's overhead with docker stats while a workload runs. Compare the memory footprint of the container to that of the same application running on the host.

Activity 4.5 — Orchestration with Docker Compose

1. Write a docker-compose.yml that runs two services: a web application (the one from Activity 4.4) and a Redis instance for caching:

   services:
     web:
       build: .
       ports:
         - "5000:5000"
       environment:
         REDIS_HOST: redis
       depends_on:
         - redis
     redis:
       image: redis:7-alpine
       volumes:
         - redis-data:/data
   
   volumes:
     redis-data:

2. Modify app.py to increment a counter stored in Redis on every request. The web container connects to Redis using the hostname redis, which Compose resolves through the internal network.
3. Start the stack:

   docker compose up -d
   docker compose ps
   curl http://localhost:5000/
   curl http://localhost:5000/
   docker compose logs web

   Each request should show an incremented counter.
4. Scale the web service:

   docker compose up -d --scale web=3

   Observe that Compose cannot publish port 5000 three times — this is why scaling requires a load balancer in front. Describe how Kubernetes solves this problem with a Service and a load balancer.
5. Bring the stack down:

   docker compose down -v

05 Exercises

06 Deliverables

Submit a single archive named lab12-<lastname>-<studentnumber>.tar.gz containing:

• mini-container.sh — Activity 4.3.
• Dockerfile, app.py, requirements.txt — Activity 4.4.
• docker-compose.yml and the extended application — Activity 4.5.
• Dockerfile.optimised — Exercise 6.
• outputs.txt — the output logs of all commands with the commands that produced them (including docker history, docker stats, lscpu, and the QEMU boot output).
• report.md (or report.pdf) — the lab report described below.

Lab report

The report should be 6–8 pages (excluding code) and contain:

1. Virtualisation support. The lscpu output showing the virtualisation flags, and a description of how VT-x/AMD-V, nested paging, and virtio each contribute to VM performance.
2. QEMU/KVM VM. The QEMU command line, the cloud-init configuration, and the output of lscpu and lspci inside the VM. An explanation of each QEMU option.
3. Container from scratch. The mini-container.sh script, the namespace listing, the cgroup configuration, and a demonstration of the isolation (visible processes, hostname, mount points, resource limits).
4. Docker. The Dockerfile, the image layer history, the container statistics, and a discussion of how layers enable caching and sharing.
5. Compose orchestration. The Compose file, the running services, and the demonstration of the counter in Redis. A discussion of how Compose's single-host model extends to Kubernetes's cluster model.
6. Performance comparison. The VM vs. container vs. host measurements from Exercises 1 and 2, presented as a table with overhead percentages, and a discussion of the trade-offs between isolation and performance.
7. Reflection. A discussion of when a virtual machine is the right choice and when a container is, and how modern cloud platforms combine both (for example, running containers inside VMs for stronger isolation). A brief discussion of the future of virtualisation: microVMs (Firecracker, Kata Containers), unikernels, and WebAssembly runtimes.

07 Grading rubric

Late submission penalty: 10% per day, up to a maximum of three days, after which the lab is accepted only with documented extenuating circumstances.

08 Knowledge check

09 References

• Silberschatz, A., Galvin, P. B., & Gagne, G. Operating System Concepts, 10th ed., Chapter 18.
• Tanenbaum, A. S., & Bos, H. Modern Operating Systems, 4th ed., Chapter 7.
• Smith, J. E., & Nair, R. Virtual Machines: Versatile Platforms for Systems and Processes. Morgan Kaufmann, 2005.
• Rosenblum, M., & Garfinkel, T. Virtual Machine Monitors: Current Technology and Future Trends. IEEE Computer, 2005.
• Mergen, M. F., Uhlig, V., Krieger, O., & Xenidis, J. Virtualization for High-Performance Computing. ACM SIGOPS Operating Systems Review, 2006.
• Agache, A., Brooker, M., Iordache, A., et al. Firecracker: Lightweight Virtualization for Serverless Applications. USENIX NSDI, 2020.
• Bernstein, D. Containers and Cloud: From LXC to Docker to Kubernetes. IEEE Cloud Computing, 2014.
• Burns, B., Grant, B., Oppenheimer, D., Brewer, E., & Wilkes, J. Borg, Omega, and Kubernetes. ACM Queue, 2016.
• Manco, F., Lupu, C., Schmidt, F., et al. My VM is Lighter (and Safer) than your Container. ACM SOSP, 2017.
• Linux man-pages: man 7 namespaces, man 7 cgroups, man 1 unshare, man 1 chroot, man 1 docker, man 1 qemu-system-x86_64.