Security and Protection

01 Lab overview

This lab contributes to U12.1 through U12.7, and to course learning outcomes CLO6, CLO7, and CLO8.

02 Learning objectives

03 Background

3.1 Security goals and the threat model

The classical CIA triad defines three security goals: confidentiality (data is not disclosed to unauthorised parties), integrity (data is not modified without authorisation), and availability (data and services are accessible when needed). A threat model describes the adversary: what the adversary can do, what they cannot do, and what they are trying to achieve. Every security mechanism is designed against a threat model; a mechanism that is secure against one adversary may be useless against another.

3.2 Authentication

Authentication answers the question "who are you?". Linux supports three classes: something you know (password), something you have (a security token), something you are (biometrics). The password database is /etc/passwd (world-readable) for the account records, and /etc/shadow (root-readable only) for the password hashes. Hashes use a modern algorithm such as SHA-512 with a salt; the salt prevents precomputed rainbow tables and identical passwords produce different hashes. Pluggable Authentication Modules (PAM) allow the authentication mechanism to be changed per service without modifying the applications.

3.3 Authorization and access control

Authorization answers "what are you allowed to do?".

• Discretionary access control (DAC). The owner of a resource decides who can access it. Linux's classic mode bits (rwx for owner, group, and others) and POSIX ACLs are DAC.
• Mandatory access control (MAC). The system enforces a global policy that even the owner cannot override. SELinux and AppArmor are MAC systems: each process and each object has a label, and the policy defines which labels may interact.
• Role-based access control (RBAC). Permissions are assigned to roles, and users are assigned to roles. sudo is a form of RBAC: /etc/sudoers defines which users may run which commands as which other users.

3.4 Privilege escalation

Privilege escalation is any mechanism by which a process gains more privileges than it should have. The classic vector is a setuid binary: a program owned by root and marked setuid runs with root's effective UID, regardless of who invokes it. If the program is poorly written — for example, it invokes another program without resetting its environment, or it writes to a file whose path is controlled by the caller — an attacker can leverage the setuid binary to run arbitrary commands as root. Modern Linux mitigates this with capabilities: instead of granting all of root's privileges, a setuid binary can be granted only the specific capabilities it needs (for example, CAP_NET_BIND_SERVICE to bind a low port). The rest of root's privileges are not available.

3.5 Sandboxing

A sandbox isolates a process so that even if it is compromised, it cannot affect the rest of the system. Linux provides several mechanisms:

• Namespaces: a process can have its own view of the file system (mount namespace), process IDs (PID namespace), network (network namespace), user IDs (user namespace), and so on. Containers are built on namespaces.
• seccomp: a filter that restricts which system calls the process may make. A compromised process cannot invoke a system call that the filter denies.
• cgroups: resource limits on CPU, memory, I/O, and device access.
• AppArmor / SELinux: MAC policies that restrict what files, network ports, and other resources the process may access.

3.6 Hardening

Hardening is the process of reducing a system's attack surface. Standard measures: disable unused services, apply the principle of least privilege to every process, restrict capabilities, enable auditing, keep the system patched, and enable MAC policies in enforcing mode. The goal is not to make a system impenetrable — that is impossible — but to make a successful attack require more effort and to limit the damage that a successful attack can do.

04 Guided activities

Activity 4.1 — Authentication and the shadow file

1. Create the lab directory:

   mkdir -p ~/comp3150/lab11 && cd ~/comp3150/lab11

2. Inspect /etc/passwd and /etc/shadow. In the passwd file, identify the fields: username, password placeholder, UID, GID, GECOS, home directory, login shell. In the shadow file (with sudo), identify the fields: username, hashed password, last change, minimum age, maximum age, warning period, inactivity, expiry.
3. Create a test user with sudo useradd -m testuser and set a password with sudo passwd testuser. Inspect the new entries in both files.
4. Verify that the hash in /etc/shadow is salted and hashed: run sudo cat /etc/shadow | grep testuser and identify the algorithm identifier ($6$ for SHA-512), the salt, and the hash. Change the password to the same value and verify that the hash is different (a new salt was chosen).
5. Configure a password policy in /etc/security/pwquality.conf: minimum length 12, at least one digit, at least one uppercase. Attempt to change the password to a weak one and observe the rejection.

Activity 4.2 — Authorization with DAC and ACLs

1. Create a file secret.txt with echo "top secret" > secret.txt. Set its mode to 600 with chmod 600 secret.txt. Verify with ls -l and stat.
2. As testuser (use su - testuser or sudo -u testuser cat secret.txt), attempt to read the file. Observe the permission denied error.
3. Grant testuser read access with a POSIX ACL:

   sudo setfacl -m u:testuser:r secret.txt
   getfacl secret.txt

   Verify that testuser can now read the file, and that ls -l shows a + after the mode bits indicating an ACL is present.
4. Set the default ACL on a directory so that all new files inherit the permission:

   sudo setfacl -d -m u:testuser:r somedir/

   Create a file in the directory and verify the inherited ACL.
5. Remove the ACL with setfacl -x u:testuser secret.txt. Verify that testuser can no longer read the file. Explain the difference between the mode bits and the ACL: the mode bits grant permissions to one owner, one group, and everyone else; the ACL allows arbitrary users and groups to be granted specific permissions.

Activity 4.3 — Privilege escalation with setuid

1. Write a program vuln.c that runs with setuid root and reads a file whose path is supplied on the command line:

   #include <stdio.h>
   #include <stdlib.h>
   
   int main(int argc, char **argv) {
       if (argc != 2) { fprintf(stderr, "usage: %s <file>\n", argv[0]); return 1; }
       FILE *f = fopen(argv[1], "r");
       if (!f) { perror("fopen"); return 1; }
       char buf[256];
       while (fgets(buf, sizeof buf, f)) fputs(buf, stdout);
       fclose(f);
       return 0;
   }

2. Compile it, change its owner to root, and set the setuid bit:

   gcc -o vuln vuln.c
   sudo chown root:root vuln
   sudo chmod 4755 vuln
   ls -l vuln

3. As testuser, run ./vuln /etc/shadow. The program reads the shadow file even though testuser does not have permission — because the process's effective UID is root, not testuser. This is the intended use of setuid: the program is trusted to decide which files to read.
4. Now modify the program to invoke a shell command using system():

   system(argv[1]);

   Compile, mark setuid root, and as testuser run ./vuln "/bin/sh". You get a root shell. This is a classic setuid vulnerability: the program trusts its caller to supply a safe command. Explain why using system() or execlp() in a setuid program is dangerous, and how to avoid it (drop privileges before invoking external programs, or use execve with a full path and a sanitised environment).
5. Remove the setuid bit and demonstrate that the same command now fails. Explain the mitigation with capabilities: instead of setuid root, use setcap cap_dac_read_search+ep ./vuln to grant only the capability to bypass file read permissions.

Activity 4.4 — Sandboxing a process

1. Run a simple command in a new set of namespaces:

   sudo unshare --mount --uts --ipc --pid --fork --mount-proc /bin/bash

   Inside the new shell, run ps aux and observe that only a few processes are visible (the PID namespace is isolated). Run hostname newhost and observe that the hostname change does not affect the outer system (the UTS namespace is isolated).
2. Use seccomp to restrict a process to a small set of system calls. Write a C program that installs a seccomp filter allowing only read, write, exit, and exit_group, then attempts to open a file. The program should be killed by SIGSYS when it tries the denied call.
3. Use systemd-run to launch a process with resource and access restrictions:

   sudo systemd-run --unit=locked --property=ProtectHome=yes \
     --property=ProtectSystem=strict --property=NoNewPrivileges=yes \
     --property=PrivateTmp=yes /bin/bash

   Inside the unit, attempt to read files in /home and /root, and observe the denials.
4. If firejail or bubblewrap is available, use it to run a browser or a text editor in a sandbox. Inspect the sandbox's mount namespace with ls /proc/<pid>/ns/ and compare it to the outer system's.

Activity 4.5 — Hardening a Linux system

1. List the services that are enabled at boot:

   systemctl list-unit-files --state=enabled

   For each service, decide whether it is necessary. Disable a service you do not need:

   sudo systemctl disable --now <service>

2. Check the capabilities of the setuid binaries on the system:

   find / -perm -4000 -type f 2>/dev/null | xargs ls -l

   For each one, verify that it is necessary and that it has been updated with the latest security patches. Remove the setuid bit from any that are not needed.
3. Enable a MAC policy. If your system uses AppArmor:

   sudo aa-status
   sudo systemctl enable --now apparmor

   If it uses SELinux:

   getenforce
   sudo setenforce 1

   Verify that the policy is in enforcing mode and that a test program is denied access to a resource it would otherwise reach.
4. Enable auditing for sensitive operations:

   sudo systemctl enable --now auditd
   sudo auditctl -w /etc/passwd -p wa -k passwd_changes
   sudo auditctl -w /etc/shadow -p wa -k shadow_changes

   Modify /etc/passwd (for example, change a user's shell) and inspect the audit log with sudo ausearch -k passwd_changes.
5. Configure a minimal sudoers entry that allows a user to run only one specific command as root, and verify that other commands are denied:

   testuser ALL=(root) NOPASSWD: /usr/bin/systemctl restart nginx

   From testuser, run the permitted command and observe success; run a different command and observe denial.

05 Exercises

06 Deliverables

Submit a single archive named lab11-<lastname>-<studentnumber>.tar.gz containing:

• vuln.c and vuln_fixed.c — Activity 4.3.
• seccomp_test.c — Activity 4.4.
• password_policy.c — Exercise 1.
• toctou.c and toctou_attack.sh — Exercise 3.
• sudoers.example — Exercise 2.
• container.sh — Exercise 5.
• hardening.md — Exercise 7.
• outputs.txt — the output logs of all commands with the commands that produced them.
• report.md (or report.pdf) — the lab report described below.

Lab report

The report should be 5–7 pages (excluding code) and contain:

1. Authentication. The /etc/passwd and /etc/shadow entries for your test user, with the hash fields annotated. The password policy configuration and the demonstration of a rejected weak password.
2. Authorization. The DAC and ACL demonstrations, with the ls -l and getfacl output for each step, and an explanation of when to use ACLs in preference to mode bits.
3. Privilege escalation. The vuln.c program, the root shell demonstration, and the mitigation. A clear explanation of why setuid programs must not invoke external commands with a sanitised environment, and how capabilities improve on setuid root.
4. Sandboxing. The namespace demonstrations, the seccomp filter, and the systemd restrictions. A table listing each isolation mechanism and the resources it protects.
5. Hardening. The list of services you disabled, the setuid binaries you removed or verified, the MAC policy you enabled, the audit rules you configured, and the RBAC policy you wrote.
6. Reflection. A discussion of the principle of least privilege and its practical limits: why most real systems grant more privilege than strictly necessary, what the trade-off is, and how modern sandboxing and capability mechanisms try to reduce the gap between the privilege a process has and the privilege it needs.

07 Grading rubric

Late submission penalty: 10% per day, up to a maximum of three days, after which the lab is accepted only with documented extenuating circumstances.

08 Knowledge check

09 References

• Silberschatz, A., Galvin, P. B., & Gagne, G. Operating System Concepts, 10th ed., Chapter 16.
• Tanenbaum, A. S., & Bos, H. Modern Operating Systems, 4th ed., Chapter 9.
• Saltzer, J. H., & Schroeder, M. D. The Protection of Information in Computer Systems. Proceedings of the IEEE, 1975.
• Anderson, J. P. Computer Security Technology Planning Study. USAF Electronic Systems Division, 1972 (the origin of the reference monitor concept).
• Smalley, S., Vance, C., & Salamon, W. Implementing SELinux as a Linux Security Module. NSA Technical Report, 2001.
• Bauer, M. Paranoid Penguin: Linux Security. Linux Journal (multiple articles).
• Linux man-pages: man 5 passwd, man 5 shadow, man 5 sudoers, man 7 capabilities, man 2 seccomp, man 7 namespaces, man 8 setfacl, man 8 setcap.
• Linux kernel documentation: Documentation/admin-guide/LSM/, Documentation/userspace-api/seccomp_filter.rst.