Module 4 · Unit 10 · Discussion 10
Anatomy of a File-System Failure
A file system is the component of an operating system most likely to lose data, and its failures are the most visible to users. Modern file systems invest heavily in crash consistency — journaling, copy-on-write, checksums — but those mechanisms are based on assumptions about the hardware, the workload, and the failure mode. When an assumption breaks, so does the file system. This discussion asks you to analyse a documented file-system failure, identify the broken assumption, and evaluate the fix that was applied.
01 Overview
| Course | COMP 3150 / CMIS 3150 — Operating Systems |
|---|---|
| Unit | Unit 10 — File Systems |
| Weight | Part of the 5% participation grade (each discussion is worth approximately 0.4% of the final grade) |
| Opens | Monday of Week 10 |
| Initial post due | Thursday of Week 10, 23:59 |
| Peer replies due | Sunday of Week 10, 23:59 |
| Format | One initial post (550–750 words) and at least two substantive peer replies (150–250 words each) |
| CLOs assessed | CLO5, CLO6, CLO7 |
02 Learning outcomes addressed
By participating in this discussion you will
- Explain the file abstraction, the inode, and the directory (U10.1–U10.3).
- Compare file-allocation strategies — contiguous, linked, indexed, extents (U10.5).
- Explain journaling and copy-on-write, and the crash-consistency guarantees each provides (U10.6).
- Analyse a real file-system failure and identify the assumption that broke (U10.7).
- Evaluate a modern file system's design against the failures it was built to prevent (U10.7).
This discussion assesses CLO5 (explain file systems, storage, and I/O), CLO6 (apply Linux tools), and CLO7 (evaluate operating-system technologies).
03 Discussion prompt
Choose a documented file-system failure from a public source. Good candidates include:
- Ext4 data loss (2009). A change to the default writeback behaviour caused files to be truncated on crash when they were created, written, and closed shortly before the crash. The problem was not the journaling — the metadata was consistent — but the ordering between data and metadata.
- Btrfs RAID 5/6 write hole. A copy-on-write file system on top of a RAID 5 array can leave the array in an inconsistent state if a power failure occurs during a stripe update. The write hole is a fundamental property of RAID 5/6, and Btrfs (until recently) did not close it.
- ZFS checksum failure. Documented cases where a ZFS pool reported unrecoverable checksum errors — sometimes because of a firmware bug in the drive, sometimes because the pool's redundancy was insufficient for the number of simultaneous failures.
- APFS (macOS) file-system bugs. Multiple reports of APFS corruption after crashes, typically on external or non-Apple-formatted drives.
- Any documented bug in the file-system code of a Linux distribution, a BSD, or a commercial system, from a bug tracker, a mailing-list thread, or a post-mortem report.
Your task
- Describe the failure. State the file system, the version, the workload, and the failure mode (power loss, kernel crash, hardware fault, software bug). Explain what the user observed and what the file system did.
- Identify the design assumption that broke. Every file-system design is built on assumptions: that the hardware writes sectors in order, that a flush returns when data is durable, that the drive's cache is volatile or non-volatile, that the failure mode is a power loss and not a torn write, and so on. State which assumption your chosen failure violated.
- Classify the failure. Is it a journaling failure, a copy-on-write failure, a metadata-consistency failure, a data-loss failure, or something else? Was the file system's metadata consistent after the failure, even if the data was not? Distinguish the two.
-
Explain the fix. What did
the developers change? (For ext4: changing
the default to
data=ordered, adding flush barriers, and issuing afsyncon close for newly created files. For Btrfs: warning against RAID 5/6, and later adding a partial write hole fix.) Why does the fix eliminate the failure? What does it cost? -
Reflect on what the failure teaches.
Consider:
- Why was the failure not caught during testing? What made it depend on timing or on a particular hardware configuration?
- What does the failure reveal about the limits of journaling? Of copy-on-write? Of any single consistency mechanism?
- How do modern file systems (ZFS, Btrfs) try to catch this class of failure before it causes data loss? (Checksums, scrub, replication, and the ability to detect and repair silent corruption.)
- What is the right defence for a user or an administrator against this class of failure? (Backups, replication, monitoring, filesystem selection.)
Your post should be 550–750 words. Cite the failure's public documentation (a bug report, a mailing-list thread, a LWN article, a vendor advisory) and at least one additional source (a textbook section, a paper, or the file system's documentation).
04 Notes on the failure modes
The following notes may help you frame your analysis.
4.1 Journaling and ordering
A journaling file system records metadata changes
in a log before applying them. If the system
crashes between the log write and the apply, the
log is replayed on the next mount, bringing the
metadata back to a consistent state. What
journaling does not guarantee is that the
data blocks referenced by the metadata have
actually been written. If the metadata is
journaled before the data reaches the disk, the
file system can be consistent while individual
files contain stale or garbage content. Ext4's
data=ordered mode (the default after
2009) closes this gap by writing data blocks
before the metadata that references them.
4.2 The RAID 5 write hole
RAID 5 writes each logical block as a data block plus a parity block. If the system crashes after the data block is written but before the parity block, the stripe is inconsistent: the parity no longer matches the data. A subsequent read of either block may return correct data, but a rebuild after a disk failure will reconstruct the wrong content. The "write hole" is a property of the RAID 5 algorithm itself, not of any particular file system. A copy-on-write file system layered on RAID 5 inherits the hole unless it takes additional steps (for example, writing the full stripe, or using a log-structured layout).
4.3 Silent corruption and checksums
A disk may return the wrong data without raising an error — a condition called silent data corruption. Traditional file systems cannot detect it: they read whatever the disk returns and trust it. ZFS and Btrfs store a checksum with each block, so silent corruption is detected on read. If the block is replicated or parity-protected, the file system can reconstruct the correct data from a good copy. This is one of the strongest arguments for checksumming file systems on hardware that is not fully trusted.
4.4 What is not a file-system failure
Be careful to distinguish a file-system bug from a hardware failure, a driver bug, or a configuration error. A disk that fails mechanically is not a file-system failure; a misconfigured mount option is not a file-system failure. Choose a failure where the file system's design or implementation is the cause.
05 Grading rubric
The discussion is marked out of 100 points. The criteria and their weights are given below.
| Criterion | Weight |
|---|---|
| Post quality (description, assumption, fix, reflection) | 40% |
| Peer response quality | 30% |
| Engagement and timeliness | 20% |
| Netiquette and clarity | 10% |
5.1 Post quality (40 points)
| Level | Descriptor |
|---|---|
| Exemplary (90–100%) | A documented failure, described clearly and accurately. The design assumption that broke is identified explicitly. The failure is classified correctly, with the metadata-versus-data distinction preserved. The fix is explained and its cost is stated. The reflection connects the failure to the general principles of crash consistency and to the file-system features that are designed to prevent it. Sources are cited. |
| Proficient (80–89%) | A documented failure, described correctly. The assumption is identified. The classification is correct. The fix is explained but its cost is not. The reflection is present but brief. |
| Developing (70–79%) | A failure is described, but the description is vague or the assumption is not identified. The classification is incorrect. The fix is described without explanation. The reflection is superficial. |
| Beginning (60–69%) | The failure is not real (invented or generic), or the description is inaccurate. The assumption, classification, and fix are missing or incorrect. |
| Not demonstrated (<60%) | The post is missing, off-topic, or a restatement of the prompt without a real failure. |
5.2 Peer response quality (30 points)
The descriptors are the same as for Discussion 1. An exemplary reply might compare the classmate's failure with a similar one in a different file system, challenge the classification, or suggest an additional reflection on the limits of the file system's design.
5.3 Engagement and timeliness (20 points)
The descriptors are the same as for Discussion 1.
5.4 Netiquette and clarity (10 points)
The descriptors are the same as for Discussion 1.
06 Resources
- Silberschatz, A., Galvin, P. B., & Gagne, G. Operating System Concepts, 10th ed., Chapters 13–15 (File Systems).
- Arpaci-Dusseau, R. H., & Arpaci-Dusseau, A. C. Operating Systems: Three Easy Pieces, Chapters 39–43 (File Systems, FSCK and Journaling).
- Tanenbaum, A. S., & Bos, H. Modern Operating Systems, 4th ed., Chapter 4 (File Systems).
- McKusick, M. K., Joy, W. N., Leffler, S. J., & Fabry, R. S. A Fast File System for UNIX. ACM TOCS, 1984.
- Rosenblum, M., & Ousterhout, J. K. The Design and Implementation of a Log-Structured File System. ACM TOCS, 1992.
- Pillai, T. S., et al. All File Systems Are Not Created Equal: On the Complexity of Crafting Crash-Consistent Applications. USENIX OSDI, 2014.
- LWN articles on the ext4 data-loss incident (2009) and on the Btrfs RAID 5/6 write hole.
- Linux man-pages:
man 5 ext4,man 8 mount,man 8 fsck,man 8 debugfs,man 8 dumpe2fs.