Coerced Cache Eviction

Modern file system carefully order disk writes to maintain consistency. However, on disks that have write caches, the write order is not preserved when the disk destages the writes from the cache to the non-volatile surface. Writes A, B, C may be later destaged in order C, A, B. A crash in the middle of destaging results in the loss of file-system consistency.

On such disks, ordering is achieved by issuing a set of disk writes, flushing the disk cache, and issuing a second set of writes. The first set of writes is then ordered before the second set of writes. Therefore file-system consistency depends upon disks properly flushing the disk cache when asked to do so. However, the disk may not correctly flush the case upon request. This may happen for a number of reasons. There may be bugs in the disk firmware. Some disks are also known to lie about flushing the data in the cache (as shown in the following snippet from the MAC OSX Manual Page for fsync).

F_FULLFSYNC: Does the same thing as fsync(2) then asks the drive to flush all buffered data to the permanent storage device (arg is ignored). This is currently implemented on HFS, MS-DOS (FAT), and Universal Disk Format (UDF) file systems. The operation may take quite a while to complete. Certain FireWire drives have also been known to ignore the request to flush their buffered data.

This increases the performance of the drive, at the risk of losing file-system consistency in the event of a crash. Coerced Cache Eviction is a technique that allows file systems to maintain consistency while running on such misbehaving disks. The technique involves characterizing the disk through microbenchmarks, and writing extra data (from a flush zone) at each ordering point to hopefully force the data that we care about from the disk cache onto the non-volatile surface of the disk.

Related Publications

Coerced Cache Eviction: Dealing with Misbehaving Disks through Discreet-Mode Journaling
Abhishek Rajimwale, Vijay Chidambaram, Deepak Ramamurthi, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau
DSN 2011
Abstract   PDF   BibTex   Talk PPT   Talk Slides

Back to the home page