The best Linux file system of all?

January 15, 2010 12:23 PM EST
Want to get Unix/Linux techies arguing? Besides classic flame wars such as whether vi or EMACS is the better text editor, another surefire way to start a fight is to talk about which file systems are the best. Google, which knows a thing or two about fast systems has decided, for their purposes anyway, that Ext4 is the best and close to the fastest file system of all.

In a recent note to the Ext4 developer mailing list, Google's Michael Rubin, a senior staff engineer, wrote, "Google is currently in the middle of upgrading from ext2 to a more up to date file system. We ended up choosing ext4."

Rubin then explains, "The driving performance reason to upgrade is that while ext2 had been 'good enough' for a very long time the metadata arrangement on a stale file system was leading to what we call "read inflation". This is where we end up doing many seeks to read one block of data. In general latency from poor block allocation was causing performance hiccups."

Ext2 is an older Linux file system. It's still often used on flash drives, because it doesn't write to the drive that often, and flash drives can be written to only a limited number of times before wearing out. But other than that, Ext2 was largely superseded by Ext3, which added journaling to Ext2. Journaling file systems keep a record, or journal, of changes to the main file system. With journaling, if the file system crashes, it's much easier to restore it to a working condition without losing any data.

The newest member of the Ext file system, Ext4, became an official part of Linux last year with the release of the Linux 2.6.28 kernel. Since then, it's become the default file system in some popular Linux distributions such as Fedora, and it's now available on all distributions.

Ext4 enables faster disk performance and better drive space management than its predecessors. While it also includes journaling, you can turn that off for a modest speed boost. I'm sure Google was also interested in it because even without disabling journaling, Ext4 is a very fast file system, and it supports file systems of up to 1 EB (exabyte) and up to 16 TB (terabyte)-sized files.

Of course, there were other contenders. Rubin went on to write, "We spent a lot of time with Unix standard benchmarks (dbench, compile bench, et al) on xfs, ext4, jfs to try to see which one was going to perform the best. In the end we mostly ended up using the benchmarks to validate our assumptions and do functional testing.

"In the end, For our workloads we saw ext4 and xfs as 'close enough' in performance in the areas we cared about. The fact that we had a much smoother upgrade path with ext4 clinched the deal. The only upgrade option we have is online. Ext4 is already moving the bottleneck away from the storage stack for some of our most intensive applications."

But Google isn't just using vanilla Ext4. The company recently hired Ted T'so, the well-known Linux kernel developer, who also happens to be the leading Ext4 programmer. I think I can safely say with Google supporting T'so and Ext4, this fast file system will only get faster still.