Linux Filesystems: Where did they come from? [linux.conf.au 2014]

An excellent, thought provoking paper on Linux filesystems was presented at the
FAST 13 conference earlier this year:

"A Study of Linux File System Evolution"
Lanyue Lu, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau, and Shan Lu,
University of Wisconsin-Madison

https://www.usenix.org/conference/fas...

The paper analysed the contents of all the changes made to the major Linux filesystems since 2.6.0 - some 5000+ commits - and focussed on classifying the code changes in a variety of different ways, such as features, maintenance, bug fixes, error handling, and so on. The subsequent analysis of this classification process gives a unique insight in the way Linux filesystems have evolved from a code persepective and deserves the accolades it received.

However, being involved in Linux filesystem development for most of the time period covered by the paper, I looked at some of the graphs and thought "I know what happened right then to cause that spike". It was this level of insight that parts of the paper gave me that left me wanting analysis from a different perspective - a perspective that the paper did not have the scope to address.

For example, the paper mostly ignored the contents of feature patches, except to say they were larger than most of other classifications. Nor did it try to explain why more than 60% of the XFS changes were classified "maintenance" patches. And the biggest question I had was "where did all the changes come from, anyway?" wasn't really in the ballpark of pure code analysis.

Hence, I'd like to take the audience on a winding journey of code archeology that walks us through the development history of the three main Linux filesystems - EXT4, BTRFS, and XFS - to look at the principles behind their architectures, where they derive from who was responsible for their design and implementation, how ideas have propagated between the filesystems and how the goals for filesystems have changed over time.

This journey will take us through many different filesystems and concepts, right up the the current state of the art that is beyond what is implemented in any current Linux filesystem. While we walk along this path, I'll attempt to tie major events and trends to people via commit history, mainling list archives, patches and - of course - flamewars.

In the end, I hope to provide a more intimate, personal view of the history and evolution of Linux filesystems than just the code alone can give us. Inspiration often comes from the strangest of places.....
Dave Chinner

Name: Dave Chinner
Current Employer: Red Hat
Current Position: Principle Software Engineer - Filesystems
Biography:

Dave Chinner has an advanced case of Filesystem Developer Syndrome (FDS). Symptoms first developed back in 2002 when employed by SGI to work on NFS. After a confusing diagnosis including NFS, gigabit ethernet driver and TCP/IP stack hacking on Irix, he was discovered in a delirious state, not knowing where he was or what he was doing. Editors with open XFS source code files were found on his workstation, and the diagnosis was clear: it was another case of FDS.

Soon after this initial diagnosis was made, he started to regain his sanity by working on XFS on Linux. Symptoms stabilised over the next few years as he became more familiar with XFS and he spent less time in a dazed and confused state. While at SGI, Dave presented papers about XFS IO scalability at OLS in 2007 and about XFS repair scalability at LCA 2008.

After leaving SGI in mid-2008, he spent time working for storage startups of one sort or another, never far removed from XFS. His case of FDS went slowly went into remission as he tackled other problems outside XFS, but it was never far from the surface.

In early 2010, Dave was employed by Red Hat to once again work full time on XFS. It was at this time he suffered from a total relapse of his FDS symptoms, culminating in feverish dreams solutions to XFS scalability problems. While he is mostly coherent and stable again (the way he likes his filesystems) he still flinches at the mention of ponies.

http://linux.conf.au/schedule/30112/v...