LogFS returns [LWN.net]

LWN.net needs you!
Without subscribers, LWN would simply not exist. Please consider signing up for a subscription and helping to keep LWN publishing.

By Jonathan Corbet
November 24, 2009

LogFS is a longstanding project by Jörn Engel to create a filesystem for contemporary solid-state storage devices; it was last covered here in May, 2007. Since then, LogFS has mostly disappeared from view. As of November 20, though, LogFS is back and, seemingly, ready for a mainline merge. Jörn says:

Logfs has been around a couple of times. Linus last word was "go and don't come back until all format changes are done". Or something along those lines at least. Format changes are done. And I don't even intend to break git-bisect for anyone crazy enough to use logfs for /.

Sufficiently crazy users seem to be relatively scarce so far. But having more options for upcoming hardware can only be a good thing; it will be interesting to see what results come out as people start to play with this new filesystem.

Index entries for this article
Kernel	LogFS

to post comments

LogFS returns

Posted Nov 26, 2009 7:16 UTC (Thu) by johnflux (guest, #58833) [Link] (4 responses)

Will this allows us to rollback the system to any point in time etc? Those sorts of features, on the face of it, would be to come for free?

LogFS returns

Posted Nov 26, 2009 9:06 UTC (Thu) by joern (guest, #22392) [Link] (2 responses)

Many people initially thought that, me included. But the answer is no.

Going back to _any_ point in time only works with near-infinite storage. If you ever write more data than fits your medium, something previously deleted will be overwritten.

The garbage collector makes things worse. As you might know, it has to copy around existing data in order to free larger segments. A corollary is that with a medium that is N% full, the worst case is that each segment is also N% full and the garbage collector will waste N% of the bandwidth, leaving 100-N% for the user to do real work. If your filesystem is 90% full, performance may drop to 10%.

So if the filesystem were to give any sort of guarantee that (some) old data could be recovered, the amount of available free space would drop, hurting performance. Definitely not a free feature.

Maybe I should add that performance does not necessarily drop as dramatically. If the filesystem did a good job when writing the data, it will have seperated long-living data from short-living data. To do so perfectly requires a chrystal ball, but even by pure accident, data that is written at the same time is often also deleted at the same time. Therefore the worst case of every segment being exactly N% full is exceedingly rare and real performance will better than the calculated worst case above.

LogFS returns

Posted Nov 26, 2009 9:46 UTC (Thu) by johnflux (guest, #58833) [Link] (1 responses)

A "typical" desktop user doesn't really use most of their typically 1TB hard disk. It could be years before it needs to overwrite any existing data.

LogFS returns

Posted Nov 26, 2009 11:28 UTC (Thu) by joern (guest, #22392) [Link]

If you have a hard disk, I would suggest you try btrfs instead. It was designed for spinning rust, logfs was designed for leaky electron traps.

But if you can guarantee that the total amount of data ever written does not exceed the size of your medium, historical data would indeed stay around. You cannot access deleted data through the filesystem code itself, but you could write a dump tool that goes back in time.

How that dump tool would behave once the garbage collector has to reuse old segments is another question. ;)

LogFS returns

Posted Dec 3, 2009 13:40 UTC (Thu) by forthy (guest, #1525) [Link]

AFAIK, in NILFS, which is another log structured file system, it comes for free - the garbage collector will reclaim those snapshots over time, though, but there's an easy way to access those snapshots, as long as they are there.

For "rotating rust", I'm far from sure that current file systems are optimized for disks at all. First of all, the "best" block size for a random access medium is one where transfer time=access time. A contemporary hard disk can read about 1MB or seek to another location, so block size should be 1MB (increasing to 2MB next year or so ;-). This should certainly contain a whole bunch of small files when used for small files, but still, access granularity below 1MB doesn't make sense. Second, metadata: For all "normal" use, main memory is big enough to keep all metadata in main memory. Keep a log on the disk where you write your metadata transactions sequentially, and on unmount (and when the log exceeds a certain size), dump the whole metadata on disk. Would make live a lot easier for the spinning rust.