Extending the Filesystem Idea

My idea of a filesystem for objects has extended itself in my mind quite dramatically. I find that a lot of things I store in a database would be really nice to store on the filesystem. But of course in a server environment, using the filesystem is a “bad idea”. There are really good reasons for not using the filesystem in a server application:

  • When you’ve got lots of users it’s really easy to run out of file-handles
  • Locking becomes a mission that you mostly have to handle yourself
  • You may be running in a cluster, and then your admins have to deploy special filesystems
  • The filsystem is “slow”
  • You loose the content based addressing of a relational-database

That said, in many cases, it makes abundant sense to use a “filesystem” of some sort or the other to store your data. So my idea extends to this:

  • A “Filesystem” API that can store any data
  • The filesystem is contained in one file (2 if you journal it)
  • The filesystem can have a caching layer put on top of it to improve performance
  • The API supports:
    • Locks on files (or parts of a file)
    • Symlinks
    • Events
  • A Network layer for clustering

If my work allows, I’ll be  developing this API and open-sourcing it.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: