Relational databases make me sad.  They are slow, they pose a serious impedance mismatch when used with an object-oriented language, they are heavy, they have non-trivial hardware requirements, and they can cost (a lot) of money.  Did I mention that they’re SLOW?  Unfortunately, some 95%* of software applications use relational databases for persistence, which means 95%* of applications are dealing with the problems I just mentioned.  Research has shown that nearly 40% of effort on most software development projects goes towards persistence-related issues.** 

*yes, that statistic is made up, but I would be surprised if it wasn’t close to accurate.

**Not a made up statistic, see Keene’s “Data Services for Next-Generation SOAs” article in SOA Web Services Journal.

As developers, I think we’ve basically come to accept that persistence is a hard problem, and that the solution is the relational database.  But wait, we’re spending up to 40% of effort on persistence?!?  That doesn’t sound like we solved it very well.  It’s taken me many years, but I’ve come to realize that while persistence is indeed a solved problem, the answer isn’t always the relational database.  I dare say that in most cases, the relational database is the wrong solution. 

There is the old saying that you shouldn’t use a hammer to drive a screw is quite true.  You need to pick the right tool for the right job.  I wish I’d seen it sooner, but I’ve actually been doing it *wrong* for a number of years.  This is a bold statement, but using a relational database, in most applications, is akin to driving a screw with a hammerIt’s simply not the right tool for the job.

“YOU ARE DUMB, MATT, RELATIONAL DATABASES PROVIDE FEATURES A-Z!!!  YOU HAVE TO HAVE THEM?  WHAT AM I SUPPOSED TO DO, JUST WRITE TO A FILE?”

Relational databases do indeed have a lot of features, and I will discuss the pros of relational databases in a future article (maybe), but I bet you actually need far fewer of them in practice than you think.   And no, having your app write directly to the filesystem for its persistence needs is probably not the right solution (though it might be).  The thing is that relational databases are far from the only tools that you can use to solve the persistence problem.  You should also be looking at:

In future posts, I plan to go more in depth with some of the alternatives (particularly the solution we’re going with here at my day job).  Right now, we’re trying to decide between Berkeley DB-XML and MongoDB, two very powerful (and very fast) alternatives to our old approach, which involved mapping everything to a relational database.