A friend recently told me that the Java developers at his company treated the database as a datastore from which they took data and if they were in a good mood they might put it back too. Ah how we laughed… however secretly I thought that was rather unfair. Java developers generally have a tough time with databases and sometimes the things they do to them makes you want to scream but overall I think most respect the fact that an object-orientated language does one thing and an implementation of relational data does another, very different thing. After all you wouldn’t try to write an MP3 player in SQL… oh, hold on bad example you might not want to try that in Java either…

Anyway, any situation where you want to compare and relate data is likely to be a situation where you want to use SQL. Handling things like threaded programming, IO, cross-platform GUIs, scalable web applications are all likely to involve Java. This was proven to me by a Business Analyst who was cross-matching the output of two different systems. I tried to solve the problem by writing a Java application that essentially built up hash maps of various keys and then compared them. I thought it was pretty nifty but it was hard work and drilling down into why the failed matches failed was hard work.

The analyst, damn her, insisted that the output of the two programs be loaded into a database. She then took days to complete the entire reconcilation and was entirely capable of explaining and proving matches and non-matches. My method would probably have taken weeks and a lot of the output would probably have to be taken on trust.

It one of those moments where you can choose to be humble or humilated. Well it sounds bigger if you’re humble anyway but I think I learned an important lesson as a result. If people have spent twenty years or more perfecting a tool for a specific task don’t flatter yourself that you’ll do a better job from first principles with your preferred tool. It’s just a case of Golden Hammer syndrome.

However there are still a lot of Java developers who do think of the database as a datastore. And it is utter madness. If you are using Berkley DB you maybe have a point. But the minute you have a relational framework to your data then the database is not a datastore. It is a monstrously potent way of making data your bitch. Hundreds of developers have tried to move data processing out of the database and into the application layer. I have never witnessed one who has actually succeeded in doing anything than replicating a relational database in memory and the majority just created a catastrophic mess.

If you think the database is a datastore then step away from the database and do something else. Don’t try to bastardise relational algebra, like a lot of things in life it’ll just end in tears.