January 28, 2008

...Learn TDD with Codemanship

Don't Rely On Database Technology For Sorting & Searching

Those of you who can remember the old ways might recall what it was like to actually write code to sort and search and do all manner of things that today's young whipper-snappers call "database work".

I'm just as guilty of this, sometimes. Faced with a requirement to search for an instance of Person by name, how tempting is it to just set up a PERSON table in a relational database and rig up a Hibernate query or some ADO.NET (or whatever) to do the donkey work for you?

The risk is that there's logic in them there searches and sorts, and we can end up with applications that only work when there's some kind of database technology bolted onto the back. Okay, so sometimes this is unavoidable, especially when we're dealing with very complex queries, or when we're dealing with extremely large collections of objects (millions).

But we'd do well to remember that in 99% of cases, a simple binary search or quicksort might do the job very ably, and this stuff is a doddle to knock up in Java or C#. Indeed, what do you think RDBMS platforms like SQL Server and CloudScape are written in? It's not magic, y'know.

100,000 instances of Person will fit quite nicely into a sorted array list in memory, and a binary search will make short work of that - even with more sophisticated matches (e.g., with wildcards or regular expressions). No need for invention, either, since the problem was solved about 45 million years ago and has been documented and demonstrated in just about every programming language under the sun. Just Google it, and a suitable implementation will probably present itself. (And, no, I'm not advocating copying and pasting it into your code - you still have to think about it.)

If your objects need to be persisted (for transactional purposes, for example), then it's better to have a read-only searchable collection sitting in memory and synchronise it with the underlying data whenever it changes. Hibernate is especially good for that sort of set-up.

And, if, like me, you don't have a computing background and haven't been taught all about the old ways, then I recommend a little self-education:

For Java programmers, take a look at Data Structures & Algorithms in Java, and for .NET programmers try Microsoft's own resource or invest in a copy of this book.

Then you might like to consider messing up your hair, growing an untidy beard, and maybe sowing some leather patches on the elbows of your tweed jacket to complete the effect.

Posted 10 years, 6 months ago on January 28, 2008