Polyglotism

Lately, I've been educating myself on the topic of what is commonly called NoSQL, and this nice presentation by Martin Fowler made me think about stuff I've already written about earlier. The same thing can also be found on a talk on data storage technologies employed in Craigslist.



The key phrase was polyglot persistence (@~51 mins in the video). Not that I wouldn't probably have heard it already earlier, but now the ultimate meaning finally dawned on me. It means, that to build a system - one that is highly performant, at least - you probably need to be able to take advantage of the different DBMSs that are around. The same way that multi-lingual developers push the idea that you need to use the language that is right for the job. So the diagram Fowler gave on polyglot persistence could be extended with programming languages with which the different parts of the system have been built with, and probably also with OSs the parts run on, and so on. There is a reason for dozens of DB brands and programming languages and whatnot existing, and it is the attempt to produce proper tools for specific problems. I can see this could also lead to a sprawl of diversifying computational environments, but I guess the constant popularity contest that goes on between products makes either the products converge or the niche options quietly fade away with the more popular ones gaining market share.

All this polyglotism further widens the options for a fresh wannabe ICT specialist. There certainly is a need for narrow deep expertise (to be able to know something like the Spring framework from top to bottom one really needs to spend considerable time with that alone), but I do not see that it would become very common for companies that are not actually huge in size and less than 100% technology oriented to hire bunch of specialists of different areas. Or they could, but then again they could hire them just as well as consultants since they are not likely to need each one of them equal amount of time constantly, and if the team members do not know much about each others areas of expertise they really can't be assigned to work on the same task together. My experience from the service sector is that there is a huge need for a team of people small enough that it can constantly be kept busy with the ongoing tasks and that you can throw at any kind of a problem and they can handle that based on their collective wide experience.

What I am after here is that even though I've been mostly using Oracle (from version 8 onward, I think) as the data back-end in my professional life,  the potential of interesting non-RDBMS encounters is highly rising. All along the road I've been running into cases where it is not feasible to stick to the good ol' 3NF of data and instead denormalising to get performance - something that the NoSQL folks are very fond of. This far it has been a convention that if an application needs a DB (bigger than can be reasonably embedded within the product on the same server), it supports one or more of the big traditional RDBMSs. Now I already see RHQ including Apache Cassandra in the package, and even though Cassandra comes bundled with the product, one basically needs to install it on a server of its own for practical use - so basically it can be said RHQ requires Cassandra in addition to one of the supported RDBMSs. Personally, this made me think of how I can sell the idea of requiring another server for application server monitoring to my superiors...

Complexity is the key

This was inspired by a slogan of one IT service company in which they promise to make things more simple. To me it looks like in the IT world the things are never getting simpler, quite the opposite, but we're trying to make them look and feel simple.

About a year ago I started with a personal programming project that is to be implemented as a Java web application. I started with just the Java Servlet API but as I have already learned long ago that making dynamic web UIs is a pain if you need to code it directly I was delighted by seeing the issue of Java Tech Journal which presented several different Java frameworks for web UIs. I ended up with Apache Wicket, since I didn't exactly want to get involved with anything as expansive as Spring to avoid having the project stall while I'm trying to learn too many new things, and Wicket seemed simple enough for my needs. The project did stall anyway, since I had some more concrete things to learn and do, but that's another story in another blog.

So, I picked a rather lightweight and simple framework and still I think I multiplied the amount of code base in my application by hundreds. The Wicket JARs take 3.7 megs (I currently use a tiny fraction of the features provided, and probably will never use even a tenth). Then there's the Servlet API which seems a lot slimmer on the outside, and there's log4j which is not that big either, but overall the web archive file is now around 9 megs. I run it on Tomcat, Tomcat runs on Java Virtual Machine which in turn runs on Linux. My very rough guess for the total amount of executable code needed for all of this is around 0.5 to 1 gigabyte with all the dependencies - and the application doesn't even do anything useful yet!

So where's the simplicity? The operating system makes things simple for the JVM, which in turn hides the underlying architecture from Tomcat so the developers didn't need to figure out all the details on how to make it run on different platforms. Tomcat, in turn, adds another layer of simplicity by offering services to the web application so that it can rely on the Servlet API and stuff like that, and finally Wicket makes it so much more simple to make HTML based UIs. I write little code, but use more than would have been needed if the whole thing would have been written from the scratch without using any APIs or extra abstraction layers. I save time and effort - more than a lifetime definately, I can't deny that. It's just so clear where all the advances in hardware are spent - to make things more simple...

Beside application servers, another good example of simplicity by complexity is the cloud concept. Abstraction upon abstraction, that is. I'm not opposing the layered software architecture, by no means no, it's a perfectly good concept for decoupling things (and thus controlling some of the complexity), just like modular code was a good "invention" back in the older times. It's just that when building upon n layers of abstraction you're already on so high level of abstraction that to do something that would be a trivial one-liner in e.g. C you need to call a bunch of methods that consist of tens of lines of code, and based on the KISS principle that's just ugly.

Now the interesting question: Is there an end for all this increased simplicity through complexity, will the stack of abstraction layers stop growing some day?