Try some LSD

After my last post on the street corner economics and the “enterprise license” in commercial software I’ve gotten some mixed reactions on Yammer and by e-mail. Most of the reactions cite some features that are not supported by lighter open source alternatives. Some also say that it is better to go with the most comprehensive product available, just in case you need one of those features in the future.

Agile - the enterprise way “Just in case...” that’s the issue here. Why are these decisions being made before a single line of code is written? Agile software development methodologies and Scrum in particular seem to be the latest fad for enterprises, albeit with a tendency to focus more on preaching agile and harvesting certifications, than on actually embracing the agile philosophies and practices.

Disclaimer: Before I delve deeper into these practices I should probably say that I don’t have a keen interest in methodologies. I’m primarily interested in producing good code, efficiently and while I recognize that the the project methods affects me, I’m no agile evangelist. If good code is easier to achieve with Scrum, then so be it, but you won’t find me preaching about any software development method at a conference. I just don’t find them that interesting. It is still important to acknowledge that the chosen method impacts the decision making process in an IT project (eg. upfront vs just-in-time).

What I do find interesting however, is the number of companies/projects that claim to be agile, and at the same time feel totally comfortable choosing a multi-million development platform at the very beginning of a project. A typical scenario is; we’ll build a new online banking system and we’ll build it using Websphere Application Server, Websphere Process Server, DB2 and IBM MQ. Now let’s set up some “self organizing” scrum teams and do this agile style.

Deferring commitment One of my favorite agile principles is that of deferring commitment, or more precisely Schedule Irreversible Decisions at the Last Responsible Moment. If you ask me, making this type of decision (what platform to use) before a single line of code is written is making the decision at the first (and incidentally most irresponsible) possible moment.

For every small problem, there is a complex solution So a platform is chosen and the developers have been hired and dispersed in Scrum teams. Before they start doing any work, they’ve already been equipped. They have an extremely heavyweight and complex toolbox in their shed. What happens? For every problem one searches the toolbox to find the answer. What does this entail? Amortization calculations? ILog rules engine! Loan application? BPM with a mix of automatic and human tasks using BPEL! Bank statements? A service on the ESB with several layers of mapping between domain models using SCA and SDO.

Why does this happen? Well, it is a known fact that when all you have is a hammer everything looks like a NailFactory.getInstance().getNail(). But a similar thing happens when you have too many tools. When you've got an OR, nurses on stand-by, a scalpel and a bone saw, everything becomes frigging brain surgery.

Who’s to blame? I think many people have a share of the blame, as is often the case when an organization makes bad choices. Developers have a tendency to look for complexity where non exists (see Lucas Ward’s excellent blog post). However, I believe that more blame can be put on the shoulders of the non-coding-enterprise-architect, the one who made the decision to go for the complex platform.

Sunk cost driven development A typical waterfall project (dressed up as agile) will have decisions made by people who are not directly affected by them. Often a non-coding-enterprise-architect will decide on the platform which will affect ~50 developers and operations, every day. Having made this decision and not being directly affected, the non-coding-enterprise-architect will typically dictate that the platform is used for all it’s worth (or as often is the case, not worth). By utilizing every feature of the platform, including the ones that should really have been stamped as experimental beta in the documentation, the investment becomes seemingly sound. I call this sunk-cost-driven-development.

The $10 million servlet-container
Another outcome of premature decision making is on the opposite side of the sunk-cost-driven-development phenomenon. It is when you have a platform with web and ejb capabilities, ESB, BPM and what not, but you are just using the web capabilities. Your application runs on a combination of opensource frameworks, perhaps in a Spring container using some ORM or just JDBC for persistence. This is when you have a $10 million (at least) servlet container. It’s better than the sunk-cost-driven-development scenario, because at least the decision is not affecting your code, just your finances. (and I’d argue your speed of development as most of these $10 million servlet containers are pretty sluggish and a huge pain to use in development).

What I propose we do
Deffer commitment! There is a saying that “nobody ever got fired for buying IBM equipment”. It does not mean that no one ever should have been fired for buying IBM. (I really hope that it is not true that no one have ever been fired for buying IBM though). Buying software before you know what you need is just stupid.

Instead I suggest you start out with a lightweight architecture. There are several open source web containers out there that can host your web apps and/or your web services, tomcat and jetty comes to mind. You can start out using them for free, and if your organization deems them sufficient (my bet is they will) you can always buy support later. The beauty of open source support, as opposed to the support for commercial products, is that you can stop paying for it without having to surrender your server runtime at the same time. The only likely difference you’ll see is that you have to stay on top of patches yourself which is fairly easy.

What happens if you for some reason down the line figure out you desperately need SOA platform with SCA and SDO? Java can run anywhere and the servlet spec is fairly standard so migration should be a breeze. As for IBM/Oracle/etc they will welcome you with open arms. Have they ever been known to say no to free money?

Budgetary impact For this to work, the way projects are financed must be changed. The software and hardware costs (complex software typically requires expensive hardware) must not be carved in stone before the project is undertaken. The upfront budgeting of software and hardware costs may be what influence projects to do upfront acquisition of development platforms. Decisions are made way to early, when uncertainty is at its peak. There is only one thing that is certain about uncartainty, you can’t eliminate it by making decisions early, and likelihood is that uncertainty will decrease over time. Thus, a decision made later when more facts are known (perhaps after a few modules/users storries have been developed) is more likely to be a good decision.

Projects should have a leaner approach. One should be allowed to incur cost only when (and if) they are needed, instead of upfront or never at all. What I’m suggesting is Lean Software development in all phases of a project. So, why don’t we try some LSD for a change?

Sunk...

Free as in heroin

Free software” is a matter of liberty, not price. To understand the concept, you should think of “free” as in “free speech”, not as in “free beer”.

By now, we all know this quote, right? And likely, we've all had the pleasure of experiencing a late night pun about free beer at developer conference. Today, I'm introducing another version of free, free as in heroin.

As a proponent of open source software I’ve met some quite interesting arguments for choosing bloated proprietary products over more lightweight open source counterparts; the site license or the enterprise license. This license is devious scheme created by proprietary software vendors, such as Oracle. The idea is that for a fixed amount you’ll get to deploy an unlimited amount databases, application servers and what not, provided they are only used for internally facing systems.

Noticed the trend towards enterprise service bus? I’m not talking about the SOA-hype which has thankfully chilled the last few years, I’m talking about these several gigabyte installs of what is essentially just a bloated implementation of the mediator pattern. The ESB sits there between your components, serving as a man-in-the-middle attack, doing point-to-point integration. How did you ever get on without it, right?

I’m sure that the ESBs have some capabilities that I’m too ignorant to acknowledge, but my experience is that once these products get in house, which they do thanks to the vendor’s very apt sales corps, their use is limited to point-to-point integration. Whatever happened to standard Java? Heck, even EJBs? They are still around, but they are not being pushed by the vendors anymore. The opensource containers are the first to adopt the new standards, while the commercial vendors are lagging. Why?

Well, this is speculation on my part, but why should they pour resources into abiding by the standards? Following standards will reduce the cost of switching to open source (or another standard abiding competitor). Instead, resources are spent on the ESBs. There is no standard programming model for ESBs, so once your applications are hooked into the ESB, moving to another vendor or removing the ESB all toghether will be expensive and time consuming as they all have to be re-written. The customers’ processes now actually belong to the commercial vendor. Sure, the code is the customer’s IP, but it is useless without the bloated ESB infrastructure which of course is not the customer's IP.

So the customer stays, dutifully paying the site license each year. Swearing slightly every time the license model change, making hardware upgrades more expensive. But at this point the addiction has gotten more severe as more applications are hooked into the ESB.

Why? Because the site license meant it was “free”. Free as in heroin. Taken straight out of the playbook on street corner economics, stimulate an addiction and live of it for life.