fredag 8. april 2011

Try some LSD

After my last post on the street corner economics and the “enterprise license” in commercial software I’ve gotten some mixed reactions on Yammer and by e-mail. Most of the reactions cite some features that are not supported by lighter open source alternatives. Some also say that it is better to go with the most comprehensive product available, just in case you need one of those features in the future.

Agile - the enterprise way “Just in case...” that’s the issue here. Why are these decisions being made before a single line of code is written? Agile software development methodologies and Scrum in particular seem to be the latest fad for enterprises, albeit with a tendency to focus more on preaching agile and harvesting certifications, than on actually embracing the agile philosophies and practices.

Disclaimer: Before I delve deeper into these practices I should probably say that I don’t have a keen interest in methodologies. I’m primarily interested in producing good code, efficiently and while I recognize that the the project methods affects me, I’m no agile evangelist. If good code is easier to achieve with Scrum, then so be it, but you won’t find me preaching about any software development method at a conference. I just don’t find them that interesting. It is still important to acknowledge that the chosen method impacts the decision making process in an IT project (eg. upfront vs just-in-time).

What I do find interesting however, is the number of companies/projects that claim to be agile, and at the same time feel totally comfortable choosing a multi-million development platform at the very beginning of a project. A typical scenario is; we’ll build a new online banking system and we’ll build it using Websphere Application Server, Websphere Process Server, DB2 and IBM MQ. Now let’s set up some “self organizing” scrum teams and do this agile style.

Deferring commitment One of my favorite agile principles is that of deferring commitment, or more precisely Schedule Irreversible Decisions at the Last Responsible Moment. If you ask me, making this type of decision (what platform to use) before a single line of code is written is making the decision at the first (and incidentally most irresponsible) possible moment.

For every small problem, there is a complex solution So a platform is chosen and the developers have been hired and dispersed in Scrum teams. Before they start doing any work, they’ve already been equipped. They have an extremely heavyweight and complex toolbox in their shed. What happens? For every problem one searches the toolbox to find the answer. What does this entail? Amortization calculations? ILog rules engine! Loan application? BPM with a mix of automatic and human tasks using BPEL! Bank statements? A service on the ESB with several layers of mapping between domain models using SCA and SDO.

Why does this happen? Well, it is a known fact that when all you have is a hammer everything looks like a NailFactory.getInstance().getNail(). But a similar thing happens when you have too many tools. When you've got an OR, nurses on stand-by, a scalpel and a bone saw, everything becomes frigging brain surgery.

Who’s to blame? I think many people have a share of the blame, as is often the case when an organization makes bad choices. Developers have a tendency to look for complexity where non exists (see Lucas Ward’s excellent blog post). However, I believe that more blame can be put on the shoulders of the non-coding-enterprise-architect, the one who made the decision to go for the complex platform.

Sunk cost driven development A typical waterfall project (dressed up as agile) will have decisions made by people who are not directly affected by them. Often a non-coding-enterprise-architect will decide on the platform which will affect ~50 developers and operations, every day. Having made this decision and not being directly affected, the non-coding-enterprise-architect will typically dictate that the platform is used for all it’s worth (or as often is the case, not worth). By utilizing every feature of the platform, including the ones that should really have been stamped as experimental beta in the documentation, the investment becomes seemingly sound. I call this sunk-cost-driven-development.

The $10 million servlet-container
Another outcome of premature decision making is on the opposite side of the sunk-cost-driven-development phenomenon. It is when you have a platform with web and ejb capabilities, ESB, BPM and what not, but you are just using the web capabilities. Your application runs on a combination of opensource frameworks, perhaps in a Spring container using some ORM or just JDBC for persistence. This is when you have a $10 million (at least) servlet container. It’s better than the sunk-cost-driven-development scenario, because at least the decision is not affecting your code, just your finances. (and I’d argue your speed of development as most of these $10 million servlet containers are pretty sluggish and a huge pain to use in development).

What I propose we do
Deffer commitment! There is a saying that “nobody ever got fired for buying IBM equipment”. It does not mean that no one ever should have been fired for buying IBM. (I really hope that it is not true that no one have ever been fired for buying IBM though). Buying software before you know what you need is just stupid.

Instead I suggest you start out with a lightweight architecture. There are several open source web containers out there that can host your web apps and/or your web services, tomcat and jetty comes to mind. You can start out using them for free, and if your organization deems them sufficient (my bet is they will) you can always buy support later. The beauty of open source support, as opposed to the support for commercial products, is that you can stop paying for it without having to surrender your server runtime at the same time. The only likely difference you’ll see is that you have to stay on top of patches yourself which is fairly easy.

What happens if you for some reason down the line figure out you desperately need SOA platform with SCA and SDO? Java can run anywhere and the servlet spec is fairly standard so migration should be a breeze. As for IBM/Oracle/etc they will welcome you with open arms. Have they ever been known to say no to free money?

Budgetary impact For this to work, the way projects are financed must be changed. The software and hardware costs (complex software typically requires expensive hardware) must not be carved in stone before the project is undertaken. The upfront budgeting of software and hardware costs may be what influence projects to do upfront acquisition of development platforms. Decisions are made way to early, when uncertainty is at its peak. There is only one thing that is certain about uncartainty, you can’t eliminate it by making decisions early, and likelihood is that uncertainty will decrease over time. Thus, a decision made later when more facts are known (perhaps after a few modules/users storries have been developed) is more likely to be a good decision.

Projects should have a leaner approach. One should be allowed to incur cost only when (and if) they are needed, instead of upfront or never at all. What I’m suggesting is Lean Software development in all phases of a project. So, why don’t we try some LSD for a change?