Friday, September 18, 2009

Why is ‘Buy’ not always better than ‘Build’

Building software is hard. I’ve been involved in building business systems for 13 years and it’s probably fair to say that I’ve been involved in more failures than successes, and I’m pretty sure that’s not merely the function of my effectiveness (or lack of it) as a software developer. Any business embarking on the development of a core system is taking on significant risk, because the core competencies of building software are probably not the core competencies of the business. After all the business of building software is a huge problem domain in its own right with a large and complex body of expertise.

Recognising this, many businesses when faced with a Buy vs Build decision will do almost anything to avoid Build; purchasing Commercial Off-The-Shelf (COTS) systems is always the preferred decision.

If your business need is a common one, then purchasing a COTS system is almost always the right choice. You would be insane to consider building a bespoke word processor for example.

But often it’s the case that your requirements are somewhat common, but with unique elements that only your business would need.

Given the understandable preference for Buy over Build, the temptation is to take a COTS system, or systems, and either fit your business around it, or have it customised in some way. So long as the features of the COTS software and your requirements overlap it’s worth buying it right?


Sure you will have some unsatisfied requirements and there will be some features of the COTS system that you don’t use, but you are still getting benefit. The unsatisfied requirements can be fulfilled by customising or extending the COTS package and the unused features can just be ignored.

You may have a requirement for customer relationship management for example; why not just buy MSCRM from Microsoft? The things your customers care about might be somewhat unique, but the Microsoft reseller will assure that MSCRM can be customised to meet your requirements. You might also have case-management style requirements alongside your customer relationship ones, why not buy a case management tool as well and simply integrate it with MSCRM?

But now, as soon as you decide to go down the COTS + customisation route, you are effectively doing software development. Except that now you have suckered your way into doing it rather than approaching it with your eyes open. You will find that extending and customising COTS systems is often a horrible variant of ugly hackery that’s almost impossible to do with good software craftsmanship. You will still have to hire developers, but instead of allowing them to build well architected solutions, you will be forcing them to do a dirty and thankless job. You can guess the calibre of developer that will be attracted to such a position.

The reason why it’s hackery is that it’s almost impossible to extend or customise software without rewriting at least a part of it; and of course, you won’t be able to rewrite your closed-source COTS system. To answer this, most COTS vendors pay some lip service to customisability and extensibility, but the hooks they provide are always somewhat clunky and limited. Anyone who has tried to build a system on top of Sharepoint, or customise MSCRM will know the pain involved. It’s the same for pretty much any enterprise software system.

Each additional requirement over and above the out-of-the-box features will cost significantly more than the equivalent additional requirement of a custom system.

You have a graph somewhat like this:


So long as your set of customisations is limited, it’s still worth going down the COTS route, but my experience is that the point where the two lines cross is further to the left than is often appreciated. The chart doesn’t of course factor in the original cost of the COTS system and that can be substantial for many ‘enterprise’ software packages.

A further consideration that’s often ignored is that business requirements often change over time. How are you going to ensure that your COTS system, even if it perfectly matches your requirements now, can continue to match them into the future?

So if you have to do software development, and you will have to unless you can find a COTS system that fully satisfies your requirements, why not do it properly: hire and nurture a great software team; put professional software development practices in place; properly analyse your business; work out exactly what you need from your supporting applications, and then build a system that exactly fits your needs.

An alternative might be to build a close relationship with a bespoke software house, but be aware that their incentives will not be fully aligned with yours and in any case, if your system is substantial, they will probably simply go to the open market and hire the same team of developers that you could have hired yourself.

So consider carefully the costs of a COTS system that doesn’t fully meet your requirements and don’t ignore the benefits that bespoke software built and supported by an in-house team can bring.


Nissan Reddi Dookeran said...

Excellent article Mike! Very timely considering I was having just such a discussion with a Trinidad software developer on why he and his team were having 'gross development pains' customising such a pre-built system (mainframe, not MS techology though) that has existed within his organisations for several years and why his management may need to look at longer term strategies that involve investing and migrating towards a bespoke system built specifically for their core business processes.

Mike Hadlow said...

Hi Nissan,

How are you doing?

Sounds like yet another case study of why COTS + customise is often a bad choice.

Michael Foord said...

Hey Mike,

The topic of this article sounds very like the reasons behind the creation of Resolver One.

Excel is often a halfway house between 'buy' and 'build' - Excel provides enough of a framework to allow you to build almost anything on top of it, if you can face the pain.

Resolver One was designed from the start to be a base for building systems on. You've inspired me, perhaps I'll write a blog entry on it.

Phil said...

Interesting post and one that resonates quite strongly with me at the moment. We're in the middle of a project to update our accounting system to the latest version, but we have a number of systems written in-house that sits on the edge of the accounting system for purchase ordering and the like. In the Promised Land of the upgraded version, all these systems will be swept aside and replaced by modules in the accounting system. Except, of course, that they've all been tailored to our requirements and workflows and 'the way we do things'. And I think we're going to get bitten on it - I think, exactly as you say, that our point of crossover is significantly more to the left than anyone realises.

blog said...

Interesting post, and in my experience I've always found the businesses I work for to be on the build vs buy side of the equation. Not sure I could work for one on the buy-side, because I can't really see myself doing that work. With in-house custom systems, eventually you reach a point where buy vs build comes up again, where some part of the business might suggest buying a COTS and integrating or replacing some of the custom software. Which is kind of the reverse of the COTS + customize issue. To try and find an off the shelve product to replace a highly specialized in-house piece of software that also can be hooked in, hardly seems feasible. Fortunately we've steered away from these endeavors but the threat remains.

Kevin Taylor said...


Nice job explaining the hidden costs of customizing a COTS systems.

I've followed up with a few thoughts of my own on when to consider custom systems:

Why Build Custom Software?

--Kevin Taylor

Mike Hadlow said...

Hi Kevin,

Great post and thanks for the link.

Chase Saunders said...