Monday, April 13, 2009

Agile practices are not just about management.

Although it’s a broad generalisation, Agile software development practices can be broken down into management practices and software design practices. The management practices are things such as short iterations, continuous integration, pair programming, programmer task estimation and co-location. The software design practices are things such as Test Driven Development, the SOLID principles and design patterns.

A common pattern I see repeated in many organisations is that the management principles are understood and adopted, while the software design principles are mostly ignored.

While the management practices may make for saner project management, they won’t do anything in themselves to improve the quality of software being produced or the productivity of the team.

Software development has a core pathology; you can see it on almost any project over a certain size. As the code base increases in complexity, the cost of adding a feature increases and defects proliferate. Often this is accompanied by ever increasing problems with build and deployment and de-motivated and unhappy developers.

Once the software gets part a certain size and age, it becomes almost un-movable and a real hindrance to the business. Often the response is to start from scratch, but this just makes things worse as the new system gradually goes the same way as the old, and in many cases never fully replaces it. Then there is the additional task of integrating the old and new systems, usually as a ‘temporary measure’.

Of course, building a robust scalable Service Oriented Architecture (to integrate the new and old systems) brings its own challenges. A team that failed to build a an initial scalable application will often fail in this task too. Soon more and more of the team’s resources are devoted to making the old and new systems talk to each other. More resources are applied and less is achieved.

The end game is often a static lock, where the business cannot move forward or react to changes in the competitive landscape because the software it relies upon cannot adapt. The software team becomes a large fixed cost that simply drains the bottom line rather than adding any real benefit. Usually the only agility the business retains is by working around their core systems and relying on more primitive forms of record keeping such as the humble spreadsheet or Access database.

I first came across the description ‘Agile’ in relation to software development by reading Bob Martin’s classic, Agile Software Development. Martin spends some of the book discussing Agile management techniques, but the majority of the text is devoted to Agile design practices including several detailed coding case studies.

Agile software design practices are the distillation of the collective experience of the software community over the past decades. A lot of very clever people have been struggling with the problem of how to avoid the pathology I described above. These practices are in no way a complete answer to all the problems we face with building large scale applications; the industry is constantly learning how to do this better; but they do add up to a sizable body of patterns and practices than can make a significant difference to the success of a project.

You can buy a whole library of books on writing better software, and if you are serious about it you should. But as a taster, here is a incomplete list of some Agile software development principles that come to mind as I write this:

Never rewrite the system. See The Big Redesign In The Sky.

Don’t repeat yourself ever. DRY

Refactor relentlessly (see above)

Only code what you need now (You Ain’t Gonna Need It: YAGNI)

Do the simplest thing possible.

Keep methods small (less than 10 lines of code). Keep classes small and with a single responsibility (see SOLID below).

Keep methods simple (low cyclomatic complexity).

Decouple components from each other at every scale (class, service and application).

Only allow components (classes, components, services, applications) to communicate through well defined interfaces.

Decouple your application into well defined tiers. Don’t mix business logic, UI logic and infrastructure.

Follow object oriented principles: Single Responsibility Principle, Open/Closed Principle, Liskov Substitution Principle, Interface Segregation Principle, Dependency Inversion Principle (SOLID)

Use design patterns. Someone has probably solved your problem before.

Use known algorithms and data structures.

Don’t write any code except to make an automated test pass. This is Test Driven Development.

Avoid code generation. But automate anything you can.

Use the right tool for the job: XML is not a good programming language, TSQL is not a good place to encode business rules.

6 comments:

Anonymous said...

I have the book and agree with just about everything but...

"TSQL is not a good place to encode business rules"

Arguably true but the cornerstone decision a company makes is their database be it Oracle, Sybase etc. Most companies data is everything and the lifespan of the database will probably span two or three changes of front end/server technology and the business logic will rarely change.

Cheers for the blog Mike.

Pinal Dave said...

Hi Mike,

Great Post!

I really wonder why it is suggested that T-SQL is not good programming language to encode business rules.

I think all the Stored Procedure created from T-SQL are the best place to store logic. It is faster, scalable, easy to modify and few other advantages.

Best Regards,
Pinal

Daniel Robinson said...

@Mike: I completely agree with your TSQL comment.

Too many times I've seen developers put all the business rules in Stored Procedures. The argument is often used that they run faster and are easier to modify.

TSQL *might* run faster than business logic written in application code for some situations. BUT the database is the wrong place to put business logic from a Test Driven or Domain Driven Design perspective.

I've worked with some developers in the past who argue that their stored procedures are easier to modify. The reason why they say this is because they change their stored procedures on the fly, with little regard for source control and release versions which becomes a nightmare to manage.

@Pinal: I'm not saying that you do things this way, this is just my experience with other developers.

While it's true that most companies have a lot of legacy business logic in databases, I don't think we should perpetuate this way of working.

Thanks for a great blog post Mike!

Regards,
Daniel.

Mike Hadlow said...

Hi Anonymous, Dave,

I think Daniel made a good case. For me it's down to a number of factors:

1. T-SQL sucks as a general purpose language. You'll experience extreme pain trying to apply SOLID principles to anything written with it. You WILL duplicate business rules if you try to encode them in SPs.

2. Layered architecture. There should be one place only for business rules, your domain model. Please don't code them into your data access infrastructure.

3. T-SQL sucks as a .... oh I already said that :p

Have a read of Franz Bouma's blog:

http://weblogs.asp.net/fbouma/archive/2003/11/18/38178.aspx

Anonymous said...

I'm not disagreeing re test driven development but just how many sites are developing to good TDD and Agile principles?

The database is the last line of defence re data input. With proper referential integrity, rules and constraints implimented it should safeguard the companies main asset.

We've stripped out almost all of our SPs in our new e-commerce app in favour of a LINQ based repository pattern but there are a few instances where LINQ or any server based code just can't cut it in the real world and that's where we go down to the metal.

b.t.w how goes the contract market in Brighton ?

Mike Hadlow said...

Hi Anonymous,

"just how many sites are developing to good TDD and Agile principles?"

I have been doing TDD (well or not, is another question:) for the last five years or so. With one exception I've been able to introduce it to every site I've worked on. I know several companies doing .NET around Brighton which practice TDD and most of the Agile principles I've outlined above.

"The database is the last line of defence re data input. With proper referential integrity, rules and constraints implimented it should safeguard the companies main asset.
"

Absolutely. I totally agree.

"We've stripped out almost all of our SPs in our new e-commerce app in favour of a LINQ based repository pattern but there are a few instances where LINQ or any server based code just can't cut it in the real world and that's where we go down to the metal."

That sounds like a very sane approach :) Saying that business rules should not live in sps isn't saying that you should never use them. I think they are an excellent solution to a particular class of problems. However, for the last four systems I've developed, I don't think I've written a single line of production SQL, which leaves me to think that the use of sps should be the (rare) exception rather than the rule. But then that pretty much agrees with what you are saying.

"b.t.w how goes the contract market in Brighton ?"

Terrible!