Wednesday, March 12, 2014

Coconut Headphones: Why Agile Has Failed

The 2001 agile manifesto was an attempt to replace rigid, process and management heavy, development methodologies with a more human and software-centric approach. They identified that the programmer is the central actor in the creation of software, and that the best software grows and evolves organically in contact with its users.

My first real contact with the ideas of agile software development came from reading Bob Martin’s book ‘Agile Software Development’. I still think it’s one of the best books about software I’ve read. It’s a tour-de-force survey of modern (at the time) techniques; a recipe book of how to create flexible but robust systems. What might surprise people familiar with how agile is currently understood, is that the majority of the book is about software engineering, not management practices.

So what happened? Why is agile now about stand-ups, retrospectives, two-week iterations and planning poker?

Somehow, over the decade or so since the original agile manifesto, agile has come to mean ‘management agile’. It’s been captured by management consultants and distilled as a small set of non-technical rituals that emerged from the much larger, richer, but often deeply technical set of agile practices.

It’s often said that ‘bad agile’ resembles a cargo cult. James Shore has an excellent post, Cargo Cult Agile, that describes how rigid adherence to the ritualistic forms of agile methodologies closely resemble South Pacific cargo cults:

“The tragedy of the cargo cult is its adherence to the superficial, outward signs of some idea combined with ignorance of how that idea actually works. In the story, the islanders replicated all the elements of cargo drops--the airstrip, the controller, the headphones--but didn't understand where the airplanes actually came from.

I see the same tragedy occurring with Agile.”

Current non-technical agile practitioners still don’t understand where the airplanes come from. They stand in their bamboo control towers with their coconut headphones on and wonder why their software projects still fail.

Agile has indeed become a cargo cult. Stripped of actual software engineering practices and conducted by ‘agile practitioners’ with no understanding of software engineering, it merely becomes a set of meaningless rituals that are mostly impediments and distractions to creating successful software.

well-ask-them-for-estimates

The core problem is that non-technical managers of software projects will always fail, or at best be counter productive, whatever the methodology. Developing software is a deeply technical endeavour. Sending your managers on an agile course to learn how to beat developers over the head with planning poker, two week iterations and stand-ups will do nothing to save spaghetti code and incompetent teams. You might have software projects that succeed despite the agile nonsense, but that would be coincidence, not causation.

Because creating good software is so much about technical decisions and so little about management process, I believe that there is very little place for non-technical managers in any software development organisation. If your role is simply asking for estimates and enforcing the agile rituals: stand-ups, fortnightly sprints, retrospectives; then you are an impediment rather than an asset to delivery.

Please don’t put non-technical managers in charge of software developers.

I don’t have an answer, or an alternative methodology to offer you, but here are some things that any software development organisation must address:

  • The skills and talents of individual programmers are the main determinant of software quality. No amount of management, methodology, or high-level architecture astronautism can compensate for a poor quality team.
  • The motivation and empowerment of programmers has a direct and strong relationship to the quality of  the software.
  • Hard deadlines, especially micro-deadlines will result in poor quality software that will take longer to deliver.
  • The consequences of poor design decisions multiply rapidly.
  • It will usually take multiple attempts to arrive at a viable design.
  • You  should make it easy to throw away code and start again.
  • Latency kills. Short feedback loops to measurable outcomes create good software.
  • Estimates are guess-timates; they are mostly useless. There is a geometric relationship between the length of an estimate and its inaccuracy.
  • Software does not scale. Software teams do not scale. Architecture should be as much about enabling small teams to work on small components as the technical requirements of the software.

Because the technical and motivational aspects of software development are so key, I’m very intrigued by the zero-management approaches of organisations such as Valve and GitHub. I thoroughly recommend reading the Valve employee handbook and Michael Abrash’s blog.  Maybe that’s the way forward? The original agile manifesto was very much about self organizing teams, it would be great if we could get back to that. In the meantime, the word ‘agile’ has become so abused, that we should stop using it.

bellware-bury-agile

99 comments:

Unknown said...

Awesome post. I totally agree.

Unknown said...

Learn about all good and important methodologies, and take what fits your character and team. Any religious treatment of any method is not natural. Thanks for the good read.

Unknown said...

I say take ownership of it and reclaim it for what it was promised to be! I whole heartedly agree that the fake-agile you are describing is an easy trap to fall into. As a product owner who used to be a developer and scrum master I fight battles every day to keep things properly agile. An empowered team that pulls from the backlog and uses best practices to deliver quality code.
We should come up with a name for the fake agile. How about FRAgile? F****ing Ruined Agile.

Unknown said...

Good rant, and I agree that what you have observed happens too much, and is not ideal.

It is a little unclear whether you are attacking "agile" or the non-technical manager.

One of the ideas behind the agile management practices, is servant leadership and self organising teams.

The agile community recognised the problems you raise, so that is why "agile", unlike traditional methods, fights against the concept of "non-technical managers being in charge of teams".

But even where managers have understood this (and at some point along the chain there will be managers that do important work that isn't technical), developers have sat there and said, we don't care about management, we don't care about contracts, we don't care about pricing, we don't care about how knowledge work flows through a system, we just want to program.

So what are you to do?

Non-technical managers don't need to know to program, but they do need to know something about the economics of knowledge work and servant leadership, they need domain knowledge, and extra soft skills like team leadership, negotiations, and dealing with customers.

Then they need to ditch the idea of "being in charge of a team". The customer is in charge, and if he is not technical I fully expect the programmers to complain just as much about the customer as the non-technical manager, but that is ok.

The team is second ranking after the customer. They do all the work, and are more important than the managers.

The manager should definitely be a third place role. They should simply introduce the customer to the team, do the paperwork, keep an eye on non-technical things, get out of the way and serve the team.

(Of course then the programers will whinge that they have too much customer contact and just want to hide in the basement and program, and start pleading for some manager to be in charge of them again)

Anonymous said...

Don't forget that there are technical managers who are simply outdated! I've been in and seen too many situations where and out of touch manager is dictating practises and implementing plans they don't understand.

Jonathan Swift said...

I have been at first contemplating, then looking into then actually studying Cult Phenomena since 1974 or maybe 1975, when from time to time I used to read about the weird news coming out of Communist Cambodia in Jack Anderson's editorial page columns.

While an arch-conservative, Anderson was really quite lucid, and thoughtful, as compared to today's conservative columnists.

He at first reported that quite suddenly and completely out of nowhere, all of the Cambodian children were turning their own parents into the authorities.

A month or two later, everyone in the entire nation who wore eyeglasses disappeared overnight.

Later I started learning about NAZI Germany.

When Jim Jones managed to move nine hundred cups of Cyanide-laced Kool-Ade from his Jonestown, Guyana refreshing sports drink stand, I had a pretty good idea as to how that came to pass.

I learned how to pull that same stunt off myself when I joined the Human Potential Movement by enrolling in the Lifespring Course in Los Angeles over the Summer and early fall of 1984.

(LA LP19 - that it, the nineteenth session of LP's Leadership Program, which was free of charge however we were highly pressured to enroll five other people. I myself enrolled three fellow Caltech students, two of whom went completely out of their trees, the third had a real hard time yet managed to graduate, however he is no longer in Science.)

When I started pointing out to other members of "The Caltech Community" that just maybe it was bad for us that no one from "The Outside World" ever so much as set foot on campus - not even the Pasadena Police! - I was beaten senseless then expelled for sleeping on a couch in a Ricketts House hallway after the Institute's Master of Student Housing told me not to.

I learned about how all manner of cults work - including cargo cults - as a result of taking Cultural Field Anthropolist Stuart Schlegel's most-excellent "Anthropology of Religion" course at the University of California Santa Cruz.

Dr. Schlegel later wrote that I had written the most-moving term paper he had read in twenty years, just twelve rather spontaneously composed pages entitled "Surviving Suicide", which informed "Stu" - as he preferred to be called - that his discussion of Kant's Construction of Reality enabled me to overcome quite profoundly suicidal depression.

The world recoiled in horror as a result of the Spring 1997 San Diego mass suicide of the Heaven's Gate UFO Cult.

They all thought that comet Hale-Bopp was an incoming alien spacecraft, coming to destroy the Earth. While Heaven's Gate was adamantly opposed to suicide, they considered themselves not to be dying, but to be surviving the oncoming onslaught, when they "Shed Their Vehicles" by eating Phenobarbital-laced Applesauce and Pudding.

Drop "Marshall Applewhite" into YouTube.

If you want to know how cult leaders work, watch Applewhite's eyes closely. Note that his eyes are wide open and he never blinks. That's a rather extreme form of NLP - Neurolinguistic Programming. While it is easy to resist if one knows what one is doing, if one practices NLP on unsuspecting people it is quite easy to manipulate their minds.

Whereas most were aghast because they didn't understand, I myself damn near checked into the nuthouse because I understood all too well, and so went almost off the deep end myself.

What concerned me wasn't that the same would happen to me, but that all of the Heaven's Gate followers other than Marshall Applewhite were, to the very best of my knowledge, perfectly happy normal young people before they came to work as web designers for the Heaven's Gate web design consultancy.

It was to explain to others, that all is not as it seems, the reality is not concrete but fluid, that I commenced what been now been seventeen years of writing for the most part on the topic of mental illness, not just my own but that of others, as well as anthropology, sociology and the like

Unknown said...

Don't work for companies where your boss sucks, and I am saying this as an executive and software manager. Just leave. A well formed team is all that is necessary to hit deadlines and meet customer expectations. Their success has nothing to do with the development methodology, if they are not a cohesive and talented team, they will fail under any methodology or under the slightest customer pressure. My job is to build a great team, not necessarily filled by A+ players, who can get along and work together. I will take 4 junior guys over 4 senior guys who think they are software geniuses (clearly software is a deeply technical discipline, not the playground of neophytes..hahaha) who can work together. My job is to find the roadblocks, including asshole developers, overbearing peer executives driving a deadline agenda, and technology/methodology evangelists and reduce or remove their daily influence on the team that is well performing otherwise. Do I let non-technical Project Managers have access to my team? Of course,
At the end of the day, very few professionals, developers included, have the skill and capacity to do breakthrough work, lets not paint them as gods that need no supervision, blocking, tackling, or code reviews. Could managers improve, both technical and non-technical, absolutely. Its a skill just like software development, pattern recognition and problem solving, some can do it, some can't.

Anonymous said...

But what's the alternative ? It's easy to criticize something , without an alternative.

Julian Kennedy said...

Fantastic post! I have seen this "cargo cult" too many times.

Anonymous said...

Got out of a six month agile catastrophe recently. Non technical agile cult management put a completely unqualified former tech support worker as our "scrum master." Everyone quit or was so pissed off they simply stopped working. The product was essential to the success of the company...it will probably take another year for them to ship it.

That product really needed good product management and design, which we didnt have. Sprinkling magic agile fairy dust on a fundamentally broken team with bad players and missing talent wont fix anything. No amount of process can make the wrong people into the right ones.

Its sad but you are dead on. There is no room for non technical people in this space. The most successful teams I have seen made use of "tech leads" who were engineers capable of running a project. Good luck hiring people that good who want to work at your dumb company run by lawyers, sales and marketing. Anyone capable of being a tech lead is probably ripping down double six figures at google.

Anonymous said...

"Please don’t put non-technical managers in charge of software developers." - Never a truer word spoken!

Chris Jones said...

I feel like this is a strawman argument... The role of non-technical manager doesn't exist in Scrum, XP, Kanban or any flavor of agile that I've ever heard of. So arguing that "Agile has failed" and then describing a not very agile process seems a bit misleading.

In our organisation agile principles mean that the team is empowered and face-to-face communication with non-technical stakeholders is highly valued.

I'd suggest anyone who is struggling to deliver effectivly go back to basics and look at the underlying principles of the Agile movement and try to understand how practices like iterations, stand-ups and retrospectives support these and take positive steps to improve your work environment for technical and non-technical folks.

Anonymous said...

This is the biggest and most self pretentious pile of crap I read in ages, and only self indulgent, bad coders could be so condescending with themselves. Software does not scale? Crap - it's because you can only focus on the nuts and bolts and not the overall picture. A project managers needs to have some degree of understanding of the technical complexity - but his/her main role is to deliver a product within a defined budget and timescale - all things that programmers cannot do. Someone always pays the programmers' salaries - and yet you seem to refuse and deny this reality and pretend to be self sufficient gods. Do grow up, people - you eat and pay rent like anyone else. Cristiano

Anonymous said...

Nice article!!

DE said...

http://www.halfarsedagilemanifesto.org/

Jan Wedekind said...

Well said!

Jan Wedekind said...
This comment has been removed by the author.
Jan Wedekind said...
This comment has been removed by the author.
Andrew Jerrison said...

Whenever I have encountered 'Agile' it does always seem to be a management tool, without real understanding of the technical side. It is not usually implemented in the favour of the engineer. As you say, this is probably what has led to its negative image in the eyes of those who it is meant to help!

It seems to have been hijacked by an unholy cabal of non-IT literate managers who think they've got a way to measure performance of their 'resources', and the kind of developers who are happy to sit at a desk code-bashing like a robot all day.

There doesn't seem to be much room for the human side of things such as a varied job such as design work and customer liaison, or career advancement, or even swapping team/jobs easily (thanks to the 'no docs' rules). Whatever happened to the old 'software engineer' role, i.e. senior technical people who managed both the whole project and the people? It kind of worked you know.

Moan over. :-)

Dave Aronson said...

"If your role is simply asking for estimates and enforcing the agile rituals"... then you can be replaced by a small shell script. And then we can delete it.

Anonymous said...

Great comment, Blaine!

Patrick McElhaney said...

Let's be careful not to look at all the failed "agile" projects and assume airstrips, controllers, and headphones prevent planes from landing.

As Chris Jones said, there's no place in the agile practices for a "non-technical manager", or even a manager. Teams are self organizing.

Scrum can be considered a "starter kit" -- practices that have proved in many different teams to be highly effective. Over time the team is supposed tweak the system through the practice of kaizen (continuous improvement).

Unfortunately, the "agile" most people experience today is the same old management practices with new labels: status meetings are now called "stand ups." Schedules handed down from above are now "estimates." Big design up front and long bug fixing cycles don't have a place in Scrum, so we've added "Sprint 0" and "hardening sprints".

The retrospective, IMHO the most valuable of the four Scrum rituals, is usually the first to be tossed out.

Believe it or not, cargo planes really do land when there are people on the ground with real radios and a plan.

Peter said...

Awesome post. reading this made my day

Mike said...

I come from the consulting world and in that environment, a good non-technical manager is a huge asset. What does a good non-technical manager look like? They trust their technical team (and especially the technical team lead) to deliver based on design and to monitor the development process and report up to them when things are going off the rails in terms of time.

They also control scope. All too often development projects come in over budget and time because scope increases go out of control.

I've always been suspicious of agile due to it's lack of project control. Little long term planning or design. In a consulting environment where the customer is paying for the product based on the hours it took to develop, these controls are critical to success. It always seemed that the agile approach was a reaction to programmers not enjoying being managed.

I do agree that it seems the superficial elements of agile (two week sprints, stand-ups, etc) were adopted thinking they were the magic ingredients for success. In fact, every successful project I've been on has been the result of talented people doing their job and recognizing where they have to adapt the process they use to the unique situation they are in.

Anonymous said...

I'm also interested in the management approaches of Valve and Github. But I believe that's much closer to political analysis than software engineering.

Valve's structure is an anomaly. If the software industry was substantially structured around workers making their own decisions, that's a direct affront to capitalism.

I believe we should pursue structures like this, including starting new cooperatively-run enterprises. But we should not pretend this is simply a matter of methodology: it's at the very core of politics.

Anonymous said...

Agree with your bulleted points, mostly, except for the bits that attack deadlines and estimates. While I nod my head in agreement that estimates are guesses, what alternative do you have? Are you against all estimation or only against estimation of long-timeline efforts? And hard deadlines, they don't guarantee poor quality software. In fact, they can prevent poor quality software. If a product manager asks me for X by date Y, and a customer won't buy unless it's there by date Y, and I know we can't deliver by that date, I can go ahead and focus on those things that have a chance of success.

Anonymous said...

Well said. Techies will get this i9nstantly, non technical managers will no doubt huff and puff and spout justifications.
This stuff really should not be controversial…and it should be standard practise but the "culture" of non-functional IT (ie. Architects and non-coding managers) has now conquered most office environments. Not only is doing software badly very profitable for some, it is also a much easier career than actually working.

We clearly need less coxswains for every rower and less peripheral BS, however programming well is *hard* and it appears that and easier career path is to forgo any technical competence and just stay "high level". Not only is this much easier, it allows time for the important grooming activities so critical in a multi-layer primate hierarchy. Techies appear to be exchangeable with cheap imports and can be linked directly with stuff ups whereas once in a high level "team" clique you are relatively safe. More pay, more safety, less work...with a bleak, soulless existence, but that's a trade off many will make to pay the mortgage.
Automation's wealth effect has led to many adult day care roles…this is just another example.

Ross said...

Reminded me of Dave Thomas post http://pragdave.me/blog/2014/03/04/time-to-kill-agile/

Cliff Berg said...

I agree completely with your sentiment.

But interestingly, the things that you say that "any software development organisation must address" are all core agile ideas.

So agile - as a set of ideas - is not the problem.

Agilists will then say, "The problem is that these organizations are not really doing agile: they are 'doing agile' - not 'being agile'".

No. That is the wrong way to look at it. The truth is that while agile is a great set of ideas, the agile community has not answered the very important question of how agile should work in a large, complex, traditional organization.

Just saying that a big company/agency needs to be truer to agile value is not saying anything: one might as well say that the reason we can't have world peace is because world governments are not more trusting. Saying that accomplishes nothing.

The fact also is that large organizations cannot adopt agile that way that it is practiced in small ones.

But there is a-lot of truth to what you say about too much focus on agile ceremonies. Most of the ceremonies that agile coaches pay so much attention to don't really matter. What matters is servant leaders. What matters is small teams. What matters is customer involvement. What matters is having small demonstrable units of work. What matters is having smart people on the team, or at least people with the right skills and a willingness to learn. What matters is having someone who is technically astute making sure that everything works as a whole. All the ceremonies don't matter.

Anonymous said...

Agree with this comment from the article and many others posted, but - most of you 'dev' guys just aren't that good. 1 in 5 or maybe 10 can develop, most can barely code. You obviously have your own headphones on and they are up your arse...
The skills and talents of individual programmers are the main determinant of software quality. No amount of management, methodology, or high-level architecture astronautism can compensate for a poor quality team.

Anonymous said...

Agile is dead! Long live Agile2!

Anonymous said...

How true !! Certainly the projects success depends on capability its technical team

Anonymous said...

What a horrible article. By extension of your logic we would have to say that technical managers should never be in a position to run the business. I've seen dozens of "non-technical" managers who have done excellent jobs managing software teams (Agile and not). The failure of Agile, and waterfall, and XP, and whatever, lies not in the methodology and not with non-technical managers. It lies squarely with the developer not taking personal responsibility for producing quality code that provides business value.

Anonymous said...

"Software does not scale." -- WTF really?

"Estimates are guess-timates" -- when last dit you look at the definition of "Estimate", http://lmgtfy.com/?q=Estimates+define

Please understand 'agile' and the words you use before you say 'agile has failed'

Anonymous said...

http://www.dummies.com/how-to/content/the-12-agile-principles.html

Is that what any system strive for?

Do I need to say more?

Unknown said...

I have to agree with others that this feels like a straw man...

The recently published 2013 State of Agile Report states (in page 7) that "respondents reported year-over-year improvements in ALL areas measured" from adopting agile methodologies and "73% reporting faster time to complete a project"and "92% reporting improved ability to manage changing priorities" (page 8). And in page 9, "51% of respondents said that the majority, if not all, of their agile projects have been successful."

You may claim that the purveyor of the numbers usually has a vested interest in their interpretation, but you may find similar numbers in various others reports from Forrester and Gartner...

Agility is now a success and the teams and companies that have not embraced it yet are laggards.

I started with RUP back in 2000. I wrote hundreds of use cases, created an infinite number of sequence diagrams and led dozens of projects from inception through transition. We had successful projects (and unsuccessful ones too), but we were always asking ourselves whether there were leaner and more valuable ways of bringing value to our clients. We were CMMI3, then CMMI5. We learned to be open-minded, inspect, adapt and encourage our organization to conduct experiments and gather lessons from the results. In 2006, we started our agile journey and two years later we were already a 100%-Agile company - full enterprise-wide adoption: from development and maintenance teams, through operation, human resources, marketing and sales. We use Scrum and Kanban and now have lean principles in our DNA. Our success rate is 97%.

We have technical Scrum Masters that obviously started their careers developing and understand the nuts and bolts of the code, but we do have scrum masters that work greatly in solving blocks and mastering communication with the business, guaranteeing integration, providing coaching, cadence and supporting the team without a technical background. It is possible.

I ask you: what is your vision for the future? How can we make sure that agility gets to the next level? How can we build a new Boeing jetliner in an agile way for example? How can we break the silos of large organizations and engage dozens or even hundreds of people in a large program? How can we make sure that one agile team of seven people learn with lessons from other teams and avoid waste? And how can they all work together to build something bigger - keeping the values and principles we so very much respect?

Best!

Marc Ziman said...

I think the problem is the emphasis Scrum puts on project management and the associated rituals. XP focusses on technical practices (TDD/Pair programming etc). Without strong technical practices the project management aspects are next to useless. Martin Fowler calls this Flacid Agile. Too few developers seem interested in good technical practices in my experience. This is a major impediment.

Eryts said...

Great post and I'm living a similar situation in my project.
This is my first "Agile" project and I can see there are many deficencies on how we are doing things. Probably is the management that always want it quick and dirty or is the lack of knowledge from the whole team on what "Agile" means.
The thing is that we should start thinking more on how to make our developers more happy instead of how to make them more efficient, because want we really want are great resources doing software, not robots producing code. If we want robots, let's code some robots to do it for us.

Anonymous said...

Whether you develop software or build skyscrapers, it all boils down to good project management. A construction project manager does not need to operate a payloader to build architecture wonders nor does a software pm need to know Java to the detail to develop a breakthrough application. Projects fail because of a number of factors and it's surely not because the Manager does not know how to operate a large hydraulic excavator nor code classes or methods. Compared to construction, software development is still in its infancy.

ForceCrate said...

Agile defenders use management metrics to defend agile?! This is another problem in itself; the belief that you can accurately measure human activity! Non-technical managers love to measure because they are incapable of judgement. Agile encourages this stupid behavior. One day I may see a deadline disappear because of programmer input but I won't hold my breath waiting.

Anonymous said...

I agree with your view of what Agile has become in many organizations. Partly I believe it is our own fault. As developers, we want to write code not "organize" ourselves. So someone with less technical knowledge is appointed SCRM Master. (Why waste a good developer, right.) So we abrogate our responsibility to organize our team.

The other issue is one of management. Someone, not developers, will be managing the company where we write software. CEOs, CFOs, accountants and they will all want to know what the development team is doing, and they should. It's their job like ours is to write code.
So there will always be a "new thing". (waterfall was new once) and it will work well until it becomes form over substance. That takes diligent, consistent, non-code related work on the part of all developers.

Unknown said...

According to my experience (SW dev, technical mng), the success of an software development project depends on far more that just the methodology.

It depends on the environment, which is used to deliver the product.

Environment
With environment I mean all methodologies, human skills, human nature, tools, offices, languages and anything you can think of having an influence on the software development project.

I see the key to success in creating the environment fitting the software purpose you want to develop and deliver, and then continuously improving the environment.

Responsibility
Blaming non-technical management for failed "agile" software in general is actually denying your own responsibility. The responsibility does not originate from the "agile", it comes with the contract you have signed for delivering the job. Some of us tend to forget that sometimes.
In agile the technical roles are responsible for the technical richness of the environment. So if you feel, that it's all about meetings, sprints, pokers and all the technical practices are lost than you have failed to keep your responsibility. To be honest, you cannot wait for a product owner to tell you i.e. that you should set-up some continuous integration server.

Constant improvement
Everybody is responsible, thus if anyone sees an impediment, than he should point at it. If impediments are ignored, than you fail to improve your environment and in the end fail to deliver the product. That's continuous improvement of the environment. Actually, it's not an invention of scrum or agile, it's the common strategy of survival in a constantly changing world and should be practiced in any methodology (even those waterfalls).

Summary
If you feel, that the agile approach combined with all the kinds of meetings and reviews and missing technical richness and talk and talk and talk is making your work ineffective, then
1. it's actually you in the first place, who is not fitting the specific agile implementation,
2a. you could try to find the impediments in you and improve your self,
2b. you could look for something more fitting to your nature (incl. the amount of responsibility you are willing to keep),
2c. you could keep things as they are, writing blogs to let out the stress,
(this is not a test, it's just a choice ;)

In fact, agile principles are NOT fitting for every software, every customer, every team ...

Misha said...

I think the single best thing about agile perspectives in software development is that it encourages frequent and direct connections between a software developer and those for whom the software is built. Those involved in direct and frequent connections should be: decision-makers for priority questions; functional experts; and technical experts. A manager without functional or technical expertise would have nothing to offer and could only get in the way of progress.

Anonymous said...

Can I recommend "The life Cycle of a Silver Bullet" by Sarah Sheard from a few years ago http://www.crosstalkonline.org/storage/issue-archives/2003/200307/200307-Sheard.pdf.

It's still a great article that fully reflects the thesis of the blog post.

Unknown said...

Dude!!
Did you read my mind while I was riding in an airplane or metro/subway?

Excellent post! +1 times 5 worthy.
It was almost as if my thoughts just popped onto the web page.

:-)

ScarvedOne said...

Every decade, another new methodology gets turned into a cult and run into the ground. I know! Let's invent some more!

Hey, Rocky, watch me pull a rabbit out of my hat!

Mike said...

Agile is no different from any other business, manufacturing, technical or management "methodology du jour". Problems arise when the buzzword is adopted but the methodology is inadequately adopted or perhaps even applied where it does not belong.

Anonymous said...

Huh. We've been doing Agile for about three years now after 12 years of basically unstructured ad hoc development practices. The productivity increase is massive. But we don't have non-technical managers leading technical teams. The non-technial (product owners) interface with the tech lead who organizes the teams. All that said, we are not rigid on any aspect of it, but have found organically that daily standups, retrospectives, and, particularly, shorter sprints (for us usually four weeks) has resulted in massively increased productivity. We went through formalized "scrum master" training phase, but have largely abandoned that as voodoo training. As others have said, use what works for you. Agile can work, even the more modern agile. Just don't force it to try to work if it isn't for you.

Steve Bjorg said...

There is some truth to the rant, but leaving programmers in charge of a project is just as bad (as evidenced by Fred Brooks who few would challenge on technical chops). Someone needs to understand the objectives of what the product needs to accomplish and that person must be at some distance from the coding effort to not be embroiled in it. Otherwise, the outcome is an over-engineered, poorly market-focused product that is a jewel of human ingenuity that serves no particular need.

Unknown said...

Some very good comments here that I agree with. Recently seems to be even more fashionable than usual to criticise Agile and I think it is often a good thing because, as explained in this article: http://everydaylean.info/2013/09/heart-agile/ the heart of agile is all about the pursuit of better ways of working, not about any specific practices or methodologies. Most flavours of Agile are not methodologies for example Scrum is an empirical process framework, nothing more or less.

Yes, there are countless companies that adopt the agile buzzwords but don't have a clue how to manage their projects or their staff. The same applies to waterfall etc.

This is the only thing I can recommend for everyone:
http://alistair.cockburn.us/Oath+of+Non-Allegiance

Anonymous said...

It appears to me you do not know Agile well or have worked with a poor implementation of Agile. The key here is collaboration. Collaboration with your end-user, architects, testers, developers. The technical people write the code, the clients identify the problem.

This sounds like a bitter rant fem someone stuck in the past.

Anonymous said...

The author identifies a real problem in organizations where they try and use the 'Agile process' as a golden hammer, without the right people and in the spirit of Agile. If a technology business places leadership in folks (managers, architects, engineers, people, insert title here) who do not understand technology, the results will be poor. Agile does not fix lack of vision and incompetence. In large organizations, Agile is another 'fad' like Six Sigma that has been evangelized and commoditized. We can only imagine how much $$$ is spent on training and consulting just to quote learn the process quote. But then the spirit of Agile is not there- estimations become deadlines, poor code is written, re-factoring does not happen. BUT the processes like two week sprints and retrospectives occur that act as checkboxes so the business can claim it is up and coming and lean. Keep in mind this blog post is most applicable to big, entrenched companies that in many cases have corrupted the Agile philosophy.

Kevin said...

great retrospective. Let's use that to improve the next iteration.

Anonymous said...

I'd like to suggest a twist on the "project management" aspect of this discussion. I find that whether the project manager is actually technical or not may not be that critical (although I agree having a technical PM is an asset in most cases). I find that the ability of the PM to manage *expectations* can make the difference, and I'm referring to ALL expectations here - those of the client, those of the development team, those of the management, etc. as they relate to the project requirements and objectives. A good project manager should be able to detect the "larger" expectation gaps and address them even before the contract is in place. As the delivery takes place, the "smaller" expectation gaps should be addressed before issues or actual conflicts arise. If expectations are adequately managed throughout the project, then the project is very likely to unfold well, and be a success for all involved. Competence of the delivery team does matter a lot, but the methodology might not be that important if correct expectations are set and maintained with all stakeholders throughout the execution.

Unknown said...

Many of the concerns raised here sound true to me. I went through this same exercise a couple of years ago.

You get lost in the backlogs, points, and measurements and other prescriptives. You almost get a licence to suck by doing some Scrum dance. As long as the points come out right, we're good!

Harald M. said...

I appreciate your concerns. But you don't seem to have a clue about humans and humanity. It seems you are a "real programmer" - as opposed to someone who is concerned for all people.

(For the record, I have been a programmer AND a manager in the last 20 years; I know "both sides" - but they are NOT "sides", as you want to make us believe).

Harald M.

Pewem said...

I disagree the fact that a manager have to be a developper to be a good manager.
I think, that like other businesses the manager have to discuss with the specialists to have the good indicators.

I agree that few developpers know how to measure technical debt, and that the main matter.
We have to promulgate the methods and tools which measure technical debt.

Tommy said...

Good one, especially the first part! I don't agree that you can't lead a software development project without technical background tho, as long as you make sure to trust the team on the technical decisions and adhere to any feedback they provide to the plan.

I wrote an article about a year ago called "What is agile?" that discuss just this that agile is now only about processes and tools even tho the first amendment in the manifesto is "Individuals and interactions over processes and tools".

You can read it here if you'd like.

Anonymous said...

I have to call BS on the "No hard deadlines" requirement. Everyone has deadlines. When business needs a product in time to satisfy a contract or other need, the deadline is real. Are there a lot of deadlines imposed that are arbitrary? Likely, but coddling programmers as coding divas, with no responsibility for the success of the business, leads to the programmers having to find new jobs and blaming the business for it.

Sai said...

@Mike - "Software does not scale." - what did you mean by this line ? (I am really asking, not pointing out anything).

Mike Hadlow said...

Sai,

"Software does not scale." I agree it's quite a broad comment, and patently untrue because there are plenty of large scale successful software systems out there. This is a blog post in itself, and one I've been meaning to write for a long time, but I guess what I'm saying is that as a software system grows, you can't just keep writing more and more code in the same way and expect it to scale linearly.

There are all kinds of scale effects that manifest themselves, and beyond a certain size, software becomes very hard to understand and maintain. You need to adopt architectural strategies to divide the software up into manageable components with clear boundaries and contracts. Software teams do not scale well either. There's a very important human aspect to software architecture; it needs to be structured in such a way that teams can be small. The worst thing an organization can do is throw more developers at a large monolithic system.

Any successful leader of a software team needs to address these issues and because they are technical, it simply reinforces my belief that non-technical managers are a waste of space.

Anonymous said...

Well, the truth is always in the middle. Managers still do destroy projects by micro-management. And developers still destroy projects by very bad technical practices.
The idea behind Agile is the self-organizing team : a team that learns even from each other and improves continuously that way. Every advancement a team member achieves is automatically communicated and duplicated to the others. And yes, this team defines with its Product Owner its own deadline and budget in negotiation with the client. They deliver estimates, provide information about what high priority features can be delivered in the requested remaining time with the offered amount of money. They deliver the solution nit by bit, get feedback and correct their course, where necessary. This all works automatically without the need of any project manager. A PM is truly just an obstacle and an unnecessary overhead for such a working team.

BUT - is every team able to become such a self-organizing unit ? NO !!! No, no, and again no !
And that is the problem.
Many people (including SW engineers) are just happy to be told what to do, they do their 9-5 job, and are in essence more interested in other things. And many managers cannot become servant leaders.

In my experience it all boils down to : trust !
If the team can show that they can deliver what they promised, it will be easier for the managers to be able to "retreat" to a function of pure servant leader. The team earns the managers' trust.
And if a manager gives the team some credits at the beginning and is not micro-managing for a while, the team begins to trust that they can actually be self-responsible and self-organized, that they are in fact truly responsible for their own fate. That provides a totally different level of motivation.

But again - it will never work with everybody. It is the combination of the right personalities in the right environment that makes it successful. Not the practices themselves (most of them should be used in traditional/waterfall environments too - the practices are good, but they do make a team or an organization "Agile").

Filippo De Luca said...

I love this post, it summarize my toughs that I don't have the "balls" to say.

Anonymous said...

Don't confuse "Agile" with "Scrum"; Scrum is a type of Agile, and in the original versions did not specify any technical practices (ala XP) but now apparently does.

The "Agile" that Uncle Bob refers to in his book is XP, not Scrum (especially not the original Scrum).

It makes no sense to declare Agile dead since so much of what was intended (more craftsmanship, smaller increments of delviery, focus on testing, etc.) are pretty much de rigeur in the software development industry now (though maybe not as much in Enterprise SD).

Anonymous said...

Well this qualifies as a rant for sure. I have to agree with many of the points. Many articles like this conflate the issue of the method with the issue of the implementation.
That is a bit of an issue here as well.
Altogether it's very amusing reading the posts of the non-technical defending non-technical management.
Well we all have mortgages I guess.
....JW

Unknown said...

Great post, more people need to feel empowered enough to stand up and speak their true feelings surrounding this thorny issue rather than being frowned upon as anti progression/trouble makers/stick in the muds/etc...

I have for a couple of large companies recently who call themselves agile, use extreme programming techniques etc... and just don't deliver any product.

This whole agile drive appears to have been hijacked by new hipster-esque developers out to prove themselves as super developers. An excuse for little or no documentation; too much time focusing on purism and writing tests that are often far more complex than the application they are testing and harder to maintain, although no extra time is allowed for the creation of such tests, like they appear by magic??????

Cannot wait for the tried and tested to come back around where what matters most is delivering what the customer has requested.....

Ignatius said...

I would just like to caveat your warning about not using 'non technical managers' to manage software projects. A good non-technical manager will recgnise that where the expertise in the team lies and will trust them to make the right technical decisions. S/he will not use the agile framework to beat the team but will make sure the framework supports what they are trying to achieve.
Your concern appears to be more around the 'command and control' behaviours of certain managers (technical or non-technical) which is the real problem with agile implementation in the real world. Without addressing the cultural/behavioural shift required in an organisation, there will be a complete mismatch between management and development that cannot possibly be fixed.

Shubhangi said...

Agile has not failed, its just the way you use agile is flawed. Actually no software methodology in complete, they all need bits of tweaks and adjustment.

Anonymous said...

Yep, it's self-pretentious crap, people who think that Dilbert defines how they should think. Agile has become an excuse and a refuge for the prima donna developer who doesn't want to commit to a deadline, refuses to estimate when they'll be done (until they're done), and sneers at doing anything else besides heads-down code, no matter what the need. Fire those zealots; they just drag down an organization far more than the much-sneered-at "non-technical manager".

Anonymous said...

Yes Agile has been subverted. Too many wrong people involved who subjugate programmers and programmers just accept this not standing up.

bryan.c.oconnell said...

Mike, thanks for your thoughts. I've collected some similar articles and shared some comments here: http://goo.gl/Gxu5qs

Anonymous said...

So I think a better "world peace" analogy to large corporations doing agile is Putin spending billions of dollars on the Olympics and annexing Crimea the next week. And then blaming peace activists for not showing him how to deliver on the promise of the Olympics.

I think the real problem with the community are the unaccountable companies behaving as petulant children, trying to blame the community for their software boondoggles. Especially the large 1950's corporations with their 1950's culture that refuse to acknowledge that they have to start acting like real software companies.

Jordan said...

Alright, cracks fingers, lets clear some of the air here.
This is clearly written by a developer who had a bad day. I get it, I even agree with some of it, but by no means is "Why Agile Has Failed" a valid blanket statement for the entire industry.
Here is a metaphor for you. If I buy a Honda Civic with my own money, out of my own pocket. I am going to be more prone to saying good things to my friends about Hondas. "Hondas are the best, most reliable cares for the money." Something to that affect, because after all, I made the decision to spend money on that particular car, and I'm a pretty smart guy, right? Therefore it is a logical decision everyone else should consider.
Now, lets substitute this argument into a similar path of thinking. "I have had bad experiences with Agile on my projects as a developer being managed by non-technical managers". I'm an industry professional, and I'm pretty smart, right? Therefore my experience should / will be the same for everyone else in the same career as me, and so they should take my statement of "Agile has failed" as the truth.
It is bullshit, and this is coming from a technical product/project manager and former BA with a background in development / network engineering.
I always get frustrated with these kinds of arguments. What purpose does it serve if you have no alternative? In what world has any of us been involved with a project that has a process that EVERYONE, including the devs, agree on? I'll wait for someone to raise their hand. No one? Ok.
I have never, repeat, never, been at a client's organization when they have implemented agile in the way it was intended. The fault doesn't lie with any one person, or role, as this article makes it out to be. In fact, it lies on a ton of different factors, from software developers building features that are unnecessary in an order not aligned with the business goals to non-technical managers dictating from Mt. Olympus, to improper requirements being gathered and communicated.
All of this is not specific to Agile, it is specific to humans. Humans, in the structure of an organization are fallible. They make mistakes, they change their minds, their ego dictates their actions. All of this cannot be solved with the removal of roles, or the implementation of any framework.
Further, if Agile was failing, and wasn't considered valuable by organizations, than how come Agile consultants have a roof over their heads? How come they get paid real money? Why do we continue to see article after article over how one framework is dying over another, year after year? Why is it that the majority of them are coming from frustrated developers and not the people who sign the checks?
Just look at stack overflow for the example. Zealotry at it's finest. This language over that one, this framework over the other, if you learn x you're not a real programmer, if you learn Y first you're an idiot, etc... Every GOOD developer I work with writes all of this kind of shit off completely. Every BAD developer I've worked with (keyword worked) relies on these kinds of thoughts to get through their day. Maybe they write an article about it.
At the end of the day, there are always going to be people who think what you do is useless; yes, even you developers (you're viewed as a commodity). Everyone thinks they can the chief in the tribe, and everyone thinks their past experiences will follow them into the present and then the future. But the facts really are, there are people who are good at what they do, and their are people who are bad at what they do; in all roles, both technical and non-technical, Agile consultants, developers, managers, etc...
So can we please, please, stop beating this dead horse by firing the people who suck, hire/retain the people who don't, and get them all to agree on the best way to get things done and stick to it until things need to evolve?

Unknown said...

some of us non technical coaches still actually get agile. the rituals are only the means to an end. it is the empowerment of the technical teams that drive product improvement. agile works well when done well.

Anonymous said...

Where are those 10x and 100x improvements eh? Snake oil at its finest...

charles mark ferguson said...

I think you raise some valuable ideas but some of them I question:

* non-technical managers: my 20 yrs as a dev doesn't gel with this. The best manager I ever had was non technical. But she was very adept at honing in on exactly what she needed to know about the domain (but not more). Excellent people skills, encyclopedic domain knowledge, never scared to go toe to toe with management on behalf of her team or to push back against bs requirements. And a number of my weakest managers have been highly technical.
* I will back the output of a solid team with a mix of high-to-medium-ability members against a team entirely of genius assholes any day.
* James Shore's article says nothing about Agile failing, just about the spread of cargo-cult Agile. Just saying.

Anonymous said...

I have seen just about every crazy thing you can imagine happen under the guise of "Agile" process. Like one week sprints followed by the weekly production release. Like two week sprints that encompassed major pieces of functionality that were time-boxed then forgotten once the next sprint started. No matter the status of said functionality. In that project, I once worked 24 hours straight fixing 18 defects the weekend before a release.

I also once saw a manager fire an "agile coach" who was getting paid $125 an hour, and the project took off like a rocket after that.

I don't have a problem with non-technical managers - I've had some good ones. What made them good? They took the heat, and stayed out of our way. Because they knew we were good, and that we delivered.

However - there are people out there with no background AT ALL in software development who are getting some sort of "SCRUM certification" and schmoozing their way onto development teams. That is never going to work in my opinion. I once worked with a guy that fits that description, who didn't understand basic, fundamental concepts about IT in general. It is a real head-scratcher how that could ever happen, but it does.

There is no magic bullet. You have to have good people.

Anonymous said...

The first sentence says it all "They identified that the programmer is the central actor in the creation of software, and that the best software grows and evolves organically in contact with its users."

The customer is the central "actor" (another agile term i hate) in the entire process. The programmer is just a tool to achieve the end result. Wake up and get some common business sense. Also, software isn't grown organically like a plant. Software needs to be precise and written preferable once based on precise rules (aka specifications), smart coding and bullet proof testing. It's all about precision not iteration.

Anonymous said...

Consider me for a moment.

Years of experience in developing software solutions, on a variety of platforms - working on small teams and large teams from time to time.

Whether it was waterfall or something else, I always had the curiosity and motivation to approach "the business" whenever I didn't understand a requirement, or if it was incomplete or simply if my intuition told me it wasn't right. I had good managers, technical AND non-technical, who recognized a good thing and let it ride. They saw results, they trusted me, and they got out of the way. Things got done, and got done well.

From 2010 to the present - I am increasingly exposed to Agile - specifically the Scrum variety. In some cases, mere lip service and we go about our business and deliver results. However, the hype reaches a fever pitch as Agile adoption starts to crest.

Finally I land in a project that is completely sold out to the Scrum methodology. Never have I felt more restricted and hemmed-in by a process. Never before have I experienced the crushing overhead of daily meetings, status reports, poker-planning sessions that didn't really "plan" anything. Never before had I been asked to plan, design, develop, and deliver complex functionality in the span of a few days. That by the way, was intended to be production quality.

My exposure to any business people or product owners was completely cut off. We had project managers, analysts, "UX architects", Scrum masters, etc. who interacted with the product owners. Not one single developer.

Documented requirements for an epic or a story were typically a sentence or two, sometimes stretched to a paragraph if we were lucky. "Planning" sessions consisted of a half-day to maybe a day of rambling discussions where no decisions were made or any direction given. The developers on the "team" were asked for input from time to time, which was summarily dismissed or immediately trumped by a contrary opinion from a non-developer.

Anyway, my good friend of 15 years, who I consider to be a better developer than myself, quit in frustration. He is now the proud owner of a Chick-Fil-A franchise in Alabama. I held on in that situation for another six months and then I left as well. I took an Architect job at a different company, who has cautiously adopted Agile methodologies on a FEW projects as a pilot. I just don't think I can function as a developer in that kind of environment. Anyone who is "selling" Agile, is going to sell management on the things they want to hear. Be warned.

Anonymous said...

Attack non-technical managers all you like. Developers left to their own devices will almost always develop a product that has no meaning in the marketplace. I've seen it time and time again at my company.

chrzy said...

Very good article. Agile methodology made my life miserable buat one point, I was brought in as a system tester on an air force project. ,
'The gang of four' developers had years to implement the project with little focus and direction. It was just a free for all to do what you want. I could not make good programmers out of childish ones and the gang of four freaked when presented with real testing and a geometric progression of increasing issues and bugs. Kill the messenger of course. This 'Agile is neither agile nor friendly.
Our government and ourselves as taxpayers are paying for this nonsense. I see Agile advertising and it makes me angry, is this the church of Scientology or something ?

Anonymous said...

Sometimes i get the feeling agile is just a movement trying to get rid of management...

I agree, a lot of managers don't have the necessary skills to earn a management job.

I would be glad if my teams were self organizing and delivering software on time and within budget with whatever methodology or software they feel to, but until now i noticed most of the software developers aren't capable of working in such way, so supervision is needed (monitor and control...). Sad but true...

There is not one "best" methodology, framework, ... and people are always the crucial part.

David Arndt said...

Amen. I've been developing software for 30 years, and being paid well for it in the process (as two mortgages, 3 private high school educations, 3 college educations and 1 wedding will attest. ).

Bottom line: Smart developers have used agile techniques for many years, before "agile" ever became a formalized method. Short, flexible development sprints, agile estimation techniques, etc., have been part of our arsenal for a long time.

What I see in the industry today is a bastardization of these concepts, in the name of adhering to a methodology. The result is a process that is about as "agile" as an overweight elephant with a ball & chain around each foot.

I'm not sure what will satisfy those yearning for an academically attested methodology that is flexible and effective in delivering projects on time. The difficult news is that fundamentally, software development is all about understanding the problem domain, understanding the development tools, common sense, communication, hard work and... honesty.

Agile? I've seen it work... sometimes. Usually, it is applied in ways that are anything BUT agile.

David Arndt said...

Amen. I've been developing software for 30 years, and being paid well for it in the process (as two mortgages, 3 private high school educations, 3 college educations and 1 wedding will attest. ).

Bottom line: Smart developers have used agile techniques for many years, before "agile" ever became a formalized method. Short, flexible development sprints, agile estimation techniques, etc., have been part of our arsenal for a long time.

What I see in the industry today is a bastardization of these concepts, in the name of adhering to a methodology. The result is a process that is about as "agile" as an overweight elephant with a ball & chain around each foot.

I'm not sure what will satisfy those yearning for an academically attested methodology that is flexible and effective in delivering projects on time. The difficult news is that fundamentally, software development is all about understanding the problem domain, understanding the development tools, common sense, communication, hard work and... honesty.

Agile? I've seen it work... sometimes. Usually, it is applied in ways that are anything BUT agile.

Anonymous said...

Agile seems to be used by people that do have a clue about what project management is about. If you do not do project management (analyzing, planning,...) and thus you have unlimited budget, time, resources, scope Agile is the way to go.

Can someone get rid of Agile PM please as it's method is opposite to what a good project is about. And too many companies do not even realize it...

What comes to the technical vs non technical discussion, project management is a technical skill. A real PM can handle projects in SW development, infrastructure, building houses, engineering,... whatever the scope or technical area. So a project manager doesn't need other than PM skills.

Good article, so people with an own opinion do exist. Thanks.

Anonymous said...

Your biggest chip on the shoulder seems to be the non-technical manager which is, frankly, a red-herring.

Pretty much all other industries are able to have non-SME managers create products, so why is software different? The answer, of course, is that it isn't, but that some of the people writting software want to believe that no one can understand them/feel their pain/delve the depths of thier suffering/ad nauseum without a technical degree. Which is just a gate. People who insist on technichal managers are simply gatekeepers, who hope, that if a suitable candidate can't pass through the gate, then they won't have to be managed.

Senior developers, or team leads, fulfill the role of people who make decisions + understand the precious little programmer snowflakes. I wouldn't call them management, but the fact of the matter is that at some point, techs will need to interact with non-techs. Some will listen, some won't.

I have seen examples of technical management who ignore issues and slavishly hold to deadlines. I have seen non-tech managers who will simply trust me, and adjust deadlines etc. when an issue comes up. Technical proficiency has nothing to do with good management. The worst managers in my experience are those that think because they can code, they can manage.

But I know I am wasting my breath here. Too many developers think because they can code, they can do ever other job at a company, better than people who have degrees in that field, because, ya know, everyone else is mentally inferiour. Sad.

Anonymous said...

You had me right up until 'self organizing teams'.

Everything else in your post is stellar and accurate.. but self-organizing teams?

That one little concept requires something I've pretty much never seen.. a team that embodies these things:

its small.
all of its members are highly experienced.
all of its members are emotionally very mature.
all of its members are fairly selfless.
all of its members accept that process must be understood, not just followed.
all of its members are all willing to 'do the boring stuff', and are all capable of leading on their own.

All the teams I've been on have pretty much broken all these rules.. namely:

very few of its members are highly experienced.
very few its members are emotionally mature.
all of its members are fairly selfish.
most of its members want to follow process, and not think.
Most of its members only want the cool/exciting work. Which means the rest end up getting the crap work all the time.
Most of its members don't want to lead.. abhor responsibility.

What I'm describing are very HUMAN factors that prevent utopian situations from forming except in very rare circumstances. Yes.. it CAN happen. I've read about it. But I've literally NEVER seen it.

I've been programming/managing programming for 30+ years now. I think the only real solution to the normal situations we find ourselves in is good management that recognizes individual effort, allows encourages freedom where there is obvious maturity, contains the immature, and seeks to motivate all based on their personality traits. There is NO process solution that works on its own. Its all about the people management side of it.

Some of the points of agile are good (especially reacting well to changes as they come in).. but on balance the human factors are the ones that bite us, not the process ones. Take good managers and they'll converge on a good process for their products, and contain the natural chaos in their teams due to lack of maturity/experience. Take bad managers, and no matter what else you throw into that mix, you'll get chaos and crap. People people people. Get it right, your golden, get it wrong, your screwed.


weezl said...

@Ignatius:

you said:

"I would just like to caveat your warning about not using 'non technical managers' to manage software projects. A good non-technical manager will recgnise that where the expertise in the team lies and will trust them to make the right technical decisions. S/he will not use the agile framework to beat the team but will make sure the framework supports what they are trying to achieve.
Your concern appears to be more around the 'command and control' behaviours of certain managers (technical or non-technical) which is the real problem with agile implementation in the real world. Without addressing the cultural/behavioural shift required in an organisation, there will be a complete mismatch between management and development that cannot possibly be fixed."

WOW.. ok.. there is a large elephant in the room you are missing. That elephant is this:

A non-technical manager is very easily duped. Some people will actively lie to you to get their way. This is not theoretical with me, is based on direct observations of PRECISELY what I'm describing. Further, a non-technical manager will not have a bag of tricks to fall back on the LEAD their team in how to solve problems. Since software teams tend to be made of of many technically and/or emotionally immature individuals, the component of leadership simply cannot be overstated. In its absence, great chaos can happen, which affects not just delivery, but morale.

I simply can't state strongly enough, non-technical managers depend on folks that can be trusted. But they lack the technical judgement to know when they are being misled.. and can easily be led into situations that are untenable and require tons of remediation. I prefer more resilience and less wishful thinking in managers.

Maybe its just me ;)

ImmortalMan said...

Agree. But even Google has this bunch of non technical idiots and pays them charity while all they cause is negative influence in the teams. It is a strange world.

Anonymous said...

Agile is no silver bullet (cue shock, horror).

Any process can be misused by idiots, even no process, and process is not a solution when the problem is "idiots".


A good team, with the right people in all roles including the developer roles, can use agile to good effect and will produce better results than if they are forced to use something else like waterfall (the straw man process).

However, "a good team" means good people in *all* roles. Venting about the process is usually a sign of trouble. If you're genuinely being constructive about this where you work, and nobody is listening ... then why do you work there?

Unknown said...

I think this is a silly article that totally misses the mark. You can fuck up Agile and you can get Waterfall to work, and vis versa. It's not about the process. It's about the people. Do they understand why they practice any particular development methodology? Do they understand which ceremonies bring the values they require, and which ones don't? Do they understand the values and risks of said methodologies? If the answer is no, then the company has hired the wrong people, and/or failed to train them. Agile development is a means to end, not an end. The end is producing business value through software (or whatever you are applying your methodology to).

With the right people, you could create great products using waterfall, or no formal process at all, in the right situations. 'Agile' (referring to all the things we might think of as agile practices) is neither good, nor bad. It's a set of tools to be used to support the efforts of an organization as they attempt to create business value. You pick and choose tools that make that effort easier, more reliable, safer, more predictable, etc. Whatever you value in a process, you choose those tools that provide those values.

Agile, or any other set of tools fail because people don't understand what they want, what they need, and what tools will provide those wants and needs. To paraphrase the Agile Manifesto, People before process. It's the people stupid.

Shreyash said...

IMHO, Agile has not failed, its just evolving.

mochi said...

Title is a bit misleading, SCRUM is probably a failure, Agile is fine.

Don't think it's also fair to blame consultants for this, you can only blame the human nature, agile is more a different philosophy of doing things and most people are still not used to it.

I see SCRUM as just a mean to an end: Introduce some agility while not scaring people and traditional management away.

I agree with the main point of the post, if you have team of great developer and a good tech lead...you don't even need any kind of "process". Highly skilled and motivated people will always find their own "process". But the reality is different.

Anonymous said...

I have been working as a software engineer for over 7 years now and I can tell you that software development and software project management are completely different things. Software project management is not a method of leading software development but a series of methods for delivering software to customers. Software project management is not a technical discipline and that's why non-technical project managers can exist.

Software development simply means taking a list of requirements and building a software that will fulfill them. Software development can exist without software project management and those doing software project management don't need to be able to do software development.

That being said the methodology used by software project management has little to no impact on the actual software development process and as such I as a developer don't care how the project is being managed as long as I receive requirements that I need to implement in code.

And another thing non-technical project managers don't actually manage the developers in the sense that they don't give them instructions on how to do their work. The influence that the project managers have on the actual development process is minimal to non-existing. In all my projects the project manager was the guy that came to us with the requirements from the client and asked us to give estimates. After a while he was coming back saying that the customer has approved that, that and that and we can start working. During development the project manager basically only tracked the progress and nothing more. If things didn't go as planed he was powerless to do anything about it other than report the issues and change the plan so that it will be in touch with the reality.

Developers are really led by senior developers and software architects and not by non-technical project managers. Even when I worked in Agile environments there was always a team member who was the de facto leader and all the developers were going to him for advice. In one of my assignments the de facto leader was actually a software architect.

So don't mind the project management methodology or the non technical project managers as they don't influence developers in their work, software development is the same no matter how the software project is being managed. And when talking about project and product managers always state the full title as these people are not real managers but rather links between the customer wishes and the development team.

kbm said...

geometric relationship between the length of an estimate and its inaccuracy.

what is "geometric relationship" in this context?
are we drawing geometric shapes or something?

or is it geometric progression? or what?

Anonymous said...

Agile seems dumb to me. As I developer I constantly am in the position of humouring semi-technical scrum masters and ceremony masters. Smiling and nodding like my job depends on it, meanwhile knowing that the real work will get done when I leave the open-air noisy office full of foosball playing amateurs and get home to catch up often into the wee hours of the night. Then if I'm tired in the morning I'm blamed for not "engaging."

To me I think agile is more about companies trying to emulate Google or something and consequently is more of a marketing and/or feel-good thing than actually putting out working software -- because I know the software being put out often sorely lacks in quality and often doesn't hold a candle to the kind software written about by greats like Martin Fowler and Robert Martin. So it's not about software any more really, it's about appearances. After all most of the software is actually being written by freelancers or in overseas sweat shops. If the corporate/government shops in the West put out any working software it's probably at a 10x cost and thus is largely a token gesture perhaps to prove something, or out of necessity due to citizenship requirements etc., meanwhile agile practitioners have a strangle hold on the onsite shop -- any opposition to the dubious practice is stamped out.
Seriously I wouldn't mind the idiotic rituals if they weren't so humuliating and resonent of some mixture of a religious cult and a communist platform. Please, just give me a deadline some requirements and leave me in peace. I think only insecure newbies need "daily standups" to attain some kind of positive re-enforcement to compensate for their lack of software skills/interest.

Unknown said...

Well... I think failure's often a matter of not understanding things you use. Being aware of the fact Agile CAN FAIL in some situations is basic, for example Kanban http://kanbantool.com/kanban-library/kanban-results/when-kanban-fails . Relying on a method without beinng aware of its limitations is somehow not wise.

Jesse said...

I literally thought agile meant "stand-ups, retrospectives, two-week iterations and planning poker", LOL. The analogy to a cargo cult seems perfectly apt. I've never understood how in the world management agile could have ever come into existence - much less remain so long.

Marakai said...

It is the year 2020 and every so often I still send this bookmarked link to threads (too often LinkedIn) where a bunch of PMs with the coding knowledge of a rutabaga bemoan the state of "Agile Project Management". Often with "we're going back to waterfall because agile doesn't work".

Nothing has changed - and likely never will.