Self-organization in agile teams

For years, I was arguing against the believe that agile teams are actually self-organizing (like a flock of birds), because every practitioner knows that there is a lot of active organization and planing needed for a software project to be successful – especially in agile environments. If the management doesn’t care, the team will barely feel the need to organize anything. From the business perspective you want to have clear responsibilities as well and not a situation where “everybody and no one” is responsible.

Speakers at various conferences were often talking about agile development as if it would solve all problems of software project management, if you just follow the rules correctly. This is wishful thinking of people who life in a world with yellow brick roads. It’s not like average coders gain advanced social and technical skills overnight when they start working in an agile team.

An experienced agile software development team is a highly social group that is self-organising around these principles and acts with coordination and collective behaviour. This collective behaviour comprises: Swarm intelligence which gives a team the ability to adapt to changes, and robustness which enables them to still perform and deliver even when one or more members fail.

Simon Baker, Rebellious CEO and extreme leader, Energized Work

No one orders a team or an individual to do specific work. The team members volunteer for work that they see needs doing, even if it is not something that is in their area of expertise. An agile team is constantly promoting learning in its people.

Mishkin Berteig, Berteig Consulting

If you rely on freelancers, trainees or junior developers, it’s almost impossible to build a team in which everyone feels responsible and is capable of doing the work of others. For companies other than Apple, Google or Facebook, it’s a challenge to find highly skilled people that are excellent team players at the same time. Some developers just want to work their hours and get money for that. That’s fair enough. Not everyone wants to read “Clean Code” and be a responsible engineer. You can just encourage them and hope they will follow your example.

Of course you should do a daily scrum meeting and use state-of-the-art technology like git, unit tests and continuous integration – but all these activities and tools can as well be counterproductive, if the scrum master has a laissez-faire attitude and nobody uses commits/branches correctly, writes good tests or is interested in the Jenkins output. This is commonly known as cargo cult programming and not self-organization. There’s really nothing that happens or gets organized by itself.

Indeed, the woes of Software Engineering are not due to lack of tools, or proper management, but largely due to lack of sufficient technical competence.

Niklaus Wirth

Self-organizing means more, not less organization

The term “self-organizing” started to make sense to me, after I thought about it in the context of development teams in large corporations (and that’s where lots of best practices traditionally come from): These teams often don’t have any type of strict internal organization and/or they are micro-managed by the product owner, who is not interested too much in the internals of the development process. The managers ask too much because of “business reasons” and the developers deliver too little because of “technical reasons”. Nobody can be honest to each other, because there are no detailed requirements, no realistic estimations and the overall productivity is quite low.

If you then start to apply certain rules and measurements to this system, the process actually becomes much more agile, because you know where you stand, what the team can do in a given time and ideally the product owner stops to constantly interfere with the developers. Engineers who work together closely, are more motivated, can give self-confident feedback to their managers and act actively. It’s the opposite of being passive aggressive.

In today’s environment, the most common methodology is code and fix. Applying more discipline than chaos will almost certainly help, and the agile approach has the advantage that it is much less of a step than using a heavyweight method.

Martin Fowler, The New Methodology

Seen from this perspective, it’s nothing less than the emancipation of the engineers – but still you need leadership to keep the project on course by intelligently enforcing rules that work and teaching others how to test and build reliable software. If nobody in the team is able to take true responsibility or has the experience required, cargo cult programming is performed. In that case, a traditional process with a person from the outside managing the team can lead to equal or better results.

The core challenge with understanding the term “self-organizing team” and the mental picture it creates is, that it’s completely open, how the team can and should be organized in practice. There’s no guidance. It’s a term, similar to emergence, that admittedly sounds nice, but probably causes confusion since it was first written down. Hierarchies and traditional management might be old-fashioned, but they provide a minimum of guidance. Engineers are a heterogeneous group of individuals, all with their personal strengths and weaknesses, unlike a flock of birds or a swarm of insects.

Programmers tend to be arrogant, self-absorbed introverts.We didn’t go into this business because we like people. Most of us got into programming because we prefer to deeply focus on sterile minutia, juggle lots of concepts simultaneously, and in general prove to ourselves that we have brains the size of a planet, all while not having to interact with the messy complexities of other people.

Robert C. Martin, The Clean Coder

For large projects, I’m a clear advocate for identifying an experienced lead developer who knows all requirements and sees the bigger picture, while the others can focus on the details, which are more fun to work on. In the natural sciences, self-organization refers to patterns that form automatically. So you might come to the (wrong) conclusion, that agile teams are characterized by the absence of leadership and personal responsibility. This is a deep trap you don’t want to fall into. A camel is a horse designed by a committee.

If I had the choice, I would prefer a more traditional term like self-managed teams. That might lower expectations and doesn’t sugarcoat the amount of rigor that’s involved. In today’s fast paced world, it’s a very rare and expensive luxury to have enough time to form a team of experienced engineers, that actually might be able to work in a way that comes close to self-organization. Who wouldn’t love to work in a team like that?

Self-organizing teams are not teams gone mad. Like all teams, they need a compelling goal, skills, information, and enough time to form and perform. And they still need managers to create a supportive context, set appropriate boundaries and constraints and connect the team to the organization.

Esther Derby, Misconceptions about self-organizing teams