Extreme Programming for Milennialls and Gen Z

You can read this post in Spanish here.

Building software is hard. And the more I learn about the history of our industry the more I realise it has always been. In 1968 at the first NATO Software Engineering Conference, some atendees coined the term Software Crisis to refer to the difficulty of writing useful and efficient computer programs in the required time. In 1972 Edsger Dijkstra mentiones it in his ACM Turing Award lecture.

The major cause of the software crisis is that the machines have become several orders of magnitude more powerful! To put it quite bluntly: as long as there were no machines, programming was no problem at all; when we had a few weak computers, programming became a mild problem, and now we have gigantic computers, programming has become an equally gigantic problem. —  Edsger Dijkstra

People stopped talking about the Software Crisis at the end of the 1980s because it is psychologically extremely difficult to remain in crisis mode for more than two decades. But the Software Crisis never really ended.

Share this post on 🐦 twitter or subscribe to my newsletter With a grain of salt and receive an email with updates from time to time.

And suddenly it was the 90s. And the Internet was booming with digital products that required a shorter time to market. Which implied shorter life cycles and requirements that changed constantly.

Disaster

Most software projects ended up in disaster. And most still do today. Because nothing has really changed. I’m sure you’re familiar with some disastrous situations:

But why do software projects fail? No, seriusly. Why? I know that you’ve been in some of these situations. But how could you possibly avoid them? Back in the 90s Kent Beck asked himself the same question. And after years of experimentation he published Extreme Programming explained.

The mother of Agile

Extreme Programming is the mother of what some people today call “Agile methodologies”. Actually that is not entirely true because other similar initiatives emerged in the 90s: Rapid Application Development, Feature Driven Development, Pragmatic Programming or Adaptive Software Developmet. But these are not methodologies. Not really. Do you know why when someone talks about implementing agile methodologies, everything sounds so empty and stale? Because Agile methodologies can’t be implemented. They do not exist. There is nothing to implement. Agile is not about methodologies. It’s about culture. Some Engineers realised this in the 90s because their life was full of methodologies. And they despised them. Extreme Programming appeared as a cultural change. A different way of creating software reducing risk and avoiding waste.

Extreme Programming proposes adopting certain values, some principles and some good practices to plan, develop, test, communicate and design. None of these values, principles or practices are written in stone. Although most of them have stood the test of time. I’m going to skip the practices because you can find whole books about each of them. Take a look at them here. But I do want to stop and take a look at the values and principles.

Values

Principles

Not that simple

But why do software projects fail? No, seriusly. Why? What does Extreme Progamming actually solve that other methodologies don’t? Because, at first sight, it seems just a ligthweight methodology. Another set of rules.

We, humans, always want something to lead to something else. But the plan almost never goes as expected. We have a tendency towards causal determinism: each thing that happens is caused by something that happened before. And this is true but we usually ignore complexity: no event has just one cause. Our minds have an overdeveloped sense of cause and effect. That’s why causal determinism allows us to design, plan and estimate how long is going to take us to put code in production. But because our linear minds are so bad at understanding complexity, we are always headed towards disaster when we ignore it.

Engineering teams are complex social systems who create and maintain complex software systems. And our first approach to avoid chaos is to create a system of rules. A methodology. But a complex system can only be controlled by a more complex system. So the methodologies we create end up being more complex than the system itself. Because mathematically it makes sense. Bureaucracy and dark agile make sense when thinking about control. There is no other way of controlling a combination of complex systems.

If you take a look at the values and principles of Extreme Programming again, you can observe that all of them are there to help you navigate complexity instead of trying to tame it. This is not something only Kent Beck realised. The headline of The Nature of Software Development by Ron Jeffries is Keep It Simple, Make It Valuable, Build It Piece by Piece. Keep it simple because adding more complexity to an already complex system always makes it worse. Favoring simplicity has its trade offs but at a socio-technical system level the benefits always outweigh the disadvantages. Make it valuable is very tight to rapid feedback. How else would you know if what you are creating actually ads value? And build it piece by piece is another way of talking about incremental changes.

The Mythical Man Month

But this is not something Software Engineers only realised in the 1990s. Frederick Brooks tells the story of how IBM built OS/360 in the 1960. In his book The Mythical Man Month he talks about the lessons learned while building such a big software project.

Adding people to a late software project makes it later. - Frederick Brooks

Brooks realised that most Software Engineers are optimists. They assume everything will go just ok. The reason is we ignore complexity. No only in software but also in our social environment. We forget that creating something in a team is complex. The interactions between people can’t be ignored. And the team doesn’t have to be large for it to be complex. Large numbers and complexity are not tied together. Take for example the Three body problem.

It’s the people, dummy

Edsger Dijkstra thought the problems were computers. But powerful computers only allow us to build more powerful software. And powerful software needs to be built in teams. Building software, specially within a team, is more about the people than about the software itself. The solution and the processes have to take into account the complexity of the social structure. Most problems in Software Engineering arise because we, people, are complex and impredictable.

Extreme Programming is a good place to start. If you find it’s principles and values too simple is because they are. Don’t underestimate how complex can a situation get when you add just one new practice. And don’t try to tame it. Controlling complexity is unmanageable. Navigate it.

Share this post on 🐦 twitter or subscribe to my newsletter With a grain of salt and receive an email with updates from time to time.