On this – all depends. Only a fully trained Jedi Knight, with the Force as his ally, will conquer Vader and his Emperor. If you end your training now – if you choose the quick and easy path as Vader did – you will become an agent of evil.

Many necessary basic skills are not entirely obvious while we are in the midst of our training.  This is true in any discipline – I was just lamenting to a long-time friend about how much I wish I had understood the necessity of practicing my trumpet when I was beginning – as it’s shocking how quickly one improves if one takes the time to practice.  She politely informed me that I *had* been told – *repeatedly* of the value of spending this time.  She was right, of course – but I can attest that it was not entirely obvious at first how valuable the skill of practice is in music.

The skills of course become even less obvious when there is a princess being held hostage by an evil overlord – or when the VP is camped out next to your desk asking, “are we there yet”.  Building incredible software is difficult – taking the easy path will result in us falling short.

A basic skill in software that engineers can miss out on is understanding and intentionally growing in their knowledge and use of Agile methods.  What is Agile?  I tend to abhor buzzwords – but in our industry we seem to generally understand that there is some kind of evil out there that needs to be dealt with and the ideas that tend to be associated with “Agile” have appeared to work to a certain extent.

To be more specific though – I would define Agile as the extent to which an engineer or engineering shop successfully balances the Four Tensions of Software.  The evil that we tend to see out there in the world occurs when we give up on one of these tensions and extremely common (though still unfortunate) structures and behaviors begin to take shape – all of which end in disrespect for people, low morale and less incredible software.  This is one reason I like to refer to Agile development more generally as “Humane” development.

The Four Tensions of Software

Here are the Four Tensions of Software, and the results of folding on either side of them:

1) The Tension of “The Whats and The Hows” – when making a decision about what to build, how you build it can inform but should not drive decisions, when making a decision about how to build a thing – what you are building should inform but not drive decisions.  Folding and letting The Hows drive The Whats means you will miss important business opportunities and/or build things that are totally unnecessary.  Folding and letting The Whats Drive The Hows results in lower-quality, less-considered code that will rot quickly and become increasingly difficult to maintain – this will ultimately end in missing business opportunities and spending big chunks of time cleaning up (which will mean yet more missed opportunities).

In both situations where we fail at maintaining a balance – micromanagement tends to occur.  This is because the in-depth skillsets involved in being a driver of The Whats or a driver of The Hows almost always necessitate that these are different people.  So, for example, if The Whats drive The Hows – the people behind The Whats tell the people behind The Hows task-specific work – which, by definition, is micromanagement.  And micromanagement communicates mistrust – and wastes a lot of precious time.

2) The Tension of “Delivery and Engineering” – when making a decision about order of delivery, the engineering can inform but should not drive the decisions.  Folding and letting engineering considerations drive delivery ordering means we are potentially delivering less important software first – this means potentially missed business opportunities.  Folding and letting delivery considerations drive engineering decisions means lower quality software that will rot quickly and become increasingly difficult to maintain – ultimately this means missing business opportunities and spending big chunks of time cleaning up (which will mean more missed opportunities).  And for the same reason as The Tension of The Whats and The Hows – folding on either side of this tension will result in micromanagement.

3) The Tension of “Progress and Adaptation” – a sense of progress – or momentum – is a gut level, intuitive understanding about the state of the collective accomplishment that an organization is pursuing.  This is incredibly important for everyone – but is especially important for folks tasked with the product’s vision – and for the organizational leadership.  If this sense is missing – even if progress is being made – it leads instinctively to a need to inspect – to aggressively find out what is happening – and to control.  This instinct is particularly strong in the kinds of individuals that drive product vision – or the kinds of people that provide organizational leadership.  Over-inspecting and over-controlling – is micromanagement.

Creating software is…well…creative.  When we build software – we never know what we’re going to have to do to make it happen – and further, our users don’t know what they want (fully) until they start to use something.  So – adaptation is *always* necessary – if an organization clamps down on adaptation in order to create a sense of progress for itself – well, all sorts of ridiculous things can happen – not the least of which is writing software that is in no way a match to the need it was meant to meet.  And if an organization considers itself flexible and allows for infinite adaptation with no need to show progress, well – the sense of progress evaporates – and folks will either start to feel hopeless about their ability to deliver, or as we went over earlier, they’ll start to be more aggresive about inspecting and controlling (that is, micromanaging).  Neither of which is a particularly good morale generator.

4) The Tension of “Focus and Engagement” –  Writing software is sort of difficult.  One of the primary reasons for this is the large collection of details that need to be kept in short term memory at any given moment.  For this reason, focus is rightly a precious commodity to developers.  Focus, however, is easily broken by simply changing the set of details that we feel we need to keep at hand – but worse than that is having to switch from creative-and-detailed mode to dealing-with-people mode.

Dealing with people, however, is an important part of creating incredible software.  People define what need we want to meet, people help to define how we’ll solve a problem, people ensure our work is up to snuff.  People, people, people.  I like to joke that software would be so much better if it weren’t for all the people.  This – though – is just ridiculous – because it is for people that we make software.

So let’s restate the problem – we need to focus while engaging with people, though engaging with people tends to destroy focus.  Easy enough to understand.

So – to fail on the side of focus, if we follow our instincts as engineers (at least they are my instincts) – we crawl into a hole with a couple of FogBugz or Jira tickets – and don’t emerge until our work is perfect and complete.  The problems this might create are obvious (and we’ve probably all tripped on them) – we have no idea if what we are creating is appropriate, because it might have changed 15 times since we started, we have no idea if it’s accurate (because we didn’t engage anyone to test), etc.

If we fail on the side of engagement and create a lot of formal structure “forcing us to communicate” – we have a bunch of meetings that destroy our focus and we don’t get any work done.

Failing to balance any of these “Four Tensions of Software” leads to bad things – mostly micromanagement and bad software…and if carried to extremes, failing to balance them can lead to death marches, mounds of spaghetti code that stack all the way to the ceiling and pervasive mistrust.  Maintaining balance with these Tensions is what Agile is all about.

The Solution

Luke learns to use the Force

…an agent of evil? Really? Are you sure?

Organizational dynamics happen in real-time.  You usually don’t get to stop and think for very long about how you’re balancing a particular tension.  You don’t get to deliberate precisely how to best manage balancing Focus and Engagement or The Whats and The Hows and then act based on that deliberation.  Many times it’s just a matter of seconds.

It’s like fighting someone or improvising a jazz solo – you don’t get a chance to plan out your next response to your opponent’s attack – or to the piano player’s last riff.

So the ideal solution here is to understand these tensions as deeply as possible and have a set of rehearsed responses that balance them in every possible set of circumstances.

Just like that fight or that jazz solo though – while these principles are simple on the surface but have an unfathomable number of permutations of circumstances that we have to be able to handle.

The *Real* Solution

This is really where software is a lot like other arts – and like other arts there is a pattern we can follow to pursue mastery…the pattern goes something like this:

1) Find people who know what they are doing – watch what they do.
2) Find out if there are specific techniques that you can just copy – to help you learn the underlying principles.
3) Use these specific techniques and anything else you can find and do real art.
4) Do whatever you can to learn as much as you can about the underlying principles.  Read, inquire, think.
5) Teach someone else.

This all applies directly to software….some advice about each of these steps:

1) Look for the best people you can find – as always, don’t assume because someone is a practitioner that they are a skilled practitioner.
2) I recommend the Scrum framework as a good set of training wheels here – there is a lot of literature out there about it – and it really hits the mark in a lot of good ways.
3) Even if you’re not perfect – just DO.
4) Find any information you can – internalize it – think about it – bounce it off of others.
5) Everyone wants to know how to make more incredible software – there should be no shortage of folks available for you to teach.  But make sure you have enough depth to really add value to whomever you are mentoring – wasting peoples’ time is a big no-no.

Delivery method is an oft overlooked basic engineering skill.  It’s also a thing that will take a lifetime to master – so it’s best to get started today.  Invest in yourself – find resources, spend the time working projects where you are able to learn more.  The more we do this – the more incredible software there will be!!

Here’s to making more incredible software!!