Nearly everything we hear about the construction of software is that it is built. This is a fundamental misconception that pervades academia and practice. This misconception is a major underlying cause of why software has so many problems getting created in a timely way and functioning with acceptable speed and quality.
The software-is-built misconception is woven into every aspect of software education and practice. You are first taught how to write software programs; all the exercises start with a blank screen or sheet of paper. You’re taught how to test and run them. In software engineering you’re taught how to perform project management, from requirements to deployment. You may learn how to automate software quality testing. Everything about these processes assumes that software is built, like a house or a car.
The idea that software is built isn’t wrong – it just applies to a tiny fraction of the total effort that goes in to a body of software from birth to its final resting place in the great bit-bucket in the sky. The vast majority of the effort goes into changing software – making it do something different than the version you’re working on does. Maybe you’re fixing a bug, maybe you’re adding a feature or adapting to a new environment. Sometimes the changes are extensive, like when you’re changing to a new UI framework. For a typical body of software, however much effort is put into the first production version, a minimum of 10X and often over 100X of the work goes into changing it.
We don't think of software as ever-changing, ever-growing partly because we think in terms of physical metaphors. We use the same project management techniques that are used for houses that we use for software. While sometimes houses are changed, mostly they're just built and then used for years. It's an extremely rare house that is endlessly extended the way software is. See this for more.
When it comes to enterprise software, the fraction of effort that goes into change sky rockets even more. A good example is the kind of software that runs hospitals and health systems. The software already exists, and it’s already had 100X the effort put into change than the original building. For example, standard software from the leading industry supplier was installed in the NYC hospital system. The original contract -- just to buy and install existing production software -- was for over $300M and 4 years! A couple years later the carefully planned project cost more than doubled. All because of changes. Here are details.
Why does thinking software is built rather than changed matter?
The idea that software is about building and not changing is responsible for most of the processes, procedures and methods that are overwhelmingly common practice either by custom or by regulation. The mismatch between the practices and the fact that software is nearly always changed instead of being created in the first place results in the never-ending nightmare of software disasters that plague us. The never-ending flow of new tools, languages and methods are futile attempts to solve these problems. No one would find the new things attractive if there weren’t a problem to fixed – a problem everyone knows and that no one wants to explicitly discuss!
Example: Project Management, Architecture, Design and Coding
Mostly what happens when you get a job in software is that you join a group that works with a substantial existing body of code. You may be a project manager, a designer, a customizer, any number of things, all of which revolve around the body of code.
Yes, you have to start any project from an understanding of what you want to accomplish. Here's the thing: what you want to get done is in the context of the existing code. What this means is that you have to understand as much of the code as possible to do things. If what you want to do is fairly circumscribed in the existing code, you may get away with ignoring the rest -- which is where comprehensive, nothing-left-out regression testing comes in, which means better make darn sure you didn't break anything.
In this context, lots of skills are relevant. But the overwhelming number one thing is ... knowledge of the ins and outs of the existing code base!
I've worked in a couple places where there were large code bases of this kind. Forget about departmental hierarchy -- the go-to person in every case of doing anything important was the person who best knew that particular aspect or section of the code.
In this context, what's the point of all the fancy project management stuff people think is the cornerstone of effective, predictable work? When you're traveling through an unknown land with lots of mountains, jungles, rivers and swamps, what do you want above all else (besides nice paved roads that don't exist)? You want an experienced GUIDE, someone who's been all over the place and knows the best way to get there from here.
Kind of turns all the project management stuff on its head, doesn't it?
Example: Software QA
Most software QA revolves around writing test scripts of one kind or another that mirror the new code you write: if the code does X, the test should assure that the code does X correctly. This is true whether you're doing test-driven development or whether QA people write scripts. As you build the code, you're supposed to build the test code. When you make a new release, you run integration tests that include running all the individual test scripts.
Everyone with experience knows the problems: there are never enough tests, they don't cover all the conditions, there are omissions and errors in the tests, with the result that new releases put into production often have problems.
The root cause of this is the never-spoken assumption that software is about building, when the reality is that it's mostly changed. What this means is that the focus should be on the existing code and what it does -- do the changes that you made break anything that used to work? When you make changes, you should be thinking champion/challenger like in data science: the new code is the challenger: did it make things better, and did it (probably unintentionally) make anything worse? The best way to do this is to run the same data through the production program and the challenger program in parallel, and have a tool that compares the output. Looking at the output will tell you what you want to know. Even better, running the new code in live-parallel-production, with lots of real customer data going through both versions but only the current champion output going to the customer provides a safe way to see if the customers are OK, while testing out the new functionality on the champion code only to make sure it does what you wanted.
Who needs test scripts, TDD and the rest? They're backwards! See this for more.
Conclusion
Software is literally invisible to the vast majority of people. Therefore, we all talk and think about software in terms of metaphors. That would be OK if the metaphors matched reality; unfortunately, the metaphors don’t match reality, and play a major role in software dysfunction. In real life, we build houses, bridges, and other structures -- we mostly don't change them.
So what are the methods that are appropriate when you recognize that the main thing you do with software is change it instead of building it? I've done my best to abstract the methods I've seen the best people use, calling them (for want of a better term) war-time software. Instead of creating a whole adult human at once, like Frankenstein, it's like starting with a baby that can't do much yet, but is alive. The software effort is one of growing the baby instead of constructing a Frankenstein.
Comments