The Black Liszt

If You Care About Good Software You Should Care About Documentation

I recently posted what I thought would be my least-read post ever. It's about the subject that has by far the greatest spread by far between "something tech people say is important" and "something tech people avoid thinking about or doing."

There are many reasons why this is the case. High on the list is the fact that documentation occupies "below zero" status on the list of things programmers and their managers actually care about. I'm also guilty of this. Look at this detailed post I wrote about the hierarchy of status in programming and you'll see that documentation is nowhere mentioned!

Here's a metaphor to help you understand the importance of documentation. Suppose an important election is on the horizon and for various reasons an unprecedented number of people are likely to vote by mail instead on in person. Suppose that there's a large region in which USPS mailboxes have been installed in scattered places. Suppose that a security failure has been discovered in the design of the mailboxes that makes it quick and easy for a trouble-maker to open it and remove all the mail, including of course any ballots. Word of this vulnerability seems likely to leak out, tempting activists to raid mailboxes where ballots  cast for the party they want to lose are likely to be found. It's a bug! It has to be fixed immediately!

In the real world, there are probably lists of the locations of all such public mailboxes. Even if there aren't, there are USPS employees who visit the boxes regularly and know where they are. Failing everything else, the public can be asked to register the location of any boxes they know about. No problem.

In the wonderful world of software, things are entirely different. The mailboxes and everything around them are invisible to normal people. Members of the public don't "go there." USPS employees don't go there. The original contractor who installed each box at various times may have been required to provide documentation of his work, but the original documentation was incomplete and full of errors, and was never updated for subsequent additions and changes. What's worse, the boxes are darned hard to find. Practically no one has the right kind of eyes and training to even be able to correctly recognize a box when driving slowly down a road looking for them.

If you're a modern, with-it programmer you might be thinking to yourself at this point "hah! That wouldn't happen with my code -- I use modern micro-services, so I'd just have to go to the right service." Uh huh. What if the bug had to do with some data that was defined incorrectly? Do you have a DBMS? Do you have a UI? Does every piece of data really appear and get used exactly once in exactly once place? Is it really so easy to not only find each instance of the data but all the downstream uses and consequences of the data that are impacted by the error?

Regardless of how your code is organized, finding and fixing a bug can take a depressing amount of time, and part of the time is often the result of not having comprehensive, accurate low-level documentation.

In the relatively simple and visible-to-everyone real world of paper ballots and mailboxes, we know there are errors and faults, some of them extensive. Bad things can and do still happen. Because of the simplicity and visibility of the normal world the faults are often noticed quickly, like when bad ballots are sent. In the incredibly complex and visible-to-few world of software, the faults can go for long periods without even being noticed, and when they finally surface it can take the tiny number of super-specialists with the right training, vision and persistence a long time to find and fix the bugs lurking in the vast spaces of the largely invisible, undocumented oceans of software we all depend on.

Documentation is unlikely to improve in the world of software because practically no one, despite the sounds that may come from their mouths, really cares. The good news is that better approaches to building software than are fashionable today go a long way to minimizing the trouble. The more we move towards Occamality, the more we'll get there. The reason is simple: if everything in a program is in exactly one place instead of scattered redundantly all over the place, at least you'll know that there's just one fierce monster bug out there and not an army of clones.

The Universally Ignored Disaster of Software Documentation

Nearly everyone knows what “click bait” is – an article title that strongly tempts readers to click and read the article. Part of being a good writer/editor these days is developing a facility for click bait.

My proposal for the most click unbaitable or click repulsion title is anything that includes the words “software documentation.” Is there a more boring subject on the planet? Even to software people? Or I should say especially to software people?

It’s too bad, because software documentation is a genuinely important subject. Every ounce of effort put into it is not just an ounce of wasted effort, but many ounces of waste by side effect. Software documentation is the subject of near-universal complaints, but few dispute its irreplaceable importance in the cycle of “responsible” software development, from requirements all the way through testing and support. We should be talking about software documentation – and documenting (heh…) its odious nature.

Software Documentation

Documentation is the backbone – or ball-and-chain – of software from start to finish. The time, effort and resulting bulk of documentation normally vastly exceeds that of the software itself. The documentation is supposed to define both generally and precisely WHAT is to be built, HOW it is to be built and tested, and then what was actually built.

If you are in a normal corporate environment, this all ends up being HUGE. If you’re in a regulated environment, it’s so big it’s amazing anything gets done. Here’s a little snippet of just some of the documentation required for medical devices by the FDA.

As you can see, you even need a document that has the plan for documents! See this for more detail and explanation.

Documenting the problems with documentation is an immense job, and given that no one cares about documentation one way or the other, I’m not going to do it. But here are a few of the lowlights.

• Requirements documentation is supposed to be the foundation of a software project, defining in MBA-understandable terms the what, why and how of the software effort to be undertaken.
• In reality, most requirements documents fall somewhere in the range between useless and ignored.
• Requirements documents are kind of like the plan to discover a new sea route to Asia in the Europe of 1490, which was basically using a lot of words to say “sail west.”
• Requirements documents are never updated to reflect what actually happened, beyond brief memos saying triumphant things like “we sailed the ocean blue and hit land in 1492!”
• Architecture documents are typically produced by people who won’t end up writing the code.
• Architecture documents resemble someone on a roof top in Kathmandu pointing mostly east and a bit north, waving and jumping around, saying excitedly with lots of words – go there and keep climbing up and up and up, and you’ll get to the peak of Mount Everest!
• Often the climb-Everest plan contains huge amounts of gruesome detail -- all written by someone who's never been to base camp, much less climbed.  
  
• The people who write the code may or may not pay lip service to the document. They certainly won’t waste time updating it with what actually happened.
• More detailed design documents may be written by people who sit in the same general area as the people who write the code.
• In the end you have to make the code sort of work, and there are time pressures. Never has a manager ever said “and I want the updated version of the design document BEFORE you release the working code.”
• Everyone cares about quality, of course. So there are test documents of all kinds, even more in a test-driven-development environment.
• Is automated testing ever actually comprehensive in the real world? Has any manager insisted that updated test plans be created before the automated test is revised? I mean, of course, a manager who keeps his job?
• Code itself needs to be documented. How are programmers who need to dive in and fix things and/or make changes supposed to figure things out without it?
• Every programmer complains about the lack of relevant, accurate documentation when they dive in to a new code base to get something done. That same complaining programmer, of course, takes extra time at the end to carefully document what he’s done and the things he learned along the way. Sure.
• One of the reasons why having code in multiple layers and scattered among loads of little microservices is so attractive to modern-thinking people who think of themselves as Computer Scientists (or at least Engineers) is that when you’re looking for something about the handling of the data you’re trying to change, the code (and related documentation – hah!) is in lots of different places with no good way to figure out what’s where – or even what language it’s in! I bet you never knew the perversity behind this modern thinking, did you?
  

Responses to the Documentation Disaster

It’s been a long time since I’ve seen a new response to the ongoing disaster that is documentation. Here are some of the typical responses.

• Real programmers like to complain about documentation. Regularly.
• They complain when documentation they think should be available to them doesn’t exist.
• When it exists, they complain about how much time they wasted in a failed attempt to get anything useful from it.
• When someone tells them they have to write documentation, they have a list of compelling reasons why higher priorities won’t let them.
• When they lose and have to write, they complain about what a waste of time it is, and how useless their work will be after the next round of major changes to the software.
• Ambitious programmers learn to promote documentation
• Most important promotion decisions are made by people who are completely clueless about software
• One of the traits most highly valued by these ignorant managers is the ability to “communicate.”
• Communication with the clueless normally means having reports, project plans and other documents with enough plain language to seem relevant with lots of acronyms to convey credibility.
• Ambitious programmers emphasize adherence to standards to convey the appearance of being risk-averse; standards tend to include piles of documentation
• Since it’s all a charade in the end, ambitious programmers learn to take short cuts and make it clear to underlings that document content quality, accuracy and completeness are weights that can be shed in the race to completion.
• Has any top manager ever done more than glance at the thick pile of project documentation, assured themselves that it was appropriately thick, and nodded "good?"
• Managers nearly always have documentation on every project plan – otherwise they’d look like they didn’t know what they were doing.
• Experienced managers know it’s a sham, and that it’s documentation that will be squeezed out when time is running short.
• Managers working on projects that fall under government regulation know they might have to pass audits, and so make sure to produce voluminous documents with all the right titles, headings and sections.
• Since auditors don’t actually know anything about software, all they do is make sure the documents look like the sort that normally passes, and grade accordingly.

In the end, real programmers know that there is no such thing as good documentation. The only thing that matters is the code. Of course, “good” code is supposed to have comments embedded in it. Some programmers do this. But the second the code is changed...

Conclusion

 All this craziness stems from the fact that software is invisible. When things are visible, we're not documentation-crazed; for example, If you're having a house built, when the project was underway would you ask to see the updated plans or would you visit the job site? Along with project management and other standard aspects of software development, documentation is one of the reasons why software takes 10 to 100 times more effort to write than it could take. There is a simple solution. For better or worse, the solution is never taught, but is re-discovered on a regular basis by small groups who write great software quickly and well. Typically the groups who discover the solution have little time and less money, but are motivated and desperate to build a good solution quickly. So they avoid project management and don’t write more than a couple pages of what-are-we-doing type documentation at the start – and don’t write anything else. Really! NO documentation!

This makes perfect sense.  If you were running away from a crazed killer, would you stop to make sure your tie was properly tied?

For more constructive suggestions about how to build software better, see my books, particularly Software Business and Wartime Software.