What can you possibly mean by "post-hoc design?" Yes, I know it means "after-the-fact design," using normal English. It's nonsense! First you design something. Then you build it. Period.
Got your attention, have I? I agree that "post-hoc design" sounds like nonsense. I never heard of it or considered it for decades. But then I did. Before long I saw that great programmers used it to create effective high-quality, loved-by-customers software very quickly.
The usual way to build software: design then build
The way to build good software is obviously to think about it first. Who does anything important without having a plan? Start by getting requirements from the best possible source, as detailed as possible. Then consider scale and volume. Then start with architecture and drill down to design.
When experienced people do architecture and design, they know that requirements often "evolve." So it's important to generalize the design anticipating the changes and likely future requirements. Then you make plans and can start building. Test and verify as you drive towards alpha then beta testing. You know the drill. Anything but this general approach is pure amateur-hour.
I did this over and over. Things kept screwing up. The main issue was requirements "evolution," which is something I knew would happen! Some of the changes seemed like they were from left field, and meant that part of my generalized architecture not only failed to anticipate them, but actually made it harder to meet them! Things that I anticipated might happen which I wove into the design never happened. Not only had I wasted the time designing and building, the weren't-needed parts of the design often made it hard for me to build the new things that came along that I had failed to anticipate.
I assumed that the problem was that I didn't spend enough time doing the architecture and design thinking, and I hadn't been smart enough about it. Next time I would work harder and smarter and things would go more smoothly. Never happened. How about requirements? Same thing. The people defining the requirements did the best they could, but were also surprised when things came along, and embarrassed when things they were sure would be important weren't.
After a long time -- decades! -- I finally figured out that the problem was in principle unsolvable. You can't plan for the future in software. Because you can't perfectly predict the future! What you are sure will happen doesn't, and what you never thought about happens.Time spent on anything but doing and learning as you go along is wasted time.
The winning way to build software: Build then Design
Build first. Then and only then do the design for the software you've already built. Sounds totally stupid. That's part of why I throw in some Latin to make it sound exotic: "Post-hoc design," i.e., after-the-fact design.
When you design before you build, you can't possibly know what you're doing. You spend a bunch of time doing things that turn out to be wrong, and making the build harder and longer than it needs to be. When you build in small increments with customer/user input and feedback at each step, keeping the code as simple as possible, you keep everything short and direct. You might even build a whole solution for a customer this way -- purposely NOT thinking about what other customers might need, but driving with lots of hard-coding to exactly what THIS customer needs. Result: the customer watches their solution grow, each step (hopefully) doing something useful, guides it as needed, and gets exactly what they need in the shortest possible time. What's bad about a happy customer?
Of course, if you've got the typical crew of Design-first-then-build programmers, they're going to complain about the demeaning, unprofessional approach they're being forced to take. They might cram in O-O classes and inheritance as a sop to their pride; if they do, they should be caught and chastised! They will grumble about the enormous mountain of "technical debt" being created. Shut up and code! Exactly and only what's needed to make this customer happy!
When the code is shown to another customer, they might love some things, not need some other things and point out some crucial things they need aren't there. Response: the nearly-mutinous programmers grab a copy of the code and start hacking at it, neutering what isn't needed, changing here and adding there. They are NOT permitted to "enhance" the original code, but hack a copy of it to meet the new customer's need. At this point, some of the programmers might discover that they like the feeling of making a customer happy more quickly than ever before.
After doing this a couple times, exactly when is a matter of judgement, it will be time to do the "design" on the software that's already been built. Cynics might call this "paying off tech debt," except it's not. You change the code so that it exactly and only meets the requirements of the design you would have made to build these and only these bodies of code. You take the several separate bodies of code (remember, you did evil copy-and-modify) and create from them a single body of code that can do what any of the versions can do.
When you do this, it's essential that you NOT anticipate future variations -- which will lead to the usual problems of design-first. The pattern for accomplishing this is the elimination of redundancy, i.e., Occamality. When you see copy/modify versions of code, you replace them with a single body of code with the variations handled in the simplest way possible -- for example, putting the variations into a metadata table.
This isn't something that's done just once. You throw in a post-hoc design cycle whenever it makes sense, usually when you have an unwieldy number of similar copies.
As time goes on, an ever-growing fraction of a new user's needs can be met by simple parameter and table settings of the main code line, and an ever-shrinking fraction met by new code.
Post-Hoc Design
Ignoring the pretentious name, post-hoc design is the simplest and most efficient way to build software that makes customers happy while minimizing the overall programming effort. The difference is a great reduction in wasted time designing and building, and in the time to customer satisfaction. Instead of a long requirements gathering and up-front design trying valiantly to get it right for once, resulting in lots of useless code that makes it harder to build what it turns out is actually needed, you hard-code direct to working solutions, and then periodically perform code unification whose purpose is to further shorten the time to satisfaction of new customers. To the extent that a "design" is a structure for code that enables a single body of code to be easily configured to meet diverse needs, doing the design post-hoc assures zero waste and error.
What is the purpose of architecture and design anyway? It is to create a single body of code (with associated parameters and control tables) that meets the needs of many customers with zero changes to the code itself. The usual method is outside-in, gaze into the future. Post-hoc design is inside-out, study what you built to make a few customers happy, and reduce the number of source code copies to zero while reducing the lines of code to a minimum. The goal is post-hoc design is to minimize the time and effort to satisfy the next customer, and that's achieved by making the code Occamal, i.e., eliminating redundancies of all kinds. After all, what makes code hard to change? Finding all the places where something is defined. If everything is defined in exactly one place, once you've found it, change is easy.
Post-hoc design is a process that should continue through the whole life of a body of code. It prioritizes satisfaction of the customer in front of your face. It breaks the usual model of do one thing to build code and another to modify it. In the early days of what would normally be called a code "build," the code works, but only does a subset of what it is likely to end up doing. When customers see subsets of this kind, it's amazing how it impacts their view of their requirements! "I love that. I could start using it today if only this and that were added!" It's called "grow the baby," an amazing way to achieve both speed and quality.
New name for an old idea
All I'm doing with "Post-hoc design" is putting a name and some system around a practice that, while scorned by academia and banned by professional managers, has a long history of producing best-in class results. I'm far from the first person who has noticed the key elements of post-hoc design.
Linus Torvalds (key author of Linux, the world's leading operating system) is clearly down on the whole idea of up-front design:
Don’t ever make the mistake [of thinking] that you can design something better than what you get from ruthless massively parallel trial-and-error with a feedback cycle. That’s giving your intelligence much too much credit.
Gall's Law is a clear statement of the incremental approach:
A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.
The great computer scientist Donald Knuth, author of the multi-volume Art of Computer Programming, was a master of shifting between assembler language programming and abstract algorithms and back, the key activities of the speed-to-solution and post-hoc abstraction phases of the method I've described here.
People who discover the power and beauty of high-level, abstract ideas often make the mistake of believing that concrete ideas at lower levels are worthless and might as well be forgotten. On the contrary, the best computer scientists are thoroughly grounded in basic concepts of how computers actually work. The essence of computer science is an ability to understand many levels of abstraction simultaneously.
Thanks to Daniel Lemire for alerting me to these quotes.
Conclusion
Post-hoc design is based on the idea that software is only "built" once, and after that always changed. So why not apply the optimal process of changing software from day one? And then alternate between as-fast-as-possible driving to the next milestone, with periodic clean-ups to make fast driving to the next goal optimal? Post-hoc design is a cornerstone of the process of creating happy customers and optimal code. It also happens to conform to the goals of software architecture. Post-hoc design is like first fighting a battle, and then, once the battle is over and you've won, cleaning and repairing everything, incorporating what you learned from the battle just past so that everything is ready for the next battle. Post-hoc design is the way to win.
Comments