While I was student at Harvard College, I worked at several jobs writing production software. I also audited a course at Harvard Business School. By wild coincidence, one of the jobs I had was with a small company that was the subject of a new case study used in the HBS course I took. What I learned from the contrast between the reality of the software company and the fantasy of the case study was the seed from which grew my current understanding of the application of standard management techniques to software. Bottom line: HBS-taught management techniques result in software that is expensive and bad.
How HBS case studies are written
Early in my four years at Harvard College I worked part time at a software company in Central Square in Cambridge called CPSI, Computer Programming and Systems, Inc. It was a programming services company, doing jobs for clients. I knew the languages they needed. I remember little about it or what I did there now; I remember not being impressed or excited by the work; with the campus chaos in the spring of 1969 I dropped it. Nonetheless the experience led to things that strongly shaped my career.
Not long after working at the Cambridge software company, I walked across the river to audit a course at the Harvard Business School, HBS.
The course was pretty new, called “Management of Small Enterprises: Operating Problems and Strategies.” It was a typical giant HBS course run on case studies taken from real life. The students competed to “nail the case,” by finding what HBS said was the best answer. The course was pioneering in concentrating on small business, since mostly HBS cranked out students to fill the big consulting companies and public companies. I attended to learn about starting and running a company.
Doing some research now, I found that the engaging professor, Arch R. Dooley, was a Yale and HBS grad. He never actually worked for a company, but nonetheless became an esteemed professor of manufacturing management, and started the programs for entrepreneurs, which have remained a small part of what HBS offers.
Part way through the course, which was interesting because of the type-A students fighting and clawing each other, a case study about a small software company became the focus. By the end of the case, I realized that the case was about CPSI, the company I had worked for during the prior semester. While a couple things had been changed to anonymize it, the company was easy for me to recognize. Even so, the case got important things about the company, its software and the market for software wrong. The HBS students were triumphant when they arrived at Prof Dooley’s approved approach to managing the company, which had absolutely nothing to do with what the software actually did. This blew me away.
After I graduated and stayed around Cambridge, along with a fair number of my classmates, I discovered that some of the super-bright, verbal people I knew were hired by HBS to … write case studies! They were given a bunch of existing material and case studies, with descriptions of generic “right answers” and thrown at a company. They would quickly figure out what they could and write up a cogent, compelling study. Maybe this worked for companies that did physical things they could easily understand, but none of them had the slightest exposure to software, and of course their managers were equally clueless. The result was the case study of the company I had worked for, which everyone involved thought was good material, but was in fact deeply flawed.
This HBS case study was my first exposure to the massive moat separating software reality and entrepreneurship from management and academia. Dooley was convinced that, although he’d never written a line of code and couldn’t even see it or read it, he could teach about software. He was equally convinced that, even though he’d never worked at a company, much less an entrepreneurial one, he could teach others how to do it. To this day, his colleagues think the same, and all the students and those who hire them are equally convinced. And much of it is based on case studies written by talented Harvard English majors fresh out of school.
This is stupidity, but it’s so common there has to be an explanation. I believe it’s one of the widespread manifestations of our class system, which nearly everyone implicitly understands but few people discuss. While the classes evolve, there’s always a clear hierarchy. One of the long-standing aspects of class is that people who do stuff in the real world generally are lower class than those who direct, observe or think. Attending HBS is clearly a way to enter the directing class. Engineers are highly technical, but usually other people build what they design. Programmers, by contrast, are people who use tools to build things, the way a craftsman might build a house. This puts them levels below the people who command what is to be built, just like the people who are served a meal in a grand hall are levels above the people who serve them and the cooks in the kitchen who make the food. All the grand professors at HBS and their students are far above the minions who write code in terms of class. Grand as they are, it’s clearly beneath them to stoop to dirty themselves with details of how things are built. They hire people for that. The fact that this attitude results in inefficiency, chaos and often disaster somehow isn’t their problem.
There are good reasons to teach management
The whole idea of a school teaching "management" is that there is knowledge about managing a group effort that is independent of the details of the effort and the products and services being created. This is a reasonable thought and generally valid. Think about finance, for example. Nearly every organization -- even non-profits -- have revenue and expenses, assets and liabilities. Expenses naturally fall into multiple categories, including payroll and capital. Different industries have different patterns of these things and the details matter, but the common elements are large and important.
Other things that are taught in business school have substantial elements that are similar across different industries. All industries have people who need to be hired, directed, and organized. Even software fits into the pattern. Suppose you're a construction company and want to build a bridge. The overall requirements of the bridge first need to be determined (number of lanes, etc.), followed by overall and detailed architectural and engineering design, and finally building. Just like software, right? It makes common sense and is widely accepted.
Why standard management fails for software
The widely accepted common sense notion that building software is pretty much like building a bridge now has a track record of many decades of failure.
Software development, security and management is an ongoing disaster, with all attempts to fix it failing. If bridges failed anywhere close to the rate of software failures, there would be revolution. See this. See this also.
One big reason is that software is different from all those things that management techniques work with. Food is cooked quickly, and you can taste the results. You can see buildings in progress and most other things. Software, by contrast, is invisible. How can you manage something you can't see?
Not being able to see it is bad, and not being able to do it yourself compounds the problem. At least with cooking and building you have a little idea of how it's done. In many fields, the best managers were good at doing what they're managing. This excellent principle is rarely followed in software.
Because of the invisibility, managers demand huge piles of documentation about the stuff they can't see. Makes things worse.
Everyone involved makes big efforts to track software as it's being built. It's often the case that glowing reports are made of software up until days prior to its being done. And then disaster ensues.
Because of its invisibility, managers tend to understand how software works in terms of metaphors -- metaphors that are inappropriate and make things worse. They want to create a software factory. They think of software as being like a building, with an architect and the rest. Even the notion of "building" software, as though it were a passive thing, is wrong. It's actually extremely rare that software is "built" -- mostly software is changed.
Because they're trained in business management, they think their methods apply to everything, including software. They use standard project management and variants created specifically for software like Agile to keep things under control. Without realizing the damage they cause, they insist on optimizing for predictability, while nimble startups beat them by optimizing for speed.The difference is like the difference between building bridges in peacetime or war.
Conclusion
Things have changed little since my bright college friends were churning out case studies to teach generations of future MBA's at HBS. Software is still invisible to the vast majority of managers who are nonetheless confident in the certified management methods they use, endorsed by all the Experts and high-paid people at the top of our class system. The methods that work to enable small teams to produce top-quality software quickly and well aren't secret; they are simply beneath the dignity of the people who dominate the upper ranks of all large organizations, including Big Tech.