The Black Liszt

Software Productivity: the ABP Factor

There is one central, screamingly obvious factor that impacts programmer productivity. It is unknown, ignored and/or undiscussed. But it matters more than most other factors. It's the ABP factor, "Anything But Programming." When a programmer isn't programming, that programmer isn't, well, writing code. I don't care what that programmer is doing! If it's ABP, that person is not -- repeat not -- writing code!

The ABP Factor

If your job is reading, any time you don't spending reading is time you're not doing your job. Call it anything you like: getting ready, preparing, taking a break, recovering, digesting, blah, blah, blah. Whatever you call it, if you're not reading, you're ... (get ready now) ... not reading! You're doing something else.

If your job is cleaning the house, any time you spend not cleaning the house is time you're spending not doing your job.

I hope you get the idea by now.

(Above: typical programmer/manager meeting in Beijing. Photo by me.)

Understanding the Status of ABP

Non-technical people, people who don't program, people who used to program but don't any more and people who still think they're programmers but have regressed to a lower form of life (like managers) often value what they can do (which is clearly ABP) highly. Makes sense. If you do it, it must be a good thing. If you can't, don't or won't do it, it must not be the kind of thing really valuable people like you do. This applies to programming in spades. ABP is highly valued. Programmers quickly get the idea that the way to increase your status is to spend increasing amounts of your time indulging in that valuable thing, ABP.

Is ABP Worth Something?

Putting my cynicism aside for the moment, the answer is a clear, resounding yes. A certain amount of planning, coordination and other stuff is necessary. Not doing it well leads to really bad things. It's even OK for some people to spend most of their time in such non-programming activities!

But let's make it even clearer. Think about manufacturing or customer service. Either you're directly contributing to the production of goods or services, or you're not. If you are, we can start talking about how effective and efficient you are. If you're not ... either your personal productivity is lower than it could be or, much worse, you're overhead!!

Conclusion

Are you writing code? Good. Then we can have a grounded discussion about your productivity -- at least you're trying; at least your shoulder is at the wheel and you're pushing. Are you doing ABP? You're probably self-important overhead; stop wasting my time and yours.

Maybe you really are doing something that makes productivity better in some mysterious way. I'm open to the possibility! But the burden is on you to prove it.

Personal note: I spent most of every day writing code for a couple decades. I know all about programming overhead. I was acutely conscious of my personal overhead, and contributing to other peoples'. Now I don't write code. I feel guilty wasting the time of programmers. The only way I can justify it is if, as a result of our interaction, their productivity goes up, so that the time spent not programming ended up being a net productivity increase. I always think: minimize the time, maximize the value.

Software Productivity Divisors: Doubling the Work

Software productivity is incredibly important, but hard to measure and hard to achieve. Nonetheless, good people seek ways to make their software productivity better. They seek out productivity multipliers.

All too often, widely hailed advances in method and technique not only fail as productivity multipliers, they are actually productivity divisors!

There is a whole family of software productivity divisors whose obvious impact is to double (or more!) the labor for any given body of code. They don't double the pleasure or double the fun -- they double the work.
And yet, these methods are taken seriously by all too many people in the field.

What are software productivity divisors?

The field of software is awash with a plethora of tools and techniques, each of which claims amazing benefits, usually boiling down to producing better code with less effort. They are, in effect, promoted as productivity multipliers.

Some of the techniques are simply useless. But a surprisingly large number of supposed software improvements actively make things worse. They aren't productivity multipliers -- they are productivity divisors.

Doubling the Work

One large class of fancy new software methods essentially involve doubling the work. There are two main methods for doing this.

One favorite method of work doubling is very simple. Instead of using one programmer to do a job, use two.

Another widespread method of work doubling is to write the code twice, in different forms. Usually, one form of the code is the code that you actually want to write. The second form of the code is normally a mirror of the original code, whose nominal purpose is to test the original code.

Each of these methods is completely brain-dead. They normally double the labor of everything that is done, while not improving the value of the results at all. Costs more ... later completion ... no greater value ... Hmmmmmmmm ... what's wrong with this picture??

Pair Programming, the Hot New Productivity Divisor

Programming is, by definition, a task that requires great mental concentration. Kind of the opposite of what you do in social situations. Two people working on the same task relate to each other socially. In fact they are supposed to interact. If one person codes and the other one sits there, you've doubled the labor with no gain at all. To the extent that the second person interrupts (as he/she is supposed to), you've made things even worse.

I know, I know, the whole idea is that the whole is greater than the sum of the parts. The second person catches mistakes the first person missed. The second person comes up with a much better way of doing something than the first person got part way done doing. Etc. It's all a bunch of Bologna. I'm relieved that silly parodies of this silly idea are beginning to appear.  

Unit Testing, the Classic Productivity Divisor

There are many variations of unit testing for code, from test-driven development on. People get into arguments about whether you should write the test before or after you write the code, whether the person who writes the code should also write the test, and endlessly on and on. They all amount to arguing which is worse: dirty stinking rotten, or slimey awful junk. Who cares?? It's all bad!!!

No one claims it's easy to write good code. Code that's accurate, fast, does the job, does only the job, has no side-effects and doesn't crash. Still, there may be bugs. The idea with test code is that someone, in some order, writes test code that assures that the original code is accurate, fast, does the job, does only the job, has no side effects and doesn't crash. If you leave out any one of these items (as is typically the case), you haven't tested everything.

The person who writes the test code has to understand the requirements just as thoroughly as the person who writes the base code. Just as the person who writes the base code can make a mistake, so can the person who writes the test. It is just as hard to write the test as it is to write the code being tested. So you've doubled the amount of work, at minimum.

Now think about making changes and debugging. Instead of just changing the code, you've got to change the test code, and either one can have bugs. The test code can fail to test something that is supposed to work, or declared that something works when in fact it does not, as a result of a bug in the test code. So changes are now twice as much work, with twice as many places for bugs, in addition to bugs of interaction.

Any variation of unit testing has the certain result of at least doubling the cost of building a piece of code, usually increasing the elapsed time, with vague promises that are never proven and never measured about the improved value of the results.

Conclusion

In many organizations, there is a dead-simple way to improve software productivity: STOP the madness of institutionalized software productivity divisors! You don't have to be best-in-class; you just have to avoid doing things that for sure double the work while doing some enthusiastic but empty arm-waving about the future benefits.

What is Software Productivity?

People in every field are concerned about productivity. They want to know how to increase their own productivity and the productivity of their group.

Software is a peculiar field in which history is ignored and fashion too often trumps objective fact. In software, we don't even think about productivity, much less have any idea how to measure it.

What is Productivity?

Productivity measurement is central to our economy as a whole, to industries as a whole, and to individual companies.

In general, the more productive you are, the more outputs you generate for a given set of inputs.

Productivity can be measured by various means, depending on the purpose of the measurement. At the level of the firm, the most basic measurement is comparing the cost of the inputs to the value of the outputs, adding in overhead and the cost of the capital and labor required to produce the outputs.

Increased productivity can be accomplished by greater automation, improved processes, reduced cost of inputs, labor or capital, and by more effective use of labor, labor that is higher quality, more discliplined, trained and motivated, etc.

This is basic economics. It is applied from the national level down to mom-and-pop firms.

What is Productivity in Software?

Measuring software productivity is incredibly important. But practically no one actually does it! It's not even studied in any meaningful way -- if it were, you'd find competing theories, and experiments to settle the issue. While there are a couple studies, their paucity and narrowness prove the case that software productivity is largely ignored, both in theory and in practice. GIven that software is increasingly the engine that drives our enterprises, this is amazing.

The situation with software productivity is a bit different than in your basic widget factory.

• The cost of the inputs is pretty much zero
• For all the talk of software factories, the cost of "machines" is generally insignificant
• While the quantity of output (lines of code, LOC) is tangible, it's not really correlated with value
• Except in a few cases of commercial software products, the value of the output can be hard to measure, and even then it can be dicey.
• The overwhelmingly important factor in cost tends to be labor!
  

Software Productivity: the best case

To understand software productivity, let's start with an extreme case, but one that does happen a small percentage of the time.

A single person understands the requirements of what is to be written. That individual, perhaps with the assistance of one or two other people, writes, tests and delivers the code. The code runs in production for a number of years, and the few bugs that arise and the enhancements that are needed are quickly supplied by the original author(s).

In a case like this, the value of the outputs can still be hard to measure. But at least the other costs are easy: it's basically the cost of the people, with overhead.

Is this a fantasy or does it happen in real life? Well, all I can say is that it happened in my life, more than once, and I know people who have done similar things. In my first job after college, I wrote a commercial FORTRAN 66 compiler and run-time system (in assembler language) for a computer company. I had an assistant for part of the run-time environment. It was in production use for about ten years. I fixed a couple minor bugs in the first year, and that was it. I personally did a couple similar projects in different software fields.

In the case of the compiler, the cost was easy to measure. The value was typically challenging, but not too hard. The company sold the compiler. But its greater value was enabling sales of the display computer on which it ran. Potential buyers were refusing to buy because there was no high level language available at the time; my compiler enabled many sales that otherwise would have been lost.

It's easy to imagine the software productivity in this simple case. You can pick your measure of value (pick a few -- why not?), including lines of code, number of users, value of the code, indirect value. You pick your measures of cost (again, pick a few) including person-days, elapsed time, salaries, etc. And finally, you relate them into simple, calculable numbers that you can track over time.

Software Productivity: the Reality

In practice, software productivity is an absolute bear to measure. Cost isn't too hard. But valuing the outputs? That's the big problem. The other problem is that software just isn't like a widget factory. The items ("pieces" of software) cranked out are simply not comparable to each other, as I've discussed.

Conclusion

Productivity is important in general, perhaps even more in software than other fields because it is so hard to measure. Software won't get better until we increase its productivity, and that won't happen until we are hard-nosed and objective about exactly what software productivity is. All the discussions in this blog about software quality, estimating and other things are best understood in the broader context of software productivity, the great unknown frontier of computer engineering.