There is strong interest in the latest developments in software. No one wants to be left behind.
At the same time, there is a peculiar (to me) lack of interest in whatever the latest thing grew out of or evolved from. What are the new thing's predecessors? What did that thing grow out of. Have similar things appeared in the past? Are there patterns we can observe here, or are the new software things that explode onto the scene just a meaningless sequence of random events? Are they things that we obsess about while they're here without questioning or wondering where they came from, without asking what else they may resemble in some way? Are they things we just forget about and ignore once they depart the stage of software fads?
The questions have obvious answers. Answers that no one seems to be interested in.
So long as we fail to ask these crucial questions, we will remain ignorant of software in deep ways, and will continue to stumble from fad to fad without improvement or understanding. History is important!
Evolution in Science
If you study different sciences and their history, you notice that, at some point, scientists begin to pay attention to evolution, i.e., change over time of the thing being studied.
The most obvious thing is evolution of an individual item over time. In biology, you have the cycle from birth to death. In geology, you have the change of mountains, rivers, beaches and glaciers. In materials science you have the change of objects as they are subjected to various conditions over time, for example the way that iron rusts.
Evolutionary studies reach a breaking point and get really interesting when you study groups and types and see the patterns of change. In biology, this was the revolution that happened with Darwin and his study of the evolution of species. Something similar happened in geology with plate tectonics.
When you put the patterns together, things get totally amazing and transformative. While no longer fashionable to talk about and study, the parallels between the growth stages of an individual animal and the evolution of species are obvious, though not literal at a detail level.
Studying evolution is an important stage in the evolution of most sciences!
It's long-since past time to study software evolution, as an integral part of software science.
Software Evolution
Software evolves at every level.
At a basic level, software languages evolve. I have given an introduction to this subject here. The evolution of software languages resembles biological evolution in remarkable ways, as species (languages) emerge and evolve in response to new conditions, including other species. Similarly to biology, languages evolve; sometimes many species descend from them; and sometimes a species goes extinct. See this snapshot, for example:
(Credit)
Consistent with the determination of nearly everyone involved with software to ignore its history, this chart and anything like it are ignored, and the evolutionary principles behind it are not studied. When you start to study them, you learn amazing things. Suddenly some of the random change in the world of software languages starts to make sense.
Software languages are the very most basic things about software. Just as interesting and vastly more important are the programs written in software languages. Everyone involved in writing or maintaining a software program (application) just thinks about that program. It definitely occurs to business people what the program can do compared to its immediate competitors, if any. Yes, there is a kind of "survival of the fittest" in software evolution.
What about the "insides" of the program compared to other program insides? Sometimes people are aware of gross differences, like the language the program is written in. But the curiosity normally stops there.
The program "insides" have huge practical consequences. Depending on the details of how a program is written, it can be more or less:
- reliable/buggy
- easy to hack (secure)
- expensive to run/operate
- dangerous/costly to change
- able to respond quickly to changing loads
- speedy or slow
People commonly talk about structure or architecture, but those things are just the tip of the iceberg.
When you dig in, you find the equivalent of islands and continents of software. I've treated this subject here, for example. For example, you may have spent some number of years after getting your CS degree working on website design, and interacted with a community of people similarly engaged. Not everyone uses the same tools or does things the same way, of course, and you're likely to think there's quite a variety of approaches.
Then you go wild, dive into hospital management systems, and your mind is blown. What you thought was the wide variety of software and tools used by website designers turns out to be a whole continent apart from what you find in hospital automation. You learn that the flagship product of the company that has more than a third of the market (Epic) is written in a language you've never even heard of! It's like growing up in New Jersey among what you think is a very wide variety of people and going to Chennai -- a huge city in India you've never heard of, whose population is larger than your whole state, where the main language is one you've never heard of, written in a script you've never before seen.
Even more ignored are the obvious (to those who trouble to look) repeating patterns that can be observed in applications as they are first written and then evolve over time.
The Study of Software Evolution
To put it bluntly, the study of software evolution has barely begun. A few isolated souls, hardy or foolhardy as you like, have dipped their toes into the deep waters of software evolution. Those few have found worlds to explore, oceans whose depths have yet to be plumbed.
This is particularly the case because software is a "the only reality is the present" kind of field, as it now stands. It doesn't have to be this way!
Very few people study the history of math. They don't need to: anyone who learns math starts with math as it existed thousands of years ago. If they do well in math, they get all the way up to ... the late 1600's, when Newton and/or Leibnitz discovered calculus! Each step in math is based on and often depends on the earlier steps.
In some sense, this is also true of software. Except not really.
First, knowing nothing about software, you can dive right into the latest language and tools and use them to make something that works. Anyone think you could do that with differential equations, I mean without already knowing how to count, add and multiply? The difference is that in software, the "accomplished" new-comer really is standing on generations of older software ... of which s/he is blissfully unaware! The new programmer, of course, uses an incredible array of operating systems, drivers, networking, file and database systems, and on and on. These form the "land" on which the new programmer "stands" to work. None of the wonderful new stuff would work without it. The big difference, of course, is that using the underlying software and understanding the underlying software are two rather different things. This makes software totally different from not just math, but most of the other legitimate sciences (leaving out the fake ones like political "science" and the rest).
Second, when you're building a program, chances are excellent that you're not the first person to try building something very much like that program. Lots of details may be different, but even the details are probably not as unique as you might think. Suppose you need a program that sets up an account for a person, and then when the person does stuff, the details are checked with the person's account, and if OK are added to the account, and if not are rejected but recorded. This high-level description applies to a sales operation with customers, a medical office with patients, a manufacturing company with customers, and a bank. Given that programs like this have been built tens of thousands of times, don't you think that the ways of building programs like this would have evolved? That there would be pre-built parts, specialized tools for using the parts, building new ones and gluing them together? Special methods that have evolved and been refined to make sure it's done in the optimal way? Did any of you with any level of degree in Computer Science learn any of that -- other than, of course, some boiler-plate about the virtues of object-oriented languages and maybe some other fashionable stuff? Of course not! Anyone with any sense of fashion and/or status knows that these are not career-advancing subjects. Anything involving software history or comparative study of software projects is career-killing.
Conclusion
Software evolves. But it evolves differently than other things studied by science. Virtually no one studies software history in any way, much less the patterns of evolution that become apparent when you study that history. There have been a couple of attempts to learn why software doesn't get better in the way many other things do, most notably Fred Brooks with his book The Mythical Man-Month and other writings. I have dissected this flawed analysis in my work on Software Project Management and Wartime Software. See this for a start.
I have worked for years trying to identify some notable patterns in software evolution, and will be releasing some of that work in the coming months.