As the use of computers grew rapidly in the 1960’s, the difficulty of creating quality software that met customer needs became increasingly evident. Wave after wave of methods were created, many of them becoming standard practice – without solving the problem! This is a high-level survey of the failed attempts to solve the problem of software development and security -- all of which are now standard practice in spite of failing to fix the problem! There are now software auditing firms that will carefully examine a software organization to see in which ways it deviates from ineffective standard practice – so that it can be “corrected!”
Houston, We've Had a problem
The astronauts of the Apollo 13 mission to the Moon radioed base control about the problem that threatened the mission and their lives.
The astronauts were saved after much effort and nail-biting time.
Should the people in charge of software make a similar call to mission control? Yes! They have made those calls, and continue to make them, thousands of times per day!!
Getting computers to do what you want by creating the kind of data we call software was a huge advance over the original physical method of plugs and switches. But it was still wildly difficult. Giant advances were made in the 1950’s that largely eliminated the original difficulty.
As the years went on into the 1960’s, the time, cost and trouble of creating and modifying software became hard to ignore. The problems got worse, and bad quality surfaced as a persistent issue. There were conferences of experts and widely read books, including one from a leader of the IBM 360 Operating System project, one of the largest efforts of its kind.
The Apollo 13 astronauts were saved, but the disaster in software development has resisted all the treatments that have been devised for it. With the invention and spread of the internet, we are now plagued with cybercrime, to the extent that ransomware attacks now take place (as of June 2021) ... get ready for it ... 149,000 times per week!! I think a few more exclamation points would have been appropriate for that astounding statistic, but I'm a laid-back kind of guy, so I figured I'd stick with a mild-mannered two. Here's my description from more than four years ago on the state of ransomware and the response of the "experts."
Following are highlights of the methods that have been devised to solve the problem of software development and security. None of which have worked, but have nonetheless become standard practice.
Programming Languages
The assembled experts in 1968 decided that "structured programming" would solve the problem of writing software that worked. Here's how that turned out. A group of experts recently convened to survey 50 years of progress in programming languages. They were proud of themselves, but had actually made things worse. Highlights here. One of those early efforts led to object-oriented languages, which are now faith-based dogma in the evidence-free world of Computer Science. New languages continue to be invented that claim to reduce programmer errors and increase productivity. What always happens is that the "advances" are widely publicized while the large-scale failures are concealed; here are a couple of juicy examples. Above all, the flow of new languages provides clear demonstration that programmers don't have enough useful work to do to keep them busy.
Outsourcing
While programmers babbled among themselves about this language or that method, company managers couldn't help but noticing that IT budgets continued to explode while the results were an avalanche of failures. Service companies emerged that promised better results than company management could achieve because they claimed to be experts in software and its management.
One of the pioneers in computer outsourcing was Ross Perot's Electronic Data Systems (EDS), which had $100 million in revenue by 1975. By the mid-1980's it had over 40,000 employees and over $4 billion in revenue. It continued to grow rapidly. along with a growing number of competitors including services branches of the major computer vendors. In a typical deal, a company would turn over some or all of its hardware and software operations to the outsourcer, who would add layers of management and process. They succeeded by lowering expectations and hiding the failures.
Off-shoring
As communications and travel grew more cost effective, basing outsourced operations in another country became practical. Off-shoring had the huge advantage that while the results were no better than regular outsourcing, the fact that employees were paid much less than US wages enabled marginally lower prices and a huge infrastructure of management and reporting which further disguised the as-usual results.
US-based outsourcers started doing this fairly early, while non-US vendors providing software services grew rapidly. Tata Computer Services first established a center in the 1980's and now has revenue over $20 billion and has been the world's most valuable IT company, employing over 500,000 people. Infosys is another India-based giant, along with hundreds of others.
Project Management
Deeper investment in the growing and evolving collection of project management methods has been a key part of software organizations. The larger the organization, the greater the commitment to widely accepted, mainstream project management methods. As new methods enter the mainstream, for example Agile, new managers and outsourcing organizations sell their embracing of the methods as a reason for hiring and contracting managers to feel confident in the decisions they are making. Project management has definitely succeeded in forming a thick layer of obfuscation over the reality of software development, quality and security. The only method of project management that has ever delivered "good" results for software is the classic method of wild over-estimation. See this for one of the conceptual flaws at the heart of software project management, see this for an overview and this for a comprehensive look.
Education and Certification
Nearly all professions have education and certification requirements, from plumbers and electricians to doctors. Software organizations grew to embrace imposing education and certification requirements on their people as a sure method to achieve better outcomes. This came to apply not to just the developers themselves but also to the QA people, security people and project managers. Unlike those other professions, computer education and certification has been distinctly divorced from results.
Standards, Compliance and Auditing
Standards were developed early on to assure that all implementations of a given software language were the same. As software problems grew, standards were increasingly created to solve the problem. The standards grew to include fine-grained detail of the development process itself and even standards that specified the "maturity level" of the organization.
Just as a new house is inspected to assure that it complies with the relevant building code, including specialists to inspect the plumbing and electricity, the software industry embraced a discipline of auditing and compliance certification to assure that the organization was meeting the relevant standards. A specialized set of standards grew for various aspects of computer security, with different measures applied to financial transactions and healthcare records for example. The standards and compliance audits have succeeded in consuming huge amounts of time and money while having no positive impact on software quality and security.
Suppress, deflect and Ignore
Everything I've described above continues apace, intent on its mission of making software development successful and computer security effective. People who write code spend much of their time engaged in activities that are supposed to assure that the code they write meets the needs in a timely manner and has no errors, using languages that are supposed to help them avoid error, surrounded by tests created by certified people using certified methods to assure that it is correct. Meanwhile highly educated certified computer security specialists assure that security is designed into the code and that it will pass all required audits when released.
How is this working out? In spite of widespread information blackouts imposed on failures, enough failures are so blatant that they can't be suppressed or ignored that we know that the problems aren't getting better.
Conclusion
We're over 50 years into this farce, which only continues because the clown show that sucks in all the money is invisible to nearly everyone. The on-the-scene reporters who tell all the spectators what's happening on the field can't see it either. What people end up hearing are a series of fantasies intended to deliver no surprises, except in those cases where the reality can't be hidden. Mostly what people do is things that make themselves feel better while not making things better. The time for a major paradigm shift to address these problems has long since passed.
Comments