The Black Liszt

Twitter can improve software quality by losing most of its engineers

Twitter has fired boatloads of software engineers and more are jumping ship. Most people predict that losing all those engineers will lead to software disaster. But then, most people don’t know much about software and don’t realize what a disaster Twitter software has been for years. With some intelligent leadership, Twitter's software could dramatically improve while dispensing with 90% of their engineers.

Twitter Customer Losses

First, let's note that the situation has users really worried. Customers are bailing the ship:

Here's some up-to-date analysis:

While some say the massive customer losses are due to the new CEO, others are sure that the loss of so many excellent Twitter software engineers will seal the deal.

Twitter Software Quality

Those cool internet software companies – their top people wear hoodies or whatever they feel like. They must be fabulous programmers, right?

Nope. Facebook, for example, produces amazingly bad software. See this and the included links for details on how bad it is – and how they hide as much as possible from their users.

Facebook, Twitter and the rest aren’t better than “normal” software companies, except in how rich they made their founders. Internet software is a horror show, as I document here.

I dove into Twitter in particular years ago to see just how bad it was. I found and documented inexcusable failures, which they went to great lengths to disguise.

I tested doing searches for tweets with Twitter’s own search engine, and documented random patterns of tweets being dropped out or included from searches just days apart. See this and this.

What this means, among other things, that the basis of “trending on Twitter” is based on bad Twitter data – the kind produced by those wonderful, oh-so-cool Twitter engineers.

Twitter hides the awfulness of its terrible software from you

Note that the errors I documented were NOT associated with an error message. They were cases where the right answer (for example search results) could be determined and compared to what Twitter provided. That’s what I did, and found that Twitter blithely would state “here’s your answer,” and then provide an answer that was demonstrably false. It would have been more honest had they given an error message instead. But no -- that would have been honest; Twitter engineers, while terrible at building software that actually works, have become masters of the simpler, nefarious job of masking bad software.

Even with such errors, Twitter would still admit failure from time to time, showing this image:

Since then Twitter has gotten better at hiding their errors. Following the pattern of bad search results, they give you your feed of tweets and you merrily scroll down, reading, retweeting, liking, and whatever. They have learned to almost never admit failure. The fail whale shown above is history. How do you know if you’re getting all the tweets from the people you follow, and in the right order? You don’t! In fact, there is loads of anecdotal evidence that part of how their engineers spend their time is figuring out how to manipulate your feed for a variety of reasons -- including masking the results of their inability to do it the right way -- instead of just giving you a common-sense, complete ordered feed.

Fail whale? I haven’t seen it in years. They can have dozens of servers crash and burn and no one will know the difference. What Twitter does is just pull together your search results or feed based on whatever servers are still limping along with whatever the pathetic Twitter software will give them, and give you whatever crap it has. How will you ever know it’s incomplete, missing important things, etc.? Unless you’re an obsessive crazy person like me and run tests, you’ll never know.

So what do all those thousands of software engineers do all day long except attend meetings while relaxing on comfy bean bags and digesting all the free food the company provides? Obviously not much in the way of useful programming.

The alternative

Small teams of smart, motivated programmers typically out-perform teams of hundreds and thousands of people with titles of “software” employed in big bureaucracies – including in “cool” companies. The concept is simple: most programming organizations are like people who build bridges in times of peace. They take thousands of people years to build. When you build bridges in war, like the bridge over the Rhine in World War 2, you have to build it in a DAY, while under enemy fire. And it has to work. Small, under-resourced groups of programmers operate in wartime mode, getting things DONE while groups of thousands continue plodding away at requirements planning. See this. For more, see this.

I see this all the time in my work of evaluating small, innovative start-ups. The small companies don’t have enough time or money to do things the “right” way. They have to get things done. Fast! The ones that do succeed.

A great example from another field is the author of the first important dictionary of the English language, Samuel Johnson. He produced excellent work, arguably 200 times more productively than the big committee trying to do the same for the French language. See this for the story.

What Twitter does isn’t hard! And what it does hasn’t fundamentally changed for at least a decade! Those thousands of engineers couldn’t make it work well nine years ago when I ran tests, and all they’ve managed since then is do a more polished job of hiding the errors and bad answers.

Whether Elon Musk or someone like him is in charge, the best thing for Twitter and its customers would be for most of the programmers to be shown the door, for the remainder to get with the program of making things actually work and work well, or join their former colleagues as ex-employees. If they really get it going, some programmers who are actually good and, get ready, WANT to write code, GOOD code, may seriously consider joining a re-invented Twitter engineering group.

It might actually happen, since Mr. Musk is violating the most sacrosanct rule of management: getting involved with the people on the front lines of actually doing the work! There's even evidence that he seems proud of his disgusting behavior.

Who does he think he is??!!

Social Media Has a Long History

It seems like the whole world is in an uproar about social media, with frequent revelations of awfulness and malfeasance. The uproar is about social media such as Facebook, Twitter and the rest. The trouble is these issues have existed in one form or another in social media going back ... hundreds of years. What we are seeing is ignorance of the present combined with ignorance of the past. In other words, business as usual.

The Core Drivers of Social Media

Most people care about their place in society -- their status. They care about how they're perceived, who knows who they are and how others relate to them. While this drive seems to come to a kind of perverse peak in middle and high school, it persists for most peoples' lives.

Intimately related to caring what others think is the drive to express what you think and what you've done. While this is related to influencing others' thoughts, it seems to be a kind of innate drive as well.

In short, you want to tell people what you've done, what you think, and you want to hear about other people, particularly those you somehow are involved with or even just similar to in some way.

Closely related to this are the core concepts of status and fashion. It's a basic urge to want to see your status reflected publicly if it's high, and many people have strong interest in what high-status people do and how they do it. Particularly as fashions of various kinds wax and wane, from clothing to activity to speech, people who want to increase their status have an intense interest in learning what the new things are.

Key Characteristics of Social Media

What makes something Social media vs. some random other kind of media? It's pretty simple: social media mostly consists of media (words and pictures) that is about and either written/created by the person or sourced from that person. It's about what a person says, thinks or does.

Now let's get to the other key characteristic: money. Who pays for social media? After all, it costs quite a bit to produce it, and that money has to come from somewhere. Historically, the people who consume the media pay a little, while ... get ready ... advertisers pay a lot. Today, the incremental cost of delivering social media to the person who consumes it is so little that no one bothers to charge for it -- the whole cost is borne by advertisers.

Social media is an amazing phenomenon, deeply rooted in human drives and emotions. People produce the content for it for free -- they are glad to have things about themselves distributed at little effort of their own to those they may like to know about it. And they read about themselves and people to whom they are socially connected with, paying to do so if necessary. They can't help but knowing that the ads that are intermixed with the "content" are going a long way (in the electronic world, all the way) to paying for their reading pleasure, but it rarely bothers them. They also know that ads are targeted to particular groups of readers. It makes common sense, after all. No big deal -- if I were an advertiser, of course I'd want to show my ads to people who are likely to buy what I'm selling!

Earlier versions of Social Media

People like to imagine that social media are strictly electronics-age things. Mark Zuckerberg invented it, didn't he? No, sorry! Social media have been around for a looooong time. I could go all the way to ancient Sumer and Egypt, but I think the point will be clear enough with more recent examples.

Here is a notice in the Pittston Pa Gazette from 1928 about a function attended by my grandmother, Agnes Black:

Here is an ad that helped pay for that information to be printed:

More recently, here is a notice in the same paper from 1955 about a visit made by my parents:

Here is one of the ads that helped pay for the notice.

The notice was actually fake news! My parents visited with their two children, David Bruce Black and Douglas John Black -- not David and Bruce. The advertisers don't care a whit -- they just want the eyeballs to persuade them to buy some hot new technology:

I could give examples from many other places and centuries. Things have evolved, but since the principles are rooted in human in human nature, not as much has changed in principle as you might think.

Conclusion

Everything is about people. A great deal about people is relationships and status. The experiences we had in middle school and high school didn't disappear into nothingness. They just evolved as each of us entered new groups of people, each with its own pecking order and rules for engagement. One of the most ironic things about modern social media is that certain groups of people are really upset about what gets published, and want to make sure that only the "truth" is published. They, of course, want to be in charge of defining what "truth" is. Sorry, guys, in the world of social relations and much else, "truth" is nothing but a pretty veneer on top of raw power. Yes, your grace, your honor. Why should it be different now that we're staring into little screens and swiping while we walk?