The Black Liszt

Software Planning is Impossible

What else is new? Everyone knows that software planning is, well, just impossible. Live with it!

I mean something else entirely. I mean that software planning, in the usual sense of the word, is literally impossible to do. Planning for a new house? OK. Planning a new road or intersection? You may not like the cost, disruption or time, but it can be done. "Planning" in the sense that everyone means it and uses it for everything else, is literally impossible for software.

Here's why, in a nutshell: a "software plan" that is the exact equivalent of an architectural plan for a building, including the materials list is the software itself. Anything less is a vague sketch of what part of the building might look like, something a builder asked to quote on it would say, "I'll be glad to give you a quote when you give me a plan. Which this is not."

Let me explain.

Building plans

Most people understand the basics of planning a building, say a house.

You start by answering some very basic questions, like How Big, and Where. This location and size information determine much of the eventual cost of the building.

You then go with architecture, i.e., the guts of building planning. Starting with size and place, you then go into the general shape and style of the house, things like How Many of What Kind of rooms and finishes. You may interact with the architect, creating several rounds of plans. As you agree to one level of generality, you dig deeper, ending with things like colors, appliances, and materials (wood clapboard vs. Hardie board, etc.). If the architect is modern, he'll show you 3-D renderings and give you 3-D walk-throughs of fully furnished rooms.

Here's an overview that I built using common software, two snapshots from different directions:

Notice the landscape, the terrain, the bushes, the shadows -- these are just snapshots from a complete 360 degree view. You can pick the angle, distance, height and even the position of the sun. Those are images of real clapboards, doors and windows that are commercially available, everything.

Here's part of a floor plan I built with the same software:

 

And here's a variation of that plan with a bunch of furniture added and some other changes:

Here's a sample 3D interior view from the software's website:

The software enables you to not just have pictures, but to do a full 3-D live walk-through, just like in a video game.

If the architect uses modern software, the software will not only assure that the building is structurally sound, the software will fill in all the structural elements, including electrical and mechanical, and produce an item-level materials list.

That's the plan! You can give it to builders for time and money quotes, and to the local building department for permits. Once you make an agreement with the builder, off starts the construction project. Hopefully it finishes about when promised, and you end up paying the agreed-upon amount. Done!

When it's done, unless the builder has screwed up, it will look in physical life just like the drawings and renderings, with all the selected materials.

Software planning

Software planning is basically the same as building planning, right? Except even better! With buildings, you're digging holes, pouring concrete, putting up framing, and all sorts of time-consuming, physical things that are costly to buy and install. By contrast, software is just writing code, a very non-physical activity -- any change you want or mistake you make is easily fixed, without ripping out building materials or breaking up concrete. Moreover, building planning is like old-style "waterfall" project management. Agile, the modern trend in software, isn't possible with buildings.

Why is it, then, that everyone with practical experience knows that software planning is a mess, a total nightmare, pretty much no matter what planning regime you follow? Why do you think the industry keeps coming up with new ways of planning things? For building planning, the methods were always effective, and with the increased use of software, the methods have stayed the same, but the ability to render the results to clients and automate error-prone architecture work has gotten better and better. For software planning, we experience a ever-ending sequence of planning "revolutions," each of which promises to eliminate the perennial problems. Except the benefits are always in the future, and the future somehow never happens. What's going on here?

At root, the cause of the problem is simple: the analogy of building planning and software planning doesn't work! But we keep trying.

Here's why: a plan for a building at the level of detail I described above is truly comprehensive. The ability to render detail in 3D makes that clear, and the ability to generate the structural elements and item-level material list makes it crystal clear. While it's incredibly comprehensive for a building plan, no software plan comes close to that level of detail in terms of building software!

The reason is simple:

A building (or road or bridge) is passive, built by workers and machines from a carefully assembled inventory of passive parts and materials, things like lumber, wiring, pipes, siding, concrete, etc. A program is active, a carefully organized set of instructions and data (mostly instructions, i.e., software) that do things in response to active inputs and stored data, as a result of which stored data is added or altered and new data and/or actions are created in the outside world (e.g., new screen displays, messages sent to distant servers).

This difference is a BIG DEAL. It's EVERYTHING!

The Building metaphor for software is bogus

Think back to the beautiful building plan I showed above, complete with its ability to enable a person to do a virtual walk-through of the building. That's like a software plan, right? It's kind of like what people call wire frames, mock-ups of the UI, right?

Now imagine that the software plan we're building is for a combat-oriented, multi-player video game. How close is that ready-to-build physical building plan to a plan for the video game? Ahhh, maybe 5%? The easiest 5%, for sure.

The detailed building plan we can walk through is like the "world" that video games create. Except the world is always changing. No building plan can deal with a player racing through the building, crashing through a window and firing a blast that blows a hole in a wall -- a hole that is customized to the location and angle of the shooter, the type of wall and the kind of blaster used. The wall explosion could further impact another game player on either side of the wall, in real time. This is the hard part, and where the vast majority of effort of building a video game goes. It's the action part.

Is building commercial transaction software like building a video game? No, of course not. There are some elements of video games that are unique. In other ways, building commercial software is harder than building video games! Video games are completely self-contained "worlds." For better or worse, commercial software is very much part of an existing, amazingly extensive world -- of other software! Existing software that is always changing, has bugs, is extremely elaborate, and whose actions depend not only on what you ask of it, but what's been asked in the past -- context. Commercial software is a complex series of actions, many of which involve interactions with other software that is no less complex.

What's important in software, and takes the VAST majority of the work, is the action part. A building is passive. It just sits there. Even the "passive" parts of software are created, on the fly, by the action part. For example, when you first go to a web page, it might be fairly passive. But the second you click on something, action starts. Once you login, the action gets fierce, and is created, on the fly, just for you: with data that's been stored for you and things on the screen that are customized for you.

This is the difference between walking into one of those Amazon retail book stores, which are what they are, and going to the Amazon website as a logged-in visitor. In the store, personalization is performed by a worker who walks up to you, who of course doesn't know your complete history of interactions with Amazon, which the software does and reflects in how it interacts with you. This goes all the way down to simple things; for example, if I go to the web page of a book I bought years ago but forgot, the software helpfully tells me I already own the book, would I like to get it again?

Conclusion

As usual, metaphors lead us astray -- even the universally-accepted metaphors for software. This is really sad, and the fact that it's gone on for decades is even sadder, with no signs of changing for the better. I've gone into the sources of the bogus building planning metaphor in great detail in my book on Software Project management, along with more detail on why the metaphor doesn't work.

All this history and logic is besides the point, in a way: the important thing is that, even today, the bogus building metaphor for software enjoys near-universal acceptance, in spite of its persisting monumental failures. Instead, the "leading minds" of the industry yammer on about irrelevancies like Agile and Scrum that may make people feel better, but don't change the underlying problem.

You wish for a better world? Be one of the renegades who actually gets stuff done; try Wartime software. Here's the background.

Patient Incentives in Healthcare: Case Study

We all know that incentives work. That's why you always read about the "low prices" and the "sale" about to start, or the "limited-time offer." They're incentives to buy this or that. A server at a restaurant is incented to provide good service to get a good tip, and a salesperson is incented to sell by getting a commission. What a good idea it must be to apply this idea to healthcare, right?

Maybe the idea of incentives applies to healthcare. I take no position on that subject. But I do know that when the sprawling bureaucracy of a health insurance company tries to apply the idea, it turns into yet another costly bit of overhead that yields no benefits beyond allowing top executives who float blissfully above the facts and reality to claim that they're modern and innovative. Right.

Incentives in business

A business has strong incentives -- to get incentives right! A giant commercial or government bureaucracy has NO incentive to get them right. The business is aware that it's spending money to get customers, money that could be spent on an endless number of good things, from advertising to improving the product/service, to improving customer service to get more repeat customers, and on and on.

Here's the key thing: incentives are old news in businesses that have customers and need to make a profit. There is a long history of giving incentives, measuring how effective they are, and adjusting accordingly.

For example, in retail there were specialists who carefully controlled each season's products and set the incentives based on the experience of measuring the results. Already in the 1990's software products began to emerge that would take line-item POS (point-of-sale terminal) data from the last few years, and predict what would be the best time to start a sale, on which products, in which exact stores, and how deep the cuts should be. After side-by-side testing in multiple retail chains, the math-driven sales proved to be more effective than the ones generated by the best, most experienced people. So the industry transitioned to the algorithmic approach -- it's now malpractice if you don't use algorithmic sale incentives in retail.

Retail has gotten amazingly effective with incentives, following the classic path to excellence in AI/ML as I described in this series of posts.

Incentives by health insurers

Given this background, when the health insurance giants finally decided to apply incentives to their insured population, naturally they carefully studied incentives in other fields and adapted state-of-the-art techniques to their situation, right? WRONG!! In every case I've seen, they've done the dumbest things possible, not just starting from incentives 1.01, but screwing up so badly that any internal measurement system (which of course was NOT in place) should have resulted in the prompt cancellation of the project and demoting everyone concerned to a starting position opening mail. None of which happened, of course.

Incentive Case Study: Anthem

Anthem is an excellent health insurance company. It and its managers strive to be industry-best and provide great service to the employers and individual customers it serves. I am using Anthem for the case study here NOT because they're the worst -- far from it! I'm featuring them simply because I'm a customer, and so get a ground-level view of how things work there.

Sadly, as a giant, heavily-regulated bureaucracy, Anthem lumbers along and, along with its peers, gets really important things wrong. I wrote about the mess they made a couple years ago, first by allowing themselves to be hacked, and second by responding to the hack somewhat, ahem, ineptly.

Some time ago Anthem decided that Wellness and Health Incentives were something they should dive into. They now appear to be thoroughly committed to it, as they say loudly and clearly on their website:

Some of these programs may be truly wonderful. I make no comment on them. But I doubt that the one that was pushed onto me was exceptional, so let's dive in.

I decided it had been too long since I'd had a general health check, so I signed up for one and got it. After a while, I got a packet of stuff in the mail. Here's the top page:

Anthem clearly had gotten the claim for my visit and auto-enrolled me in their incentive program to get me to do the stuff they say, including getting such check-ups regularly. Wow, the incentive-program-babies who designed this program thought, we'll pay him some money for getting a check-up and maybe he'll do it again next year, hoping to get another card in the mail -- though of course we won't breathe a word about that.

Let's check out the rest of the package. Next page I get a wonderful, inspiring picture  of how exercise makes me healthy:

Next page, reality starts to hit. No more nice pictures and color. Just the facts, ma'am (the card itself was glued onto this page):

It's a gift card kind of thing! Except it's "pre-paid," and is accepted where Visa debit cards are accepted, but when you use it, you've got to lie and say it's "credit." Hmmm. And NOWHERE does it say how much money is on the card! I have to call or go online to find out. And of course the card doesn't say Anthem, it says SVM. Who are they?

Maybe I'll find out. Let's keep on. Next page:

Great. When I get a regular credit card, I call a number to make it live, and then away I go. For this one I have to read something and then check somehow what's on it. I wonder how much IS on it? I hope eventually they'll tell me. First I better read the rest.

Man, this print is getting mighty small! I wonder if I have to pass a test before I'll be allowed to use the card that may contain some secret amount of money that no one will tell me. Minus whatever fees and other stuff that they jam into it when they feel like.

Reading carefully, I find out that, even though it's a debit card, like for your bank, I can't just get the cash out of the card! What the &*&&$&*()$ is THAT about?? It's an incentive, darn it! A incentive that's a money incentive -- and I can't get the cash and spend it?? What is this, if I go to Burger King and order something they don't approve of the card won't work?? Who knows??!!

Reading more, I can't use it at an automated gas pump. Or at most restaurants. And there's a PIN, which I have to call to set -- in TINY print in the middle of the TINY PRINT page? I thought I was supposed to select CREDIT, and credit cards don't have PIN's. What's going on?

Maybe the next page will help:

Or maybe it won't. Or the next 3 after that, which are more of the same, and which I refuse to read.

Finally, last page, bigger print:

This appears to be the cheat sheet. This I can read. It appears I really do have to go online and enter a bunch of stuff, and remember all the conditions on using it, including the ones they don't repeat on this sheet. Maybe I can even find out how much money they're giving me!

If I were a regular person with some mild interest in whatever this incentive is, I would have dropped out by now. But I'm a fanatic and want to see how this story ends, so I'm going on to the next step. On-line we go! First step:

 

Get my card, copy the numbers in. Next step:

 

 

 

Put some more numbers in. Next step:

 

 

I appear to be in, but not really. I have to go back, enter the security code again and also enter the hard-to-read code they put in to stop robots. Wow -- all I can think is, this incentive must be huge. Why else would they be making it so hard??

Finally, I get logged in:

 

I think I've said "wow" a few too many times, but I should have held off for now. Tucked away in the upper right corner is the size of the golden goose I've spent all this time and effort seeking: $50! Maybe. Sort of. Except not in cash. And minus whatever fees. And not at restaurants or gas stations unless you follow the rules. And maybe there's a PIN, check the amount before each time you use it because you might have been dinged a fee, and...

And before I can touch any of it anywhere, I've got lots more information to enter. I'm outa here!

Conclusion

This incentive card program is one of the more bone-headed, dysfunctional things I've encountered in a while. Lots of lawyers, bureaucrats, managers and even publicity/image people contributed to it, but did anyone with, you know, real knowledge of how things like this work ever get a shot at it? The number of steps to get rewarded and all the uncertainty and conditions are guaranteed to produce maximum drop-out. In the reward card business this is called "breakage," and unethical rewards people try to maximize breakage; Anthem should be up for "rookie of the year" in the breakage stat.

It's one thing to try to create good behavior by giving an incentive. It's another to dangle the promise of an incentive, trick people into going through a horrific maze that most won't make it through, to get the incentive (of unknown value!), with the primary result that your basic impression of the health insurance company as incompetent and wasteful and something you should ignore whenever possible is strengthened.

Is anyone out there listening?

What are Software Fashions?

“Fashion” is a word we associate with clothes. Software is hard, it’s objective, it’s taught in schools as “computer science.” Software can’t have anything to do with “fashion” if it’s a “science,” can it?

Sadly, software is infected by fashion trends and styles at least as much as clothes. Fashion has a huge impact on how software is built. Understanding this, along with other key concepts like those involved in Wartime Software, can contribute greatly to building great software that powers a business to great success.

Fashion

We all know what fashion is, exemplified by fashion shows like this one:

 with impossibly thin female models strutting down cat walks wearing clothes that no one is likely to wear in real life.

Not as often, but guys too:

Far more important than models wearing extreme clothes is everyday fashion. I grew up looking at men dressed like this:

It's what men wore to church and to aptly-named "white-collar" businesses all the time. But there's nothing special about a suit and tie. Here's a look at Dutch fashion in the early 1600's:

Fashion is arbitrary! it's just what people wear. Everyone judges you by what you or don't wear, according to prevailing fashions.

Of course, "fashion" goes way beyond how people dress. It's how you act, how you speak, the accent you use, the interests you express, just about everything. If you don't think it matters, just trying wearing NY Yankee regalia into a South Boston sports bar and start spouting trash about the Red Sox. Lots of people who think they're the kind of people who are "above" fashion are driven by it nonetheless -- just look how they respond to people walking into a room, and if there's any doubt, seconds after the newcomer opens his mouth.

Fashion is about people. It's about belonging, status, fitting in or "making a statement." We live in a fashion-dominated world, like it or not.

Fashion in Software

I was one of those people who was convinced I was "above" fashion. Being above it made me superior, in my mind, to those who were slaves to it. During and beyond college, I bought the few clothes I wore from a local used clothing store and wore hiking boots most of the time. Once when I was in my first post-college programming job, I was called out of the cubical where I spent most of my time, heads-down, programming away, and asked to come into a meeting in the front of the building. I walked in to a meeting populated entirely by men in suits. One of the men I didn't know glanced at me and immediately exclaimed "finally! We get to talk with someone who knows things!"

I had been called into a sales meeting, and one of the visitors had software questions no one knew the answer to, so "the suits" had called in the guy who knew the answers. How I dressed and acted in fact made a statement to the visitors -- I dressed the way a programmer dressed, the kind of programmer who wanted to program, not one who aspired to management. So, like it or not, I was making a "fashion statement," while fooling myself thinking that I was "above" fashion. The hard fact is, no one is "above" fashion. The way we dress and act and talk, the choices we make, says loads about us. Those unavoidable choices clearly establish our place in various groups, social and status hierarchies.

Software Fashions

Given how fashion-driven our lives are, it would be shocking if programmers weren't fashion-driven in their shared activity of software. In fact, they are! Sadly, the vast majority don't think their choices are fashion-driven. They believe they're modern, with-it software professionals who are using the proven, advanced methods for doing software. The trouble is, few of them take the trouble to cast a knowledgeable, cold, hard eye on the arguments, experience and facts concerning their chosen methods and tools. They've made their choices so they can be with whatever software social group they identify with and/or aspire to. It's all about relationships and status. If you're ambitious, you may want to be with the "cool kids," members of an elect social group, yes, in software. And it works! If it didn't "work" (elevate their software group status), they wouldn't do it.

We like to think of people who wear fashion-forward clothes as being empty-headed, shallow people. Surely, programmers aren't that! But to the extent that they adapt fashion-forward software, that's exactly what they're doing -- only worse! They're lying to themselves, deceiving others, and making believe they're pushing some trendy software thing because it's advanced technology, yielding results superior to the obsolete stuff that used to be the standard.

Just like with clothes, software fashions evolve. Fads start and may become hot. The fad may evolve into a fashion as it spreads, with people who haven't adopted it taking notice. The fashion may further evolve into standard practice, with eyebrows being raised for anyone who dares to question it. More often, the fashion simply fades away.

It's rare for any fad or fashion to be explicitly repudiated -- oh, that was a terrible idea, people are turning away from it for good reason and here's why. No one says that! In the "advanced clothing" area, everyone knows that fashion is "just fashion." In software, fashions aren't considered "fashions;" they are considered "advances," emerging modern techniques that are objectively better than what came before, like a new drug or operation that has emerged from clinical trials and now saves lives that used to be lost! Saying that a widely adopted software fashion was never proven, was always a bad idea, but got widely used and promoted anyway would expose the game. So when software fashions die, they fade slowly away and simply stop getting used and talked about.

After a fashion fades away, it's generally forgotten by nearly everyone, usually except for a band of true believers. Some of the more intellectually heavy-weight fashions retreat to academia, where they live on, always with "exciting futures."

Some of these flourished-but-died fashions rise to live renewed lives. In one pattern, the fashion was so broadly accepted but such a failure (though rarely discussed as such) that when it becomes fashionable again, it has a new name. No one ever refers to last time, why the older fashion didn't work out, and why this slightly altered version of the same thing will. In another pattern, the fashion baldly re-emerges with exactly the same name, and nearly the same blazing-bright future as the last time. Sometimes there are even some successes. But it remains a fashion and therefore has disappointing results to anyone who cares to look, which is essentially no one -- such is the social power of fashion!

Not all fashions die. Some fashions have such powerful support that they become locked in as part of modern mainstream practice, sometimes even becoming part of so-called Computer Science, or at least IT Management. I don't fully understand how and why this happens, but I know that in part, the fashions that become standards address some widely felt need in the people involved in software. When this happens, there is often a series of waves of renewal or reform -- while the reformers refuse to acknowledge fundamental problems with the fashion-enshrined-as-best-practice, they latch onto some minor tweak or addition and promote it, usually with a new name, as the best way to get results with standard-practice X.

What are these software fashions exactly?

I have already talked about a few important toxic software fashions. I have gone into huge detail for a couple of them, with multiple blog posts and even books. I'm gradually starting to understand this bizarre phenomenon in terms of powerful social fashions with bad results, masquerading as "advances." I'm seeing the resistance to seeing the Emperor's New Clothes for what they are because of the self-delusional conception of software as a science/math-based STEM field, rather than as the pre-scientific collective group-think that it largely is.

I have already challenged a couple of the modern hot fashions, for example my series of posts on AI/ML -- a classic example of a once-hot fashion that has died away and been re-born multiple times. This particular fashion is distinguished from some of the others because there are some truly excellent algorithms at the heart of it that can be applied to great benefit -- and this has been true for decades! But because of widespread fashion-itus, the money and effort spent on them is mostly wasted.

In future posts and at least one future book (in process), I will continue to dive into and expose specific software fashions for what they are. I do this in part to strengthen the resolve of those special people and groups (some of them ones we've invested in) with the understanding that the "wrong" or "uncool" things they're doing give them a fundamental business/technical advantage, and they should stick to their guns and ride their truly effective methods to success, to the benefit of all concerned.

Further reading:

Resistance to treating scurvy compared to software disease treatments.

The modern AI/ML fashion.

The story of how I discovered the fashion vs. what-really-works issue.

Deconstructing project management.

The story of fashions-becoming-standard-practice.

The Cloud fashion.

Big Data fashion. Big Data bubble.

Evidence-based software methods don't exist.

The recurring fashion of data definition location.

 

Medicine as a Business: Medical Testing 5: The Results

I've gone through quite a bit to get the results of my MRI. See here for the previous installment, and here for the start of the saga. I glanced at the report and it looked good. In this post, I'll describe the unsettling things I found when digging deeper. In sum: the whole baroque nightmare of scheduling, performing and delivering results of medical tests is not only inefficient and riddled with needless high cost and waste, but more important there are serious quality problems leading not just to delay and waste, but bad results.

I fully acknowledge that what has happened to me pales beside the waste, incompetence and fraud that pervades the worst medical systems. My point is that, even in the best healthcare systems, bad things are happening.

The Results

I glanced at my hard-won test results and felt OK, mostly because I had been told that radiation therapy took a looooong time to show results, and I shouldn't expect anything to change at the first MRI. There was no OMG or IT'S GROWN LIKE CRAZY in the notes when I glanced at them, so I let it rest. I was tired out from all the effort of getting the darn thing.

Then I looked more carefully. I'm still OK for my personal case, but on close examination, I realized that the final report of the MRI was consistent with the crazy things that led up to it: scheduling the test, taking it, and getting the results. It's all part of a bizarre system that has GLARING flaws that seem like they should be easily fixed, but nothing much happens.

Here is the key paragraph from the earlier of the scans, the one that led to the radiation treatment:

Here is the corresponding paragraph from the second scan, the one I struggled to get:

The first thing that jumped out at me was the simple observation that there are no standards! Radiologists are incredibly smart, well-educated people. College degrees. Super scores on GMAT tests. Degrees from incredibly-hard-to-get-into medical schools that have TINY numbers of students. Then more years getting further training to become medical imaging specialists -- usually five more years, on top of the four years of college and the four of medical school! 

If I showed you the whole report, you'd immediately see that even the paragraph and subject-matter organization was different. About the most glaring thing to me was that the second report gave actual dimensions of the tumor, while the first did not! Don't you think that when tumors were involved, specifying the actual size would be the standard?

There's lots more that could be said, but I'll leave it with these simple observations:

• There is no system in place to record and assure that the required location is being imaged. The key thing can be missed because it wasn't imaged.
• There is no consistency of exactly what is reported on and how it is reported. It is difficult to compare reports and assure that what you need is there.
• For tumors, there is no consistent positioning and measurement of size. You could miss a tumor altogether, and easily miss size/location changes.

Conclusion

I'm in remarkably good shape, having a scary diagnosis of an extremely rare cancer. I received great treatment from highly skilled professionals at every step of the way. I received chemo that had only 25% chance of working, but it shrank a rapidly growing tumor. Then, when it started growing again, I got right into radiation, which has at least prevented further growth, and should finally stamp it out. I have nothing to complain about, and a great deal to be thankful for, including all the professionals who treated me. 

I've written this series of posts about Medical Testing NOT as an indictment of the individuals who have treated me, but as a serious indictment of the system in which they work. Here is a summary list of the things that "could be improved;" details are in the prior posts of this series:

• Scheduling and getting a pre-auth for a test can be a labyrinth and delay-filled nightmare.
• There are multiple issues with the wasteful, expensive and time-consuming blood test.
• There are multiple issues with specifying and following an exact procedure for the location and mechanism of the scan itself. Instead of being in the system, the nurse has to get information from the patient and guess about other things!
• The equipment and software is built in a regulation-protected bubble, which results in 10X or greater cost and trailing-edge technology.
• Getting results that have already been created by the radiologist can be an obstacle-filled maze, even if you try to use a “patient portal” that is supposed to make things to easy and transparent.
• The patient portal is mostly a sales pitch about how the hospital is wonderful – getting the information is a big problem, and then important information is wrong or just plain not there.
• Finally, the results produced by super-highly-trained doctors based on these expensive and questionable inputs don’t meet any modern standard for content.

Why doesn't anyone in management seem to care? I've often wondered this, and speculated about why several times. What's clear is that there's a hierarchy of prestige in every society, including ours, and that the top of the hierarchy is populated by people who focus on strategy, policy, direction and messaging. They are, for the most part, "above" getting "lost in the weeds." Sorry guys; the action is on the ground, where real things happen to real people. That's where you discover what's wrong, and when you "fix" something, that's where it's got to change.

Medicine as a Business: Medical Testing 4: Getting the Results 2

I've done everything I can to use the Mount Sinai patient access portal to access my test results, without result. (See here for the start of this saga, and here for the previous post.)

Now it's time for desperate measures. I finally take the radical step of picking up the phone and calling for help. Surely the results are there!

Here's what happened.

• I called.
• I was put on hold.
• I explained the situation.
• I was put on hold while the CSR checked something out.
• More questions. More holding. Rinse and repeat several times.
• Hold while I check with my supervisor.
• Rinse and repeat several times.
• Final result: we can't help you, call your doctor and have them help.
• But what can they do that you, the specialist can't??
• They have a number they can call to get help.
  

More than half an hour on the phone, and I get to ask my doctor to call someone who won't be able to help either. And I'm sure my doctor would jump at the chance to fix this problem, since he looooooves the EMR so much!!

Desperate and out of options, I call the doctor's office.

• I got transferred to a 5 minute wait before getting a dial tone.
• I got transferred to voice mail.
• I got transferred to nowhere again.
• Again.

Finally, someone picked up whose voice I recognized -- the office receptionist. I explain the problem, and he tells me that the Mount Sinai Radiation Center uses a different EMR than the rest of Mount Sinai!! Apparently one that doesn't send patient data to MyChart.

He promises to get me into the Radiation Center EMR patient portal AND send me the results. "What's your fax number?" he asks. "Umm, can you send it by email?" Pause... "Sure, I can figure out how to do that. What's your email?" I gave him the information, and five minutes later, I got an email with a PDF document attached. The document had the test results and instructions on how to get into the patient portal.Thank you!

Problem solved! I read the report, and the news was good. The thing that had been growing had stopped growing. But self-sacrificing guy that I am, I didn't stop there. What would have happened had I not persisted in my calling, and connected with a helpful and knowledgeable receptionist? After all, the report was supposed to be in the patient portal.

So I persisted. I decided to get into this special patient portal and finally see that the test results were actually posted there and available to me.

The Radiation-only EMR and patient portal

Leaving out all the details, I followed the procedure and after only a moderately odious amount of work (I had an access code!), I got into the portal:

 

Then I went to the test results, where my report should be:

It's not there, of course. Why am I not surprised?

The test results report should have been in My Mount Sinai Chart. It was not there, as confirmed by multiple levels of customer support people. It should also have been in the Radiation Oncology patient portal. It was not there, as you can see above. Given that an insider was able to access the report quickly and send it to me, the report was certainly in both EMR's. It was in the normal Mt. Sinai EMR, because that's where the doctor who wrote the report put it. It was also in the Radiation Oncology EMR, because that's the EMR of the doctor who requested the test -- and as I learned early in the process, it was easy for people in the radiation center to put orders into the "main" system.

Here's the key point:

Neither of the two EMR's at Mount Sinai that were involved with my test put a copy of it into the relevant patient portal so that I could see it. While I managed to avoid the usual doctor's appointment to find out the results, it's not clear how much time and frustration I saved in the end. Here's what was promised:

What was the reality?

• The test results were not available in MyChart. Is Mt. Sinai management unaware of this? Are they just lying and hoping to avoid embarrassment, as they do with other important "low-level" things? See this for a juicy example, and this for context. Either choice is unacceptable.
• The customer support service, when finally available, was unable to help.
• The original doctor's office was unavailable.
• The SURPRISE! special, different EMR used by my Mount Sinai department also didn't have the report.
• I only got the report because of repeated calling and a chance encounter with a kind receptionist.

Yeah, yeah. I'm computer and math guy, and I know statistics, and I know this is just one example. But can you really imagine that what I went through was a giant, almost-never-happens, tiny blip in a uniform fabric of excellence? Right. Wanna buy a bridge? I've got one real cheap for ya...

The E-mail!

Wait! There's more! After I drafted the saga of getting my greedy hands on the MRI results, something happened.

About a week later I got an email:

WHAT!!?? This test result was supposed to be on the radiation center's portal!

What's more, the only reason I got the email telling me the result was available on the Mount Sinai patient portal was because I was previously a patient and had signed up for it. If I had come into the Radiation Center directly, without having a history at the broader hospital system, I'd still be waiting.

On July 24 I'm told that the result was posted and available to me. A result from a test that was posted to Mount Sinai's system on July 3, 3 weeks earlier. It's a good thing we've got computers -- if it took 3 weeks to make a copy of a short document from one place in the Mount Sinai computer system and store the copy in another place in a related program in the same computer system, imagine how long it would have taken to do it manually! Years, probably!

I'm writing this on July 31, 2018, so by now the result surely will be posted on the patient portal for my doctor, nearly a month after the test was taken, right? Let's check:

Nothing is available. So much for the Radiation Center's patient portal.

Now I'm curious. Is the test really there, even though on the wrong portal? Here's the results list:

Yes, it's there, top of the list.

MyChart also provides a convenient to-do list, things I'm supposed to do, and there's something on the list. Better check it out, even though no one's told me there's something for me to do; this subject is important to me, to put it mildly, and I wouldn't want anything to slip through the cracks. Here's the to-do:

Oops. The MRI that was expected to be taken on June 11, actually taken on July 2 because of my initiative, and posted to the portal on July 24 is listed as a to-do item. The EMR evidently failed to connect the work order with the fact that the work ordered was performed and the results delivered.

This sounds benign, but it's actually scary. Deeply scary. The system doesn't match orders placed with results delivered, which means that orders could hang in space, ignored, with patient-essential work undone, unless a concerned and involved patient tracks it. In my case, there was a concerned and involved, not to mention detailed-oriented patient. What about the normal case? How many important things just hang out on a to-do list, undone, until they are "cleaned up?"

There's more trouble coming. When I glanced at the results, I got the impression things were OK. But when you dive in, ... see the next and final post in this series.

Blockchain is like the Wizard of Oz

If you haven't already seen the classic movie "The Wizard of Oz," I highly recommend it. It's entertaining and instructive. Its lessons remain applicable today -- they can even teach us about the amazing Blockchain technology that is poised to transform so many industries, solve so many long-intractable problems, and that is attracting such massive attention and investment.

The Movie

Dorothy, along with her dog Toto, you may recall, is swept up from her home in Kansas by a tornado, and eventually comes to earth in the land of Oz.(Credit.)

It's quite a place, populated by witches and munchkins, among others. The good witch of the North, Glinda, tells Dorothy that the wonderful wizard of Oz may be able to help her get home, so she sets out on the yellow brick road to the Emerald City, where he presides. Along the way she meets the Scarecrow who needs a brain, the Tin Woodman who desires a heart and the Cowardly Lion who needs courage. The four of them join forces to ask the wonderful Wizard's help.

The Wizard promises to help them all -- but only if they bring him the broomstick of the Wicked Witch of the West. So off they go and confront the Witch.

Eventually they defeat the Witch and bring her Broomstick back to the Wizard's palace. They walk down the intimidating hall 

Until they reach the Wizard's throne.

Gathering up their courage, they present the broomstick to the terrifying Wizard...

and ask that he fulfill his promise to them. 

The Wizard stalls. Meanwhile, Toto the dog, noticing something, pulls aside a curtain and reveals a man talking into a microphone. It's the real Wizard: the Great and Awful Wizard of Oz is just an ordinary man!

Dorothy confronts him. He admits he's just an ordinary man, and a humbug at that.  

He gives the Scarecrow a diploma, the Lion a medal and the Tin Man a heart-shaped ticking watch, helping them see that the attributes they sought were already within them. He offers to take Dorothy and Toto home in his hot air balloon. Then there's a mishap, and he leaves without her!

The story ends happily, because the good witch intervenes, and shows Dorothy how to return home under her own power, repeating three times "There's no place like home."

Blockchain

What can the Wizard of Oz possibly have to do with the marvelous emerging technology of Blockchain, which is set fo transform so many domains that are badly in need of help?

The movie has amazing lessons for us. I can't spell them all out in a single blog post. Here's a start:

Dorothy is stranded in a strange place and doesn't know how to get home. People who run financial systems have problems like lengthy settlement times that aren't getting solved.

Dorothy meets other people in the strange place who also have serious problems. People in other domains, like healthcare, have long-standing problems like EMR interchange that aren't getting solved.

The Good Witch tells Dorothy that the Wizard of Oz can help her get home. Authoritative people tell us that Blockchain can solve those problems.

Dorothy travels a long way to the Emerald City with her friends to ask the Wizard's help. After lots of work, people commit to the money and effort of a Blockchain trial.

The Wizard tells Dorothy that she has to bring the Wicked Witch's broomstick before he'll help them. Blockchain experts explain all the work we have to go through to get a test that has a reasonable chance of success.

Dorothy and her friends go through battles to get the broomstick, finally killing the Wicked Witch to get it. After lots of money and time and experts, a trial is finally underway.

Dorothy and her friends approach the Wizard and ask him to do what he promised. The sponsors of the blockchain project insist on results.

Toto pulls back the curtain, and reveals that, far from having amazing powers, the great and awful Wizard is just an ordinary man, and a humbug at that. The sponsors finally see that Blockchain solves no problems and is worse in every way than a normal DBMS.

The Wizard makes nice words that make her friends feel better, and after promising to solve Dorothy's problem, abandons her. Blockchain can't do much of anything, outside the context of Bitcoin, and when it appears to "work," the results are awful.

Glinda the Witch tells Dorothy to close her eyes, tap her heels and say the words three times. She wakes up in her bed in Kansas. Her relatives think she's had a dream. The Blockchain executives quietly let the project fade away. They do their best to calm their minds, refuse to admit defeat, and go back to their normal lives.

Conclusion

The world of Blockchain is indeed like the Wizard of Oz. While you're "in" the movie, you're convinced it's real, and so is everyone around you. When you wake up, you're back in normal life and understandably reluctant to think the amazing experiences you've had were "just a dream." But everyone else knows that's all it was. A dream that seemed good at the time, but turned out to be, yes, a bad dream. See this for a fact-based dissection of the bad dream.

 

Social Media Has a Long History

It seems like the whole world is in an uproar about social media, with frequent revelations of awfulness and malfeasance. The uproar is about social media such as Facebook, Twitter and the rest. The trouble is these issues have existed in one form or another in social media going back ... hundreds of years. What we are seeing is ignorance of the present combined with ignorance of the past. In other words, business as usual.

The Core Drivers of Social Media

Most people care about their place in society -- their status. They care about how they're perceived, who knows who they are and how others relate to them. While this drive seems to come to a kind of perverse peak in middle and high school, it persists for most peoples' lives.

Intimately related to caring what others think is the drive to express what you think and what you've done. While this is related to influencing others' thoughts, it seems to be a kind of innate drive as well.

In short, you want to tell people what you've done, what you think, and you want to hear about other people, particularly those you somehow are involved with or even just similar to in some way.

Closely related to this are the core concepts of status and fashion. It's a basic urge to want to see your status reflected publicly if it's high, and many people have strong interest in what high-status people do and how they do it. Particularly as fashions of various kinds wax and wane, from clothing to activity to speech, people who want to increase their status have an intense interest in learning what the new things are.

Key Characteristics of Social Media

What makes something Social media vs. some random other kind of media? It's pretty simple: social media mostly consists of media (words and pictures) that is about and either written/created by the person or sourced from that person. It's about what a person says, thinks or does.

Now let's get to the other key characteristic: money. Who pays for social media? After all, it costs quite a bit to produce it, and that money has to come from somewhere. Historically, the people who consume the media pay a little, while ... get ready ... advertisers pay a lot. Today, the incremental cost of delivering social media to the person who consumes it is so little that no one bothers to charge for it -- the whole cost is borne by advertisers.

Social media is an amazing phenomenon, deeply rooted in human drives and emotions. People produce the content for it for free -- they are glad to have things about themselves distributed at little effort of their own to those they may like to know about it. And they read about themselves and people to whom they are socially connected with, paying to do so if necessary. They can't help but knowing that the ads that are intermixed with the "content" are going a long way (in the electronic world, all the way) to paying for their reading pleasure, but it rarely bothers them. They also know that ads are targeted to particular groups of readers. It makes common sense, after all. No big deal -- if I were an advertiser, of course I'd want to show my ads to people who are likely to buy what I'm selling!

Earlier versions of Social Media

People like to imagine that social media are strictly electronics-age things. Mark Zuckerberg invented it, didn't he? No, sorry! Social media have been around for a looooong time. I could go all the way to ancient Sumer and Egypt, but I think the point will be clear enough with more recent examples.

Here is a notice in the Pittston Pa Gazette from 1928 about a function attended by my grandmother, Agnes Black:

Here is an ad that helped pay for that information to be printed:

More recently, here is a notice in the same paper from 1955 about a visit made by my parents:

Here is one of the ads that helped pay for the notice.

The notice was actually fake news! My parents visited with their two children, David Bruce Black and Douglas John Black -- not David and Bruce. The advertisers don't care a whit -- they just want the eyeballs to persuade them to buy some hot new technology:

I could give examples from many other places and centuries. Things have evolved, but since the principles are rooted in human in human nature, not as much has changed in principle as you might think.

Conclusion

Everything is about people. A great deal about people is relationships and status. The experiences we had in middle school and high school didn't disappear into nothingness. They just evolved as each of us entered new groups of people, each with its own pecking order and rules for engagement. One of the most ironic things about modern social media is that certain groups of people are really upset about what gets published, and want to make sure that only the "truth" is published. They, of course, want to be in charge of defining what "truth" is. Sorry, guys, in the world of social relations and much else, "truth" is nothing but a pretty veneer on top of raw power. Yes, your grace, your honor. Why should it be different now that we're staring into little screens and swiping while we walk?

 

Medicine as a Business: Medical Testing 3: Getting the Results

If you went to the time and trouble of a medical diagnostic procedure, chances are ... you want to know the results. ASAP!

It's a perfectly reasonable desire. In most areas of life, getting the results of something you paid for is pretty easy. If the results are information, most organizations just send it to you -- by snail mail, email, text or whatever you've arranged. For example, think about the crucial tests you take that have so much influence on your schooling and career, things like the SAT, MCATS, LSAT, and professional certification tests. You take the test and they send you the results in a standard way.

Not so in the wonderful world of medicine! In that world, you go to considerable trouble to arrange the test, and once it's been taken ...the fun of getting the results begins!

Getting medical test results

The usual pattern of getting the results from a medical test appears to be based on the assumption that patients are both stupid and illiterate. No way can you just send the results! The patient has to make an appointment with highly qualified medical person, who then patiently explains to the patient what the results were and what they mean. Plus, there's an office visit to be paid for.

We are told, however, that there's a revolution going on with medical record transparency. In this wonderful new world, patients can access their medical records themselves!! The major EMR vendors now support a "patient portal" for making such results available online, and major hospital systems brag about it.

Hmmm, I wonder if that's how I could get my results. Oh, I remember now, Mt. Sinai has a patient portal! I'm even signed up for it! Oh, good, this should be easy...

Getting my results from the Patient Portal

The test was ordered at Mt. Sinai. It was performed at Mt. Sinai. I have a MyChart patient portal account at Mt. Sinai. This should be a piece of cake. I pull up the main screen:

Isn't it nice? The EMR software provider, Epic, has a patient portal module called MyChart, which Mt.Sinai has cleverly called My Mount Sinai Chart. All I have to do is login, and I'll surely be able to access my recent test result, just like they say!

I login. I'll spare you the details, and keep it short: the MRI report is not there.

How is this possible? What happened to "no more waiting for a phone call or letter -- view your results..."??

I have just one thought. Maybe the fact that my original doctor left Mt Sinai and that I signed up for the MRI with a new doctor at Mount Sinai confused the system. Maybe I was signed up under a different identity!?

I poke around on MyChart a bit more. In reality, I visited the Mt. Sinai radiation center 30 times over about a six week period, and had separate consults with the doctor in charge of my radiation at least four times. NONE of these visits are listed. In fact, the last visit recorded was from 2016!

MyChart is still a wonderful program, probably ready, willing and able to show me all my stuff, but probably human error resulted in me being entered as a new person. All I have to do is create a new account, and I'll find all my records.

Signing up for the patient portal account

I'll dive right in. Given how important this is, the portal is probably written to make this effective and efficient. Here goes! I click on set up new account and get to here:

What's this activation code business? I look around and find this:

Odd. "Sign up online?" I thought that's what I was already doing! At least there's something relevant for me to click. I click it and get this:

That's more stuff to enter than I've seen in a while. There's a lot that could be said about this form and how it works, but I'll just point out one unique aspect of it:

When was the last time you had to enter your county? Even better, even if you've already entered the state, you get a list of all the counties in the whole USA!

Once you get to this point in the form, you realize that the creative people who built this software have actually created an obstacle course, a long and challenging one, hoping that most people will drop out from exhaustion long before completing it. And we haven't gotten to the really good stuff yet.

Establishing identity for the patient portal

Apparently it's really, really, REALLY important to make absolutely SURE that only the person themselves signs up for chart access. After filling out the form you see above, I got my identity hammered at:

Next, where have I worked:

A home equity loan:

My bank:

My former home:
Finally, after accurately answered all of these questions, and risking totally awful 100% identity theft if their system is compromised, I get this:

 

At this point, a sensible person would have given up and tried to make an appointment with a doctor, so the doctor can access the results document and essentially read it to me. But convinced as I am of my ability to read documents (egotist that I am), I decided to plunge ahead and try another path to getting the document. The next post continues the story.

The Hierarchy of Software Status

You might think that the hierarchy of status in software closely tracks the hierarchy of software skills, which I explained here. Hah! It is true that there is a small but important subset of people whose internal sense of status tracks the skills hierarchy reasonably well. But not for most programmers. The reason is simple: non-programmers' assessment of software status is unrelated to skill! It's a whole different hierarchy that has nothing to do with the ability to conceive and write good code that works!

Here is an earlier attempt to explain this issue.

This is NOT just an "academic" subject, BTW. It is absolutely crucial to getting good software done. See this for another angle at addressing the same subject.

Status

Status is part of human existence. There has always been a status hierarchy. Not long ago, we had lords and peasants in which the status differences were overt:

Status itself has barely changed since those days, though the clothing and other means of expressing it has changed a great deal.

Status is expressed in clothing, language, money, power relations, human interactions and in much of what we do. It would be shocking if generic human status relations did not apply to software. What's interesting is the translations that are made, which track medieval lord-and-peasant relationships quite well.

Status in Software

The easiest way to explain how status works in software is to look at how "close" you are (along multiple dimensions) to real people using code today. The closer you are to actual code and/or people using code, the lower your status; the farther away, the higher your status.

• One dimension is time. The highest status of all is enjoyed by people who think great thoughts about how some software might be built by some undefined group at some undefined future.
  
• One dimension is management layer. In any group, the "lowest" person in terms of management levels has the lowest status, the manager of the worker's manager has twice-higher status.
• One dimension is closeness to a real user. Anyone working in any way on code that isn't yet used by anyone has greater status than anyone working on code that is in actual use.
• One dimension is tied to the flow of interactions with users. If you work in any way on the flow of code or changes to code that is on its way to users (i.e., to production), you have higher status than anyone working in any way on the flow of information and communications that flow from users to the supplier or deliverer of the software. In other words, building software is higher status than customer service.
• One dimension is trendiness. It's really important to identify yourself with tech trends that are exploding in fashionable talk.
  • If you're too early, you lose status. When someone hears you babble on about something, if they check it out on conferences, publications and the web and it's obscure, you seriously lose status.
    
  • If you're too late, you lose status -- you're pegged as someone who's part of the crowd. Better late than never, of course. You also lose status if you keep bringing up something that's starting to fade away.
  • Maximum status is early-peak attention.
  • You can't actually know anything or do anything in the fashionable area and have any chance of maintaining status. You've got to direct, hire, establish, prioritize or strategize. Those all keep your hands safely clean and enhance status.
• A dimension that becomes crucially important during the hiring process, and which often persists long after the hiring date, is the visibility and perceived success of the person's prior company. The more visible and successful, the higher the status.
  • No one knows whether the person had positive impact on the company's success or the opposite. The person from Google has a halo, regardless of what they did.
    
  • This is a subject I treat in considerable detail in the Software People book, along with other important hiring considerations.

Naturally, there are nuances.

• For the people who deal with the flow of data back from users:
  • There are people who respond to customer complaints and problems. Super-low status.
    • Some of those customer issues are real bugs in the software! The very lowest status goes to the people whose job is to patiently educate the stupid users whose inability to do the simplest things with such a wonderful application is obvious to everyone except themselves. The people who get called in to see if there really might be a bug have higher status.
      
    • The people who fix such bugs have even higher status, but still remarkably low.
  • There are people who analyze conversion rates and details of customer use of the application.
    • All these have higher status than anyone involved with customer complaints or bugs.
      
    • If they deal with things like conversion rates, that's related to high-status strategy and marketing and therefore higher status than anyone whose job is to tweak the application to make it better for users.

There's another important aspect of status. I suppose I could jam this into the "dimensions" list, but I think it's simpler to say this other important thing about status this way: the more code you write, the lower your status. Writing no code is, with some important exceptions and subject to the general dimensions above, higher status than writing any amount of code.

Yes, writing code requires highly specialized knowledge. But so do lots of other low-status things, like repairing garage door openers or installing lawn sprinkler systems. The people who manage the repair people are, of course, higher status than the people who get dirty or use their muscles doing physical work. Similarly, the people who metaphorically dirty their minds while doing demanding, hard mental work are lower status than those who direct them in any way.

Another way of thinking about this is asking, what's your main focus? To the extent that it's a body of code, your status is lower. To the extent that it's people, your status is higher, with some exceptions, see customer service above.

Like anything else, there are exceptions. I have met highly productive coders who are older and well-regarded. But they're rare, and usually they're tucked away in a corner if they're tolerated at all.

Conclusion

The way status is perceived in software is perverse, and accounts for a great deal of the dysfunction we see in the software world. It is sad. Even sadder is that, with all the fashion trends that sweep through the tech world, none of them addresses this issue. People with high software skills often perceive this and shake their heads, knowing there is nothing they can do about it.

There are, of course, tiny groups of people where status is conferred as it should be: to the people who can conceive and then build their way to victory, rapidly and effectively. It's from those warriors of the abstract world that I learned the principles I discuss in my various books related to wartime software. There is more information and context on this and related subjects in my book on Software People.

Medicine as a Business: Medical Testing 2: Doing the Test

This is the pinnacle post of the series on medical testing, which starts here.

It's the pinnacle because I've finally climbed the mountain of scheduling, and I'm going to the radiation center for my test. Hooray! I'm at the top of the mountain! It will be easy after this, just getting the results and the bill.

I've been to the imaging center before. I'm well aware of their attempts to hide behind misleading signage:

Its nearly-secret location is several floors deep in the basement -- only those who really want to get there, and have the persistence to get there, make it.

I arrive more than the half hour early they requested, to allow plenty of time for the front-office staff to do their work. What's there to do? I've been there before; how can there possibly be anything about me they don't already know?

First of all, it's an iron-clad tradition to give entering patients a clip board full of paper that needs to be filled out, with lots of boxes to check. Have I been through this before? Yes. Every single time I visit. There's a simple explanation for this. Think back to cop shows you've seen where there's a witness or suspect the cops think might be lying or leaving something out. Or where there are two people who they think have concocted a story, and they interview them separately, trying to trip them up. The cop usually starts by saying, "I know you've been through this with my doughnut-eating colleague X, but I need you to take me through it again slowly, step by step." If it works for the cops, it should work even better for the medical staff, right? They carefully check every answer I give and cross-check it with all the previous answers I've given and analyze the differences. This way they can tell when a patient is lying, or when their memory is crashing because of whatever is wrong with them. Or simply to gauge the patient's intelligence and memory, to rank all the patients and do something wonderful with the results that only members of the doctors' cabal know about. AlI I know is that I have to waste time on each visit, only to have the staff glance at the first page, and file it.

I end up waiting for about an hour. Finally someone calls my name, and I follow her out of the waiting room through the trackless maze of hallways. After a bit of walking, I'm introduced to a person who, she tells me, will take my blood for testing.

The pre-MRI blood test

I've been through this before, and nothing bad happened. It just caused, as usual, another delay in starting the MRI, because they wait for the results of the blood test. Which results (of course) no one gives to me, the person blood was tested.

But there are a couple things to note about this practice.

• The main purpose of the test is to see if I'm likely to have an adverse reaction to the extremely safe contrast material that will be injected for some of the images.
  • The main concern is with the few subjects who have abnormal kidney function.
    
  • I had an MRI with contrast just 3 months prior. How likely is it that my renal function went south during the interval?
  • Doing the test for everyone is just not needed. See this, for example.
  • Doing the test for me was a waste of time and money.
  • In any case, it's clear that there are no standards that are followed here!
• Given that you're going to do a test, of course my blood needs to be drawn.
  • My blood was drawn by ... an RN.
    • Registered Nurses are amazing people with years of training, often including an undergraduate degree and more.
  • My blood could just as well have been drawn by a phlebotomist.
    • You can train to become a phlebotomist by having a high school diploma, taking a month-long full time course, and taking a certification test. Boom, you're done.
  • I don't think I need to comment about the different in cost.

After more waiting in a special waiting room, I'm finally called into the MRI room.

The MRI itself

The MRI nurse/technician was courteous and professional, like everyone else I encountered during the testing process. But the process was inexcusably bad, wasting time and money and reducing quality.

First, the nurse asked me where the tumor was that was to be imaged. This could have been good. It's classic checklist, the sort of thing you should do to avoid error. See this for details. Why wasn't it good here? She wasn't double-checking to make sure the computer-based instructions were correct -- she was asking to find out!

Imaging studies have been done on me of this area. Multiple times. Including at Mount SInai. Mount Sinai has incredibly detailed information about exactly where the tumor is, more accurate by far than anything I know. Nonetheless, I was the nurse's primary source of information about exactly where the pictures should be taken! She placed pieces of tape on my body indicating the limits, and those pieces of tape were her only source about where to take pictures.

Next, I laid down on the MRI bed. The nurse had me slide my shoulder into a little compartment, something which had never happened on any prior MRI. Clearly the dial on my paranoia control was set way too low, because I just vaguely thought, hmm, this is different, well she must know what she's doing. After adjusting me a couple times, I got to enjoy the usual loud noises in a confined space during which I was to remain rock-solid motionless; this pleasure went on for 20 minutes or so. Then I got rolled out.

The nurse tells me that the compartment my shoulder is in is a "camera." Unfortunately, the camera wasn't capturing all the area of the tumor, so she would have to use a different camera and do everything again. She gets out a thick, flexible plastic sheet and places it on my shoulder. I recognize it immediately, because it's exactly the same device that has been used on each MRI I've had, regardless of the imaging center that has done the work.

Amazing. Frightening. When I go a hair-cutting place, they record my visit and the choices and selections I made for getting a cut. When I go again, even if it's a different person, they'll ask something like "same as last time?" And then they'll normally do the checklist thing of confirming their understanding of what last time was. The point is: they know what I got last time. They recorded it. A hair-cut place. The only thing I can imagine is that the advanced technologies that hair salons use for keeping information about their customers haven't yet made it to the world of medicine. The plain fact was that Mount Sinai had either not recorded (probable) or not used (possible) key information about the image that was taken and how to take it. I would use the word "inexcusable" for this, but without a few choice 4-letter words, such a word would be far too mild to describe what went on here.

About 3 hours after I arrived, the MRI had been taken and I was free to go.

The MRI technology and equipment

This isn't part of my test specifically, but it's on my mind every time I encounter medical equipment. I'm a computer guy since forever (see this for details of my background), and I know too much about the technologies and the companies that are used in this equipment, and the hardware and software processes that create it.

The highly regulated companies using highly regulated processes to build this hardware and software are unique in technology. The regulation is supposed to protect the public and assure high quality. In fact what it does is assure that only a couple companies can supply the equipment in a government-protected monopoly, at absurdly high cost.

The net result of this is that specialized equipment and software are built to meet the regulations, even when COTS (commercial off-the-shelf) equipment is widely available to do the job with high quality and great performance at a fraction of the price. A prime example of this is the PACS (Picture Archiving and Control System) that all medical imaging systems include. This is basically a standard file storage system with a database that logs everything put in and enables access to images.

At the heart of the MRI is a body of software that could be built, maintained and enhanced at a tiny fraction of today's cost -- a 10X improvement is the minimum one could expect under a rational set of rules. Here is a detailed post with examples of the insanity and, just as important, a specific proposal for how to fix it.

Conclusion

I got the MRI. Nothing awful happened to me. I'm grateful that medical science/engineering has gotten to the point that something as truly amazing as an MRI is even possible. I can certainly imagine things being much worse than they were.

That being said, the opportunities for improvement on multiple fronts are HUGE. The patient's time and inconvenience could be greatly improved. The operational cost of performing the MRI could be considerably reduced, and the quality and consistency improved. Finally, the capital cost and rate of innovation of imaging machines in general could be HUGELY enhanced by drastic changes to the regulations controlling the design and manufacturing of the devices.

Even more good news: my saga was not yet over. I don't have the results yet! Wait until you read about what I went through to get them...

The Hierarchy of Software Skills

Outsiders generally have no knowledge of the hierarchy that exists among programmers in terms of skills. While everyone can understand that a programmer has had a job in a particular industry or company, and everyone can understand what place they have in the management hierarchy, the place that a programmer has in the skills hierarchy is invisible to most outsiders; moreover, it's not something they're interested in.

I've talked before about the difficulties of managing programmers, particularly when you're not a programmer yourself. I've talked about how strange this is, particularly in comparison to other fields like sports, music and journalism. My book on Software People addresses this subject. It turns out that understanding the skills hierarchy is important not just for understanding the differences among programmers, it has HUGE impact on innovation and the success of entrepreneurial companies.

The usual view

Most outsiders know there are software people, programmers. They know there are professors and that you can get degrees in Computer Science. They know that experience with software in banking is different than software in healthcare, but they're not sure how. They know there are levels of management. Many people even know some of the common buzzwords referring to languages and other things. But then the knowledge, such as it is, dribbles to nothing, other than things that apply to everyone: is s/he a "good" programmer? A team player? Etc.

When new software fashions emerge, many people know the basic buzzwords. Everyone at this point knows about AI, and even Machine Learning. Terms like Big Data are fading in frequency now, while Data Science is on the rise. Self-respecting managers want to be on top of things, and want to assure their organizations are investing in certified AI experts, and leading-edge Data Scientists -- otherwise, we'll lose out!

Sadly, most of these decision-makers don't bother to learn the first thing about what they're spending money on. Or if they do, it's just details that sound impressive, but don't matter. This explains in part the billions of dollars wasted on software fashion, while true innovators working far from the fashion runways make real-life, practical advances. I've illustrated this phenomenon with AI in healthcare.

The hierarchy

The skills hierarchy in software is generally not fashion-driven, though programmers can get caught up in fancy trends like anyone else. So I don't say that programmers are immune to fashion, but that the skills hierarchy itself changes very slowly.

A book could be written on this subject -- it deserves a book, both about the hierarchy itself, its evolution and how it impacts everyone involved. Actually, a chapter in a book has already been written about it, in this book. This is a long way of saying ... this blog post is just a brief introduction to an important subject that is broad and deep and consequential, but sadly largely ignored.

The first thing there is to understand is the incredible depth and breadth of highly specialized knowledge that is required to get anything meaningful done in the world of software. The start of the chapter may give you a flavor that we're talking about WAY more here than "learning a language."

A tremendous amount of specialized knowledge is required to work quickly and well in practical software environments. The knowledge begins with fluency in one of the many extensive, exacting and demanding computer languages. The language itself, however demanding it may be, is just the beginning. The associated “library” (depending on the times, it could be called the run-time library, the foundation class library, or various other things) has to be mastered as well, because it provides the functions that actually get a good deal of your work done. Just by comparing the bulk of documentation, you can easily deduce that the library is as large as the language itself. You can think of the library as being the set of idiomatic expressions, like “raining cats and dogs” that extend any natural language. Depending on the situation, you may need to know two or more complete languages and associated libraries in order to do anything. For example, if you are writing a database-oriented application, you may need to know the application language (e.g., Java), the language’s library (J2EE), the database’s query language (SQL), the database’s stored procedure language (e.g., PL/SQL), and the special library that connects the application language to the database (e.g., JDBC).

Languages and libraries, however elaborate and challenging to master, are just the raw materials for a real program. Knowledge of these is like a carpenter knowing how to cut and nail lumber – he can work with the raw materials and perform the low-level operations on it. Using raw materials to build a wall of a house requires knowledge of the “design pattern” of walls, and the typical way of assembling them, with the special way of framing windows and doors. While the saws, hammers and nails enable you to build a wide variety of wall-like structures, there is just one way to build them that accommodates standard size windows, doors, sheet rock, etc. There’s a special design pattern for when two walls meet to form a corner, and another one when one wall meets another in a “T.” Once you know all that, you know how to build frame walls, and you get to start over learning how to frame a roof, which itself has many details and variations. Design patterns like this exist in software, unfortunately not as standardized as in home construction, but still important and real. You really don’t want someone building a transaction system who is thinking about the issue for the first time. Just as a person building a wall for the first time might forget about windows or not know that walls are typically built on the ground and then raised up, a person building a transaction system for the first time might not think about audit trails or know that paying attention to row-level locking is a typical concern.

Unfortunately, once you know all that, you are still just at the beginning. If you know them well, you are as well-qualified as a college graduate with a major in English would be to lead a corporation’s response to a lawsuit – while it is true that all the legal documents are written in English and all legal proceedings are conducted in English, there are structures and procedures and patterns of speaking, writing and interaction intrinsic to legal English, and unless you conform to them, you and your case will be summarily dismissed. While the law-challenged English major may think in terms of making a date with the opponent and a judge and talking things through, there are formal complaints, responses, interrogatories, motions for discovery, rules of evidence and procedure and on and on and on.

The chapter goes on to cover things like all the tools you have to know to write, build, test and run your program, not to mention find and fix the bugs.

Once you've done all that, you've mastered a single set of myriad tools that are out there to build and test software, in a single application environment. When you move on to the second set of such tools, you begin to appreciate just how different those different environments are -- at least as different as classical Chinese is
from modern American English (this blog post), from medieval Celtic runes.

Having learned a couple of these amazingly different language systems and the wildly different worlds in which they "live," you're still pretty low on the skills hierarchy.

One of the dimensions "up" from "simple" ability to work successfully within one of these language systems is the ability to find patterns in extensive bodies of code and invent ways to leverage the patterns to transform the code in valuable ways. One frequent transformation takes knowledge out of code and puts it into editable data of some kind, with the net result that there's less code and that changes can be made more quickly and safely by changing the data instead of the code.

A closely related dimension to pattern-finding is working with the tools. If there's a problem with one of the tools you use, can you fix it? Can you build a better tool?

This may not sound like a big deal, but there is a HUGE difference between a programmer who is able to write an application that uses a DBMS (database) and a programmer who is able to create a DBMS. Among programmers who work with the internals of a DBMS in any way there is an extensive hierarchy. At the bottom of that hierarchy are people who don't change source code but are nonetheless expert in the use of it, running through various steps up to those who can make strategic changes or even create a new DBMS. This is a dramatic a hierarchy as starting with people who can drive a car, onto car mechanics who can tune up an engine and going up to engineers/scientists who can create a whole new kind of engine.

Within tools, there's even a broad range. For example, someone who can work with a text editor tool is normally worlds away from someone who can build a compiler or a code generator. As different as calculus is from algebra.

This brings us to a whole new dimension. There are people who are so skilled that, not only can they master the available tools to build effective, working software quickly, they can conceive, create and build tool-building engines to automate the process of using the tools to build software. I won't try to describe this here, but it's a kind of level-shift, much like algebra is a level above arithmetic, fully encompassing it while going way beyond it -- while still "boiling down" to numbers in the end.

With tools, we begin to dip our toes into the deep waters of "systems software." Things like the operating system or networking software or storage software or machine/device control software, each of which constitutes a whole world in itself. The people who are proficient in any of these things are truly rare, and near the top of the skills hierarchy.

Just to take one small example, in the world of the internet there are levels of software and skills that are a little exposed to people who are administrators, but like with most of these things, the bulk of the iceberg is hidden from the surface. Something that everyone uses for example is DNS, which is the software that translates the URL that appears in the top of your browser to the low-level IP address that identifies the server which will deliver content to you. DNS operates in a way that is mysterious to the vast, vast majority of software people, even those who are deeply skilled in other areas. You can be skilled in DNS and still know little about the devices and software that get the moving job done, the TCP/IP protocol engines (and extensive supporting software) that moves data through the network.

Hey, you might say, isn't that esoteric stuff only relevant to the nerds who keep things running, like of like the maintenance people in the basement of our office building? Yup, that's typical thinking. There are maintenance people, sure. And there are people who understanding things so well that they can create a new heating and cooling system for your building that is silent, effective, and uses a fraction of the power of the old one.

Conclusion

There's a great more to be said about the multi-dimensional, multi-domain software skills hierarchy than I've said. I've only scratched the surface. But I hope you've gotten as a take-away that the depth of skills in software is much greater than most outsiders, and even most normally-skilled programmers, imagine. Understanding this skills hierarchy is essential to attracting and retaining the very best software people, and unleashing their considerable talents to transform software-using businesses in dramatic ways.

 

Medicine as a Business: Medical Testing 1: Scheduling

Before you get a medical test, it has to be scheduled, right? Just like getting a reservation at a restaurant, both you and the place you're going have to agree on a time. Here's the real-life story of my recent MRI, starting with scheduling it. The point of this post isn't to whine about what happened to me, but to illustrate deep, widespread problems in the medical business by means of a concrete example.

Background

First some background. My doctor was at Mount Sinai. While he was treating me, I had several MRI's at Mount Sinai. Then he moved to another major hospital system in the NYC area, Northwell Health. Both are excellent places with modern, up-to-date systems. I told the story here of the amazing breakthrough in EMR electronic interchange that allowed me to get the MRI's that had been taken at Mount Sinai and get them to the doctors at Northwell so they could do their jobs. It was true breakthrough technology, since the interchange was almost completely electronic, with the truly minor annoyance of a dozen phone calls, a couple paper forms and faxes, and a couple packages carried by hand. I know it's hard to believe that such giant systems could be so modern and electronic, which is why I give all the facts and associated proof here.

I felt my tumor growing again. I quickly got an MRI that showed that, unfortunately, I was right. My doctor, the MRI and the follow-on to it were top-notch. The subsequent billing events are an object lesson in how the business of medicine can be improved, see the posts here for details.

My doctor, who had treated my tumor and greatly reduced it with chemotherapy, now felt it was small enough that radiation was the best approach for treatment. He recommended I go to a radiation specialist, who happened to work at Mount Sinai, my doctor's former employer, and where I had originally been treated. Naturally, I took his recommendation. I got radiated. 30 times.

It is standard practice to check the results of radiation 3 months after the treatment. In my case, that meant getting an MRI.

Scheduling the MRI

Here's what I went through to get the appointment. Nothing horrible here; I'm pleased that something so effective as MRI technology is available and that I was able to get it. But the cost, time and convenience all reflect a broken business model. Fixing the model isn't hard in principle, but would require serious change. This is an example of how it works today.

On my final visit with my radiation doctor post-radiation, he told me about the follow-up MRI I should have in 3 months. He told me his office would contact me and get it scheduled "shortly."

Nothing happened. I waited for the 3 months to pass. No one contacted me. Fortunately, I kept track of the time, so I got on it myself.

I started with my radiation doctor's office at Mount Sinai. Sure, they said, you can use the Northwell imaging center to get your MRI. It's convenient for you, so why not? We'll set it up for you.

I wait a few days. Nothing. I call again. Oh, sorry, we'll set it up. I wait a few days. Nothing. I tried again. Nothing. Finally I called the Northwell imaging center and explained the situation. Sure, no problem let's make the appointment. What about the pre-auth? Oh, don't worry, we'll take care of it. You've got all my information in your system from last time, right? Yes. Don't worry.

Two days before the scheduled MRI, I get a call. Hey, about this MRI you're scheduled for, there is no pre-auth for it, so we're going to have to cancel the appointment, there's not enough time to get one. Great. I don't bother complaining, what good would it do?

I realized that I had been totally wrong in my strategy about this. This medical stuff must be making me lose my mind. Or I'm getting colossally stupid. What was I thinking?? I was choosing where to get my MRI done based on what was best for me. WHAT an IDIOT I am!! I know that the smart thing to do is always do what's best and easiest for the medical system. Duhhhhhh!!

Having returned to sanity, I called the Mount Sinai radiation doctor's office. They said they'd set it up. The whole thing, pre-auth and appointment. A couple days pass. I call again. Same thing. A couple more days pass. I call again. Oh, sorry, excuse excuse excuse. I said, no problem can you just please send me the doctor's order? After more shenanigans, I get the order. Giving in completely to the world as it is, I call the Mount Sinai imaging center, where I've been a few times in the past, and make the appointment, making sure that they've got the doctor's order.

Conclusion

I got the appointment. That's the good news. I know that others have had it far worse. But I also know that this peculiar state of affairs, where there's no equivalent of, say, the Open Table scheduling service for making MRI appointments, is a HUGE time-waster for everyone involved. Do you want a job doing what the people I talked with on the phone sort of try to do? Would you want to cope with constant hassle and frustration?

My insurance company knows the treatments I've had. All the information is in the EMR's of the two major systems I've used. It wouldn't be hard to know that getting an MRI at my stage of treatment is something to authorize. The process could be completely automatic. And I could maybe even have made my MRI decision based on my needs rather than the peculiarities of a deeply flawed, broken system.

Next step: getting the MRI.

Getting Results from ML and AI 4: Healthcare Examples

While the success patterns laid out in the prior posts in this series may seem clear in the abstract, applying them in practice can be hard, because nearly everyone who thinks or talks about AI (these sets over overlap very little, sadly) takes a different approach.

http://www.blackliszt.com/2018/03/getting-results-from-ml-and-ai-1.html

http://www.blackliszt.com/2018/04/getting-results-from-ml-and-ai-2.html

http://www.blackliszt.com/2018/04/getting-results-from-ml-and-ai-3-closed-loop.html

So here are a couple examples in healthcare to illustrate the principles.

The spectrum of problems

One useful way of understanding the winning patterns in AI is to understand the range of problems to which it may be applied. It's not difficult to arrange the problems as a spectrum. While there are many ways to characterize the spectrum -- here's a prior attempt of mine to characterize the spectrum in healthcare -- perhaps it's easiest to understand it in terms of the typical salary of the person whose work is being replaced or augmented by the AI technology.

At one end of the spectrum are low-paid people performing relatively mundane, repetitive tasks. These people have relatively little education and minimal certifications compared to those higher on the spectrum. Think back-office clerical staff.

At the other end of the spectrum are highly paid, educated and certified people performing what are understood to be highly skilled and consequential tasks. Think doctors.

The very name "artificial intelligence" tells you at which end of the spectrum AI is normally applied. The popular image, supported by the marketing of the relevant vendors, is that AI is amazingly smart, smarter than the smartest person in the room, just like the way that IBM's Deep Blue (predecessor of Watson) beat the human world champion playing "Jeopardy," and beat the then-reigning world champion playing chess.

To put it plainly, while these achievements of Deep Blue were amazing, they were victories playing games. They were not victories "playing" in the real world. Games are 100% artificial. The data is 100% clear and unambiguous. There are no giant seas of uncertainty, ignorance or unknowability -- unlike the real world, which is chock full of them. Nonetheless, IBM and whole piles of people who self-identify as being "smart," and are widely perceived as being smart, jumped on the "AI does what smart people do" bandwagon.

This was and is incredibly stupid and 100% bone-headed wrong. Not only is it bone-headed in terms of intelligent application of AI, it violates simple common sense. If you knew a talented high school kid who played a mean game of chess, would you drop them into a hospital and give them a white coat? Even after the kid claimed to have read and understood all the medical literature?

The smart thing to do is to apply AI to tasks that are relatively simple for humans, at the "low" end of the spectrum, and see if you can get a win. If you can make it work, by all means graduate to the next more complicated thing. It turns out that replacing/augmenting human tasks that are mundane, "simple" and repetitive is amazingly challenging! Yes, even for super-advanced AI!

IBM Watson in healthcare

I know I've made some strong statements here. It's little old me vs. a multi-billion dollar effort by that world-wide leader in AI technology, IBM. Who's going to win that one? Turns out, it's easy. See this, for example.

IBM claims to get many billions of dollars in revenue from Watson. But everything about getting it to do what doctors can do has proven to be vastly more challenging than anyone thought, and its advice rarely makes any difference, even when it's not wrong. And this, after years of work by top doctors at top institutions doing their best to help IBM "train" it!

Here is a summary of the situation:

Let's note: the Watson effort is built on the most famous "smart computer" technology ever, funded to the tune of billions of dollars, with technology acquisitions and expert help from all corners. The "disappointing" outcomes are not the result of having picked the wrong algorithm or something easily fixed. The failures are a direct result of not following the success patterns described in the earlier posts of this series, combined with applying AI to the wrong end of the job-complexity spectrum described earlier in this post.

Olive in healthcare

If IBM can't manage to pull off a win in healthcare, after years of applying the most advanced AI and spending billions of dollars with the best help that money can buy, I guess it's impossible, right?

Wrong. IBM made a fatal strategic mistake. They used AI to attack the hardest problem of all, at the wrong end of the complexity spectrum. Has anyone done this the right way? Applied modern AI and related automation technology to the right end of the complexity spectrum? Yes! Olive has!

Olive is making a positive difference today (please note the use of the present tense here) in many hospital systems by reducing costs, reducing error rates and getting patient information where it needs to go more quickly and efficiently, saving time and aggravation of medical workers along the way. The money and time it saves in the back office may not seem glamorous or "leading edge," but every minute and dollar it saves is time and money that can go to making patients healthier, instead of disappearing down the "overhead" sink-hole.

Getting a pre-auth for a key procedure so it can be performed. Submitting all the right information so a claim can be paid. Getting information to pharmacies so patients can get the life-saving drugs they need. Getting all the information from incompatible, hard-to-navigate EMR's so doctors have all the information they need to give patients the most appropriate care. These absolutely essential tasks are largely performed in windowless rooms far removed from patient settings by people who work hard at largely thankless jobs that aren't well-paid -- but are absolutely essential to providing care to patients. And they're harder than they look! Anyone who's spend any time with a modern EMR can't help but think of the endless meetings attended by skilled professionals at the software vendor trying to find yet new ways to confuse and confound the users. And anyone who has dealt with getting insurance companies the information they demand can't help but think of a cranky three-year-old who lost emotional maturity when he grew up. Bottom line: this stuff is hard!

Olive gets it done, using an exotic collection of works-today technology, silently learning from the people who do the work today. And gets it done without having to upgrade or replace existing computer systems. Amazing.

The founders at Olive are doing AI the right way, attacking the right end of the complexity spectrum. They follow most of the rest of the success patterns laid out in the prior posts of this series, above all attention to data and detail, working from the bottom up in terms of algorithmic complexity, and using closed loop. It's a hard problem, and it was hard work to get it done. But they did it, without the massive armada IBM fruitlessly assembled.

Disclosure: Olive is an investment of Oak HC/FT, the venture firm at which I'm tech partner.

 

Medicine as a Business: Billing 4: What's Wrong

Making fun of medical billing, as I have done with gusto in the previous series of posts, is way too easy. Everyone involved knows it's a problem. But it's not getting better. Money that should be spent helping people be healthy or get healthy is instead being spent in completely unproductive ways, annoying and harassing everyone along the way.

It's amazing how many issues are illustrated by just two bills from one healthcare system. Sadly, this is not an isolated example: it illustrates business-as-usual in healthcare billing.

I make no claims to be comprehensive, but fixing the medical billing issues I've illustrated would be plenty!

Here are the prior posts:

http://www.blackliszt.com/2018/07/medicine-as-a-business-billing-overview.html

ttp://www.blackliszt.com/2018/07/medicine-as-a-business-billing-1.html

http://www.blackliszt.com/2018/07/medicine-as-a-business-billing-2.html

http://www.blackliszt.com/2018/07/medicine-as-a-business-billing-3-insurance.html

Here are some of the highlights:

• The first obvious issue that makes medical billing different from the rest of the world is that there are no price lists. You have no idea what you will have to pay. When you sit down at a restaurant, you get a menu with prices. Not in the medical office.
  
• The next glaringly obvious issue: unlike most other services you can think of, the bill was not presented at check-out time! Fixing this would fix a whole host of problems!
• A single health network has multiple billing systems, each amazingly different from the others, each with its own staff, software, costs, etc. It doesn't have to be this way.
• The bills can arrive months after the service was rendered. What service organization you interact with lets billing slide for months? It sure sends a message that they're not serious about collecting.
• When the bill arrives, the address that it comes from and the place to which you send the payment can have totally different names and places from the organization that served you.
• When you get a bill, you sort of expect to know exactly what the bill is for: what service was rendered, when it was rendered, where and by whom it was rendered. Without those key facts, how can you be sure about the bill? Both bills were a strike-out on this subject. Why is it hard to provide this simple, common-sense information?
• For many people, receiving bills and paying electronically is convenient. For many organizations, sending bills and receiving payment electronically is more efficient, and encouraged. As I've illustrated in these bills, the health system's electronic payment is like a programming 1.01 course project -- one that failed.
  • They didn't even try to have e-bills.
    
  • E-payment was offered on the paper bills, but the process was amazingly bad and error-prone.
  • In the end, e-payment simply did not work. Period. And of course, there was no electronic way to get help or even register a problem!
• The second the insurance company is involved, things get real baroque in the bills, with confusing additional information that, in the end, makes no difference to the patient. And without even the name of the insurance company correct.
  

Wow-za! Not that any self-respecting healthcare system manager will spend money on fixing billing instead of promoting innovation, AI and ML anytime soon! Why, if they stooped to merely making things better for patients while reducing costs, they would rapidly lose prestige among their peers in the industry!

 

Medicine as a Business: Billing 3: Insurance

In the prior post in this series, I dove into detail of the bill I got from a doctor visit. The doctor was wonderful. On the other hand, the billing amounts to a deep well of opportunity for innovation, innovation of the kind that doesn't involve blockchain, machine learning, AI or even Big Data! Merely the kind of innovation that reduces costs and makes things better for everyone. That's all.

In this post, we get to dive into a treacherous bay in the sea of healthcare billing the likes of which can be found nowhere else.

The doctor visit bill

Again, here’s the bill I got for a visit with the doctor:

The bill I got for the MRI was pretty discrete about the fact that an insurance company was involved. Here's what they said:

That's all. It flies by so fast, even assuming you read it, most people won't notice that according to the bill (I'm not sure I believe it), they were paid just weeks after the service was rendered. No talk about what was billed, who was billed and what they paid. I'm just a patient, I have no "need to know." What I do need to know is that I owe them $85, and I'd better pay up.

But this is billing for a doctor visit. Different department. Different software. Different bills. Different payment mechanisms. This bill makes clear that the insurance company is a major player here. Here's the first part:

Unlike the MRI bill, this bill tries to tell who was billed how much and for what. Who was billed? "BCBS OUT OF STATE." My insurance company is Anthem. Yes, I know the industry lingo that BCBS means Blue Cross Blue Shield, but the name of the insurer is Anthem. Sorry.

What was the bill for? This:

Remember, we're dealing with a HUGE IT department here, stuffed to the gills with experienced professionals. But I guess that looking at the bills and making sure they make sense is low on the priority list. Do you know what a "comple" is? Because I spend WAY too much time on this stuff, I do know what it means. It's truncated from "complexity."

This is our first glimmer of a fierce, take-no-prisoners war that's actually going on beneath the surface of these innocuous-seeming bills. What presents itself as a bill is in reality a communique from a war zone. The "high complexity," which is a translation of the ICD-10 code that Northwell put in their claim that they sent to the "BCBS OUT OF STATE" is their rocket launched over the trenches to the Anthem side to try to get Anthem to pay more for the 20 minutes the doctor spent with me telling me what I could have read from the radiology report, if the medical system had stooped to giving me the results of the reading of my images, paid for by me. But those trenches are already dug deep, and aren't going to change because a mild breeze of common sense wafts by.

Because of inserting the code for "high complexity" in the claim, Northwell is trying to get the enemy ... oops, sorry, the honorable insurance company ... to pay 641.00 for that visit.

An inquiring mind may wonder, what exactly does Northwell want, given that they're asking for:

Do they want Euro? Peruvian Peso's? Bitcoin? I suspect they want plain old US Dollars, but unlike any other bill you've ever seen, they can't be bothered to get it right.

(You may wonder why I trouble my pretty little head about such "trivial" issues. Simple. I wrote software for 30 years, and led the effort for credit card billing software that now processes half a billion accounts world-wide. I know software in general and billing software in particular. In the same way that an editor has trouble taking seriously a writer who doesn't bother to spell correctly, and that a conductor has trouble taking a candidate musician seriously who flubs lots of the notes, there is good reason to believe that a software group that lets obvious flaws like these appear on patient bills has far deeper problems, and that the "underground" parts of their software are probably nightmares. Which all the evidence shows that they are.)

Now let's shift to the right column. Here's what we see:

More than 2 months after my visit, Northwell claims that "BLUE SHIELD," not BCBS and not Anthem, paid them 232.89 Ether, or whatever currency they ended up agreeing to. So the response to the HIGHCOMPLE rocket was a grenade that, when it exploded, screamed "I'll pay you 36% of what your rocket demanded. BOOOM!!"

Northwell sadly reports to me how badly they lost the battle (they're used to losing), and cleverly inserts a "OK, we lost. Fine." line item of 358.11.

What the &*()&*() is that about? How did they ever arrive at that amount??!

This leads to our next juicy topic...

Insurance Co-pays

Medical systems have a myriad of ways of putting it. Some of them just say something like they did for the MRI bill: "This is what you owe. Really. Pay it. It's your responsibility." Others, like this branch of Northwell, handle it totally differently. They make a pathetic, flawed attempt to do the standard accounting/billing thing of "This is what you started owing, this is what you paid, and this is what's left. Please pay it." Except you haven't paid for a thing! The insurance company somehow decided to pay 36% of the bill, and then Northwell somehow decided to subtract an "adjustment," magically leaving the nice, round amount of 50.00 Yen, Bitcoin or whatever to be paid.

Just to be helpful, they put a line item in there "Patient Payments    0.00." Duhhh. Like, you haven't billed me, man. This is the first bill you've sent me for this, a mere 3 months after my 20 minute visit. Of course I haven't paid. And it's in bold, no less. I guess I'm supposed to feel guilty? Or perhaps just hurry up and pay (via the doesn't-work online payment website) the 50.00?

This whole thing is a fake, of course. As everyone who's dealt with insurance knows, way back around the time the Pope divided the New World between the Portuguese and the Spanish (which is why they speak sort-of Portuguese in Brazil and sort-of Spanish in the rest of South American), a group of genius-level experts, the kind of people who decide important things so that the world will work as it should, got together and invented the notion of "co-pay."

"Co-pay" is one of those ideas that only true experts, people who see farther and deeper than us mere mortals can see, could come up with. The core idea is to give patients an incentive to care about the cost of their health care. If they have to pay something every time they "consume" health care, they'll exercise caution and not use too much of it! That's co-pay. Sheer genius! Even better, we'll make the co-pay something that they owe to their doctor. Genius again -- it's the doctor who's providing services, so of course it's the doctor who should be paid. Insurance companies are hated enough as it is. By shoving the burden of billing and collecting onto the medical systems, maybe they can see what it feels like to be disliked. And get collectors involved. And see what substantial levels of double-digit payment defaults look like on the financials. It's all a good thing because we're influencing patients to be careful about what medical services they consume, and from whom! I really don't understand how this kind of galactic-level genius can sleep at night, quivering from the excitement and self-regard of being responsible for such a transformative idea.

Now back to reality. Do co-pays "work?" I mean, do they influence patient behavior in the way intended? No, of course not. But now they're deeply dug into the trenches separating the payer and provider armies, and extricating them will take a real act of courage.

In this example, suppose Northwell decided to bill 591 instead of 641. Suppose (humor me here) that BLUE SHIELD paid the same lousy 232.89. Suppose Northwell made the same 358.11 ADJUSTMENT. Net result: Bill paid. PAID IN FULL!!

Now was that really so hard? Of course, there are some awful consequences of this. A truce would have to be called on a major part of the front. There are jobs and important bodies of software at stake here, on both sides of the war. And support people. And collection agencies. What would they do with all their time?

Probably the worst consequence would be patient behavior -- patients would start consuming healthcare services like crazy because there's no 50.00 co-pay! Not. The second people respond with the same amount of serotonin to the phrase "don't worry, this giant needle won't hurt a bit, just a pinch" as they do to the question "what kind of massage oil would you like me to use," we'll know we have a problem. Until then, I think we're OK.

Conclusion

This post was supposed to focus on the insurance aspect of medical billing, using an example bill. The bill I used was a typical, benign example; not the kind of extreme example you'd expect when reading something that dives into a problem. I said nothing about pre-auth, denials, deductibles, insurance company coverage notices, or any of the other all-too-common joys of the medical business. That was the point! The transaction described here, with the on-the-surface messes and below-the-surface nightmares are business-as-usual!! And that's sad, for everyone concerned -- which includes pretty much everyone, except those of us who are looking at a small patch of grass from the side of the grass where the roots are.

Medicine as a Business: Billing 2

In the prior post in this series, I presented a couple bills and dove into detail for one of them. Now it's time to see what pleasures there are in the second bill.

The doctor visit bill

Here’s the bill I got for a visit with the doctor who ordered the procedure:

This bill is a bit of a relief compared to the one for the MRI. While the return address (some PO Box in New York) and the address to which payment should be sent (NSLIJ at a PO Box in New York) are opaque and confusing, at least the box in the middle of the page names the doctor I saw and gives the date of the visit. I know what I'm being billed for: a visit with this doctor on that date. That's good!

Let's look a little more closely.

First, there's something interesting about the date. The visit with the doctor was Dec 11. Now look at the statement date: 3/13/18. Yes, that's right: the statement was dated a full 3 months after the visit! Wow. Northwell has clearly optimized their systems to march everything through so they can bill and receive payment promptly, right? Sadly, no.

Second, I'd like to point out an important issue: paper vs. electronic. With all the noise, billions of dollars of federal subsidies, and the obvious fact that electronic is better than paper, you would think that a major NYC hospital system would be entirely electronic. You would be wrong. Here is a post about this. But about this bill:

I got the paper bill in the mail. They could have gotten my email from me at any time, but didn't.

There is no opportunity to sign up for paperless billing, unlike even notoriously backwards bureaucracies like utility and phone companies, which constantly harass you to sign up.

Two things on the bill are highlighted to make them stand out: The amount to pay and the URL to pay it:

I think it's fair to say they're trying to get me to pay online. So I tried. But what a pain! Copying that looooooong URL without error isn't trivial. Then once I entered it correctly, here's where I landed:

They re-directed me: that long string I copied could have been tiny, because it wasn't actually the place they wanted me to go!

But the fun has just started. Now I have to fill out the form:

Once I filled it out, here is the result (with my DOB cut off for privacy):

Fail!

Dutiful person that I am, I got out my ancient check book, revved up my hand-printing skills, and ... yes, put the check in the mail.

There is more joy and fulfillment to be found in this simple-seeming one-page bill, but that's enough for now. For the next installment, we can look forward to some only-in-healthcare wonders of billing.

 

Medicine as a Business: Billing 1

If you were to argue that my fascination with medical billing, including the endless-seeming minutia of it, is somewhat, well unusual, I could not dispute it. "Guilty as charged" (or billed?) would be my only response.

I've learned that the obvious, annoying uniqueness of paying for medical services, different than pretty much anything else in our lives, is the tip of an iceberg of financial pain, metastasizing bureaucracies and festering IT dysfunction for the institutions and people involved in it. See this for some context.

This is a strong, broad statement. Let's dive into some real-life, on-the-ground facts and see what we see.

Background

I’ve been under the care of an excellent doctor, now working at Northwell Health, for a super-rare, long-term condition. I called his office because there was evidence that the condition was worsening. His similarly excellent NP buddy responded to my phone call report by authorizing an MRI of the area in question, and making an appointment for me to see the doctor 4 days later. I truly appreciate how exceptional this accessible and responsive behavior was.

I went to the MRI and then had a consult with my doctor. Sure enough, the evidence I noticed was confirmed by the MRI, and he immediately started the appropriate treatment.

From a medical and scheduling point of view, this experience would be hard to beat. Anywhere, under any system.

This post, however, is about the hospital system billing, with the heavy involvement of my insurer. Every step of the process was chock-full of stupidity, waste, friction and “opportunities for improvement.”

The bills

Here’s a bill I got for the MRI:

 

Here’s a bill I got for a visit with the doctor who ordered the procedure:

 

Take a quick look at the two bills, each issued from the same health system for visits which took place in Manhattan a few days apart. Could they be more different?

Right from the get-go, we know we have a problem. Totally different systems producing bills that are radically different, involving different systems for generation, tracking, and collection. Wow.

Totally different systems

Even though both bills clearly have the Northwell Health logo clearly displayed at both the top and bottom of the page, nearly everything else about them is different. The return address is different, the address to which you send the bill is different, even the little box where the payment is defined is different. It blew me away that even the web page for electronic payment was different!

This means different software systems for generating the bills and collecting the payments, among other things.

The MRI bill

Let's look at that MRI bill. It's written in the form of a letter, which is interesting. It was amazingly prompt: the "service" was "rendered" on Dec 7, while the bill was dated Jan 13, only 5 weeks later. In terms of medical billing, lightning fast!

Who is the bill from? Clearly, Northwell Health. But that doesn't help, because Northwell is a giant system. The return address of the bill says "Lenox Hill Hospital," with a PO Box in New Hyde Park, NY. That's nice, except that there is no place called "Lenox Hill Hospital" in New Hyde Park -- though there is a huge complex that is part of Northwell called "Long Island Jewish Medical Center." So where is "Lenox Hill Hospital?" That's easy. It's a big place on E 77th St in Manhattan. The addresses they give aren't helpful. What about the letter itself. Maybe there's a hint there?

Now we're getting somewhere -- the letter clearly states when and where the service was performed:

Excellent, the service was performed at Lenox Hill, the place on the Upper East side of Manhattan. The trouble is, I didn't go there for my MRI!. Not there, and not LIJ. I went to an imaging center run by Northwell in the Chelsea neighborhood of Manhattan.

Maybe they're talking about something else than my MRI. Let's check out that bill again, and see for sure that it's a bill for my MRI. Uh-oh. Trouble. Nowhere is MRI mentioned in the bill, or medical imaging of any kind! Look again at the bill and the snip of it above. The words used are "services rendered." The wonderful people at Northwell go to the trouble of putting a Spanish-language version of the bill on the other side of the paper, but they can't be bothered to tell me what service was provided or where it was provided. Actually, it's worse. They clearly state in the snip above that the service was provided at Lenox Hill Hospital. Which it was not.

Now try to pay the bill. Hah! They give a website. Let's go there.

I have to guess that they want me to pick "hospital," so I do and pick the one on the bill, Lenox Hill. I hit "submit." Here's the result:

Software is wonderful, isn't it?? Saves trouble, filling out paper, stamps, and all that old-fashioned stuff. Just go on-line and pay. Except when the software doesn't work.

That's plenty for a single blog post. We'll have some fun with the other bill next.

 

Medicine as a Business: Billing Overview

If you break your arm, broken arms are suddenly at the top of your list of least favorite subjects. But after the arm gets better, the billing process for the arm-fixing services is probably pretty high on the list.

Medical billing is something too many of us are way more familiar with than we'd like, but nonetheless will serve as an excellent first subject for this series on the business of medicine.

Let’s start by putting medical system billing in perspective.

Going to a hospital or doctor to get a service is in most ways like going to any place and getting a service.

• You’re hungry, you go to a restaurant and get a meal.
• You’re shaggy, you go to a barber or salon and get a cut.
• You’re bored, you go to a movie or show and get entertained.
• You’re bored and hungry, you go a bar with a show
• You need a book or chair or laundry detergent, you go to the appropriate store.
• For most of the above, you just go. Sometimes, when the service is popular, it pays to make a reservation. You call or do it online, and your space at the place is assured.

A pattern is clear here. When you need something, you go to the place that has, does or serves that thing and you get it. Pretty simple, and universal. Same thing with medical issues!

• You broke your leg, you go to the hospital and get it fixed.
• Your skin gets weird and painful, you go to the doctor and get it checked out.

But there’s something really important that I’ve left out of the above list; I suspect you know what it is. Let’s see if you’ve guessed what it is. I’ll wait and give you some time. Click … click … click … OK, time’s up! Did you get it? You probably did, but just in case, here’s the answer:

The billing and payment are totally different! And the scheduling/reservations are often a nightmare!

Here's how it works for nearly everything:

• For the movie, the price is posted, you pay when you enter.
• For the restaurant, salon, and retail store, you choose from a menu or list or wander around picking things, and pay when you leave based on the services/goods you got.
• For the bar with show, you pay the cover charge when you enter, and your bar tab when you’re done drinking/eating.
• For all of these, you can pay with cash or card. The card could be a credit card, which enables you to pay later or make payments.

Now, let’s look at the big exception: medical billing.

• If it’s an emergency, you might go to the ER and wait for hours.
• If not, making that “reservation” may require a “pre-auth” and a variety of other things that are often painful, and sometimes denied.
• You don’t pay when you enter.
• There are no posted prices, unlike the salon or movie. No menu. No price tags.
• Some stores let you special order things, sometimes one-of-a-kind. Before getting it, you get a price and make a payment arrangement. Not in medicine!
  
• There is no “check-out register” where you find out what the total comes to.
• Unlike a restaurant, where no one has any idea what food and services you’ll receive when you walk in, there is no bill at the end.
• In medicine, it’s common to leave after services have been rendered, and eventually a flurry of bills may arrive from different places and/or notices from insurance wanting information or telling you what’s been “covered” and what they’ll pay.

Go. Get services. Leave. They're all the same. Schedule it and pay for it: it's totally different in medicine!

• “I broke my leg” is like going into a car repair shop knowing you want it fixed, but not being able to do it yourself or knowing what it will cost. It’s your only car, so it’s got to get fixed.
• The car place tells you about what it will cost and you agree to pay when done. Not so at the leg fixing place!
  
• Bills from the ER and a couple doctors could show up over the next couple of months, with lots of fine print. The insurance company probably joins the fun with multiple pages.There's loads of stuff that has no counterpart in the real world of product and service buying.
  
• “I have funny skin” is like going into a salon knowing your hair and makeup just doesn't work for you, and you don't know what to do. You need a new "look" and need help and advice. After a discussion you might look at some pictures and try a couple things out. A specialist may need to be called over.
  
  • You get your new look and pay for the products and services on the way out of the salon. Not so at the skin fixing place!
  • There might be "co-pays" that you owe someone, or maybe not if they're "in network." Letters about any "coverage" some family member may or may not have. Not to mention surprise bills. Nothing that any salon of any kind would ever try if they wanted to keep their customers.
    

All of this is bad. Really bad. It's terrible for patients, terrible for medical people and institutions, and no fun at all for insurance companies. Believe it or not, the behind-the-scenes, underlying reality is even worse! Which is part of why it doesn't get better.

In the later posts in this series, I will go into specifics with real-life examples. In each case, while there are clearly systemic barriers to improvement, there is a clear path to improvement.

Among the issues I'll cover are these:

• The delay between the service and the bill
• Bills from different places
• The difference between provider billing and insurer paying
• What service was provided, where and by whom?
• The challenge of paying the bill

This is going to be fun!

 

Medicine as a Business: Overview

Medicine is a business. It’s a big business. But it’s like no other business, often in ways that are not good for anyone involved.

The good news is that there are many amazing people who work hard, have compassion, and do their best to make things better for their patients. More good news is that dedication and hard work have created drugs and procedures that can cure or alleviate awful conditions, conditions that resulted in pain, suffering and premature death in the past. We are truly blessed.

It’s not news to say that the business of medicine could be greatly improved. When this subject is discussed, it’s usually done in terms of rocks and hard places, or irresistible forces and immovable objects. Then hands are waved, thrown up in the air, and nothing but desperate hopes are expressed.

I believe that the business of medicine can be improved. Greatly. From the outside, it’s impossible. But when I look at the computers and software systems and procedures, I find a particularly ripe example of something I’ve seen many times: decrepit software running on outrageously expensive hardware, surrounded by ineffective business processes, and run by experienced, well-meaning people trapped on a steam-age island in a world that has long-since gone electronic. While there is no magic wand to fix everything in a flash, the technology is available to make dramatic improvements. Today.

To a broadly experienced software person, the solutions are obvious – once the exact technical flaws have been precisely identified. Much of this series will be devoted to spelling out those flaws, not to wallow in them, but to put cross-hairs on them so they can be fixed!

As context for the rest of the posts in this series, let me observe that the issues I will raise are waaaay too "little" to engage the big minds and the great powers in the medical industry. These prestigious and powerful people much prefer to give talks, issue press releases and hang banners about how they and their institutions are "innovative," how they are leading the industry in their use of "cognitive computing" or "artificial intelligence." They hold and/or attend conferences on the subject, and pour money in those general directions, each of which is sure to deliver real results real soon now -- why, the trials are just so promising!!!

That's not my perspective, to put it mildly. See this for some plain words and facts on the subject of fancy-pants innovation in healthcare. And see this for facts and logic that help explain why this subject gets almost no interest or attention from the higher-ups in healthcare management.

Getting Results from ML and AI: 3, Closed Loop

Getting practical, real-world results with ML and AI involves more than getting data, doing calculations, and building models. You can do everything else right, but if you don’t get this last step right, you’ll join the rapidly growing ranks of people who may have tried hard, but ended up accomplishing little in real-world terms.

The first part of this series laid out the issues and concentrated on the indispensable foundation of success, the data. The second part of this series dove into the analytic methods that can be used to generate value, with some advice about how to sequence the methods used.

In this post, we’ll concentrate on the relationship between the real world and the back office analytical work. What we’ll find is that an integrated, collaborative, closed-loop relationship between measuring, calculating and real world application is the path to success.

Loops, open and closed

Whether you run an operation closed loop or open loop is one of those absolutely key concepts, highly correlated with success, that is rarely discussed. Generation after generation discovers it by itself, or not, nearly always without fanfare. Who talks about the key role played by the invention of the governor in 1788 by James Watt – the invention that made his steam engine practical? In that case, the governor was the newly-minted part of a steam engine that kept the pressure of the steam reasonably constant. With a governor, steam engines no longer blew up, as they regularly did before its use.

It’s important to understand that the reason a governor works is that it’s an integral part of the steam engine. Steam goes into the governor, which then controls the throttle valve of the engine, slowing it down when it’s getting dangerously hot. This is closed-loop.

In more modern terms, running open loop means going on and on down a path without real-world feedback and testing of your work as it is being developed. It’s a little like trying to walk to a goal post at the opposite end of a football field with your eyes closed, using a carefully planned sequence of steps and turns. That’s open-loop, which essentially mean no feedback. But shockingly enough, a huge fraction of highly technical efforts in software and analytics operate in just this way!  The people in charge insist they’re experienced, they’ve got a thoroughly vetted plan, and everyone should let them alone to get their work done.

There are many similarities between war-time software and running closed loop for analytics. Driving towards a goal, letting nothing get in the way. Optimizing for speed, not expectations. Leaping to a place that's better than today, and then cycling improvements.

The easiest way to see the difference is thinking about the previous posts in this series. Have you spent lots of time with data, and applied simple calculations to it? If not, you should. Once you have, … you should put your new understanding into practice! It may not be the very best solution that’s possible, but if it’s marginally better than what’s in place today, you should roll it out at least in a limited way and see how the world reacts to it. You’ll learn stuff! You may end up learning there are more variables you need to account for, different ways it needs to be applied, all sorts of things! In other words, don’t sit on the beach by the water for months – wade right in and see what it’s like. That’s when you’ll really start learning.

The World Responds and Changes

The key concept to understanding why running closed loop is so important is that the “world” is an incredibly complex, ever-changing set of actors. When you do something – almost anything – the world changes in response to what you did, if only in a small way. You have to run closed loop to respond appropriately as the world responds to your actions.

Oh, you may say, I’m just the genius in the back room who’s an expert in this or that branch of ML. I’m not acting on the world. I just need the time and support to get my amazing modeling work done.

That may be true. And that’s the problem! The whole point of doing ML/AI/etc. is to change something in the world –  and it’s guaranteed that the world will change in response! Accounting for the responsive changes is just as important as whatever it is you first put out there. Even worse, the world constantly changes independent of anything you may do. So the solution you modeled for may not be valid, given the changes that happened.

Think about the carefully planned walk on the football field to the goal posts I described above, and how hard it would be to accomplish with your eyes closed, i.e., with no feedback. Now think about the same situation, except there's an opposing team on the field! You carefully study everything about the opposing team. You know who they are and where they are. Then the play starts and you start to execute your exquisitely planned march to the goal posts. Here's the trouble: opposing team members see what you're doing, and they change their positions! They move! Even worse, they run towards you and try to tackle you. And you are helpless, because you are carefully executing your wonderful plan with your eyes closed, unable to react to the other team's movements. Is that stupid or what? It's not just stupid, it's inconceivably stupid. That's why I spelled it out, because that's exactly how most ML and other analytics efforts are carried out. Open loop. Assuming that the world does not change in response to what you do.

Of course, the world is unlikely to be quite as single-minded and determined as members of an opposing football team. But you'd be surprised! You're making changes in the real world. Whatever you do, there are probably losers. Losers who won't be happy, and will change their behaviors so they become winners again. Or simply fail to act in the predicted ways.

Conclusion

Running closed-loop is absolutely indispensable to achieving success. Put something simple in the real world and then cycle, making it better and better, using increasingly sophisticated techniques. Whatever your final crowning technique is, whether it's ML, AI or something else, success will be yours, and you'll enjoy it all along, without the risk, anxiety and likely failures of the usual highly planned methods.

Getting Results from ML and AI: 2

Getting practical, real-world results with ML and AI isn’t just a matter of hiring some people with the right credentials and throwing them at the project. Most such efforts start with fanfare but then fade into failure, usually quietly. The first part of this series laid out the issues, described the path to success, and concentrated on the indispensable foundation of success, the data. Data that has to be collected, corrected, enhanced and augmented – a time-consuming process that has no “advanced algorithm” glory, but MUST be done, and done well.

In this post, we’ll concentrate on the analytic methods that a successful project uses to generate value from the data so arduously collected and corrected.

A Little Background

I say I’m focused here on ML and AI. I just said that because it’s what everyone is talking about. What I’m really focused on is algorithms for understanding and getting value out of data. So I lied. Even worse, I’m not sorry – because just thinking that what’s important is to use the latest ML and AI techniques is central to the failure of most such efforts to deliver value.

I guess I can get over my programmer-ish prissiness that things are getting new names. What I refuse to get over is that lots of important, really valuable techniques are usually left out of the grab-bag of “ML and AI.” I won’t be comprehensive, but I think a glance at the landscape might help here.

There are a couple different ways to understand useful algorithms and how they came to be. Roughly, they are:

• Follow the algorithm, taking a fuzzy lens for the naming and details
• Follow the academic departments that “own” the algorithm
• Follow the problems the algorithm has proven to be good for

These ways overlap, but provide useful angles for understanding the algorithms and where to find them.

Let’s illustrate this with an amazing, powerful algorithm that is usually sadly ignored by people who are into ML and AI. It’s most often called linear programming (LP).Those who are into it think of it as being one of a category of algorithms called mathematical programming. More broadly, it’s normally “owned” by academic departments of operations research (OR). OR studies repeating operations like responding to repairs for appliances or controlling the output of oil refineries when prices and costs vary and optimizes the results. It’s been used for decades for this purpose in many industries, and is being rolled out today to schedule infusion centers and operating rooms in hospitals.  

This isn’t the place to spell it all out, but knowledge of amazing algorithms like LP is scattered over departments of Engineering, Computer Science, Math, Operations Research, Statistics, AI and others. The point is simple: the world of useful algorithms and modeling techniques is vastly greater than ML and AI.

The Natural Sequence

There are dozens and dozens of methods that can be used to analyze and extract value from data, which after all is the point of ML and AI (and, by implication, all the other great algorithms). As I described in the prior post, there is a natural progression or sequence of methods, which roughly follows their order of discovery and/or widespread use. Success usually comes from using the methods in the rough order of their invention as you climb the mountain of understanding from simple and obvious (in retrospect) results to increasingly non-obvious and subtle results.

I often see the following reaction to this concept, rarely articulated but often acted upon: “Why would I want to waste everyone’s time playing around with obsolete, outdated methods, when I’m an expert in the use of the most modern ML and/or AI techniques? I’m sure that my favorite ML technique … blah, blather, gobbledygook … will yield great results with this problem. Why should I be forced to use an ancient, rusting steam engine when I’m an expert in the latest rocket-powered techniques, ones that will zoom to great answers quickly?!”

The unspoken assumption behind this modern-sounding plea is that analytical techniques, ranging from simple statistics and extending to the latest ML, are like computers or powered vehicles. With those things, the latest versions are usually WAY better than prior versions. You would indeed be wasting everyone’s time and money if you insisted on using a personal computer from the 1980’s when modern computers are many thousands of times better and faster.

The trouble with this line of thinking is simple: the metaphor is inapplicable. It’s wrong! Analytic techniques are NOT like computers; they are like math, in which algebra does not make simple math obsolete – algebra assumes simple math and is built on it. Calculus does not make algebra obsolete – calculus assumes algebra and is built on it! And so on. Each step in the sequence is a refinement that is built on top of the earlier one. No one says, now that I know calculus, I refuse to do algebra because it’s old and obsolete. See this for more on this subject.

So it does make sense to quickly apply simple methods to the data to get simple answers, and at the same time vet your data. No time is wasted doing this. On the other hand, if you jump straight to someone’s favorite ML technique, not only is it likely that inaccurate and incomplete data will render the results useless … you won’t even know anything is wrong! Because most ML techniques do nothing to reveal problematic data to the researcher, while simpler methods often do!

Fundamental Analytical Concepts: Calculate it methods

The simplest and most useful methods are ones in which you simply calculate the answer. There’s no modeling, no training, no uncertainty. These methods are highly useful for both understanding and correcting the data you’ve got. The basic methods of statistics like regression apply here, and so do the methods of data organization and presentation usually called OLAP, BI and dimensional analysis. The tools associated with a star schema in a DBMS apply here, which are roughly the same as pivot tables in Excel.

Graphing and visualization tools are important companions to these methods; they help you really understand the numbers and see to what extent they make common sense and match reality. For example, you can see to what extent a doctor’s years of experience correlate with ordering tests or issuing prescriptions of a certain kind; or simply identify the doctors whose actions stand out from the rest. There could be a good reason why they stand out; wouldn’t you like to find out why? Maybe the doctor should be emulated by others, or maybe the doctor should be corrected; either way, you should figure it out.

Until you’ve pursuing all lines of thinking based on these simpler methods, it’s premature to move on.

Fundamental Analytical Concepts: Solve/Optimize it methods

These are, IMHO, the gold standard of algorithmic improvement. When applicable, they tell you how to reach a provably optimal result! No training. It takes experience and judgment to apply the generic algorithms to a particular problem set, and sometimes the problem needs to be adjusted. But the results are stellar.

First, you create an equation that measures what you’re trying to optimize. Is it fastest time? Lowest cost? Least waste? Some combination? Whatever it is, that’s what you’ll maximize or minimize as the case may be.

Next, you determine the constraints. You only have so many operating rooms? This kind of machine failure requires a repairman with that kind of skills?  Then you put in the inputs and solve. While I’m leaving out lots of detail, that’s the basic idea.

These methods, usually of the OR kind, have been applied with great success for decades. In certain fields and industries, they are part of the standard operating procedure – it would be unprofessional to fail to apply them. And you would rapidly lose to the competition.

Fundamental Analytical Concepts: Train it methods

The training methods all require sample data sets on which to “train” the model. Selecting and controlling the data set is key, as is avoiding over-training, in which the trained model can’t generalize what it’s been trained on, and thus loses most of its utility.

Fundamental Analytical Concepts: Train it methods: white box

What characterizes these methods is something incredibly important: what the model does can generally be explained in human-understandable terms, i.e., it’s “white box.” This has huge value, if only to gain acceptance for what the model does – but it may also bring up problems with data that can lead to further improvements.

There are lots of ML algorithms that are in this category. All the decision tree methods are here, among them the very important random forest method, along with methods that arose within the field of statistics such as CART.

Fundamental Analytical Concepts: Train it methods: black box

These methods can produce amazing results, and should be used whenever necessary, i.e., whenever earlier methods in the sequence can’t be used. The fact that the model is “black box” means that it’s difficult if not impossible to understand how the model makes its decisions in human terms – even for an expert.

These methods include neural networks in all forms, including all the variations of “deep learning.”

Fundamental Analytical Concepts: Rules

Finally, I add the indispensable attribute of success in many practical systems: human-coded rules. These can be inserted at any point in processing, as early as enhancing the data before any methods work on it, and as late as modifying the results of final processing. While not often explicitly discussed, few practitioners with successful track records avoid the use of rules altogether. They may not be pretty or fancy or elegant – but they work, darn it.

More elaborate than sets of rules is the technique in AI of expert systems. This is a whole big subject of its own. Generally speaking, if you can get useful results from one of the sequence of methods up to and including white box training systems, you should do so. But important categories of problems can only be solved using expert systems, which ideally should be as white box as possible.

Conclusion

There is a broad range of analytic techniques that can be applied to a given problem. There is an optimal sequence for understanding the data and the problem. Going from one step in the sequence to the next, when done correctly, isn’t abandoning a method for something better, but first picking the low-hanging fruit and then moving on to catch tougher stuff. Prejudging the best technique to use before really getting your hands dirty is a mistake. Being a specialist in a particular method, e.g. “deep learning,” and confining your activities to that method alone can get you hired, paid and busy, but may lead to no useful results, or results far less useful than they could be.

Getting Results from ML and AI: 1

We hear quite a bit these days about ML (machine learning) and AI (Artificial Intelligence), as the drumbeats of Big Data and Analytics fade away. You’ve got to be using these things to get great results and transform your business! You’d better round out the staff of Data Scientists you hired last year, and add appropriate numbers of ML and AI experts to the mix. Otherwise, you’re hopelessly behind the times, and you’ll eat the dust generated by the winners!

Most efforts to apply these technologies fail. Not loudly, of course – no one admits failure. But after enough time passes with exciting results being right around the next corner, people stop talking about catching a glimpse of light from the end of the tunnel, and accept the fact that they’re digging a tunnel deeper and deeper, a tunnel to nowhere.

Is there a way to get amazing value out of these exotic technologies? Yes. Decades of solid results show the way.

Typical Failure Patterns

In real-life cases, what happens all too often is that “data scientists” are called in to apply their magic. They take the available data and apply their favorite techniques. They may not produce results that are promising. If the results are promising, there is trouble applying them to the real-life situation. Or if they can somehow be applied, they don’t work or produced the promised results.

The failures are the direct result of taking a naïve approach to applying these kinds of techniques in the real world. There are proven methods for attaining good results, but those methods are rarely discussed, for reasons I don’t understand. If they’re discussed, they’re lightly brushed over – instead of being given center stage as they deserve. Following are the ways to be successful with ML and AI.

Build from the bottom up

No one tries to sell a house based on how solid its foundation is; but a house with a crappy foundation will collapse. No one brags about their arithmetic skills when trying to get a job as an ML expert; but if you can’t add, how can you do ML? There is a clear sequence of learning ML and AI. You start with learning how to count; if you don’t know the numbers, you know nothing. Then you learn basic manipulation of numbers, like addition and multiplication. Then it’s on to algebra and the following stages. You don’t try to learn exotic techniques until you’ve mastered the more basic ones on which they’re based, and on which they depend. These are things that are nearly universally accepted.

What’s not so common is applying the same sequence when analyzing any particular problem. The foundation of the sequence, the first step, is the numbers, the data. All too often, the data is incorrect and incomplete. If the data is bad, the results will be worthless.

I’ve never found anyone who disputes this, once the issue is raised. But I also rarely find people who act on it, and take it seriously. Why? Among other reasons, it's a multi-trillion dollar issue.

I have worked with some of the most advanced people in the field, including someone who’s been the chairman of one of the top academic departments in the field. This person and his methods have been behind a few of the most widely used success stories. Here’s his “secret” for when he dives into a new problem: he loads the data into Excel and looks at it, first line-by-line, and then using functions and visualizations. Yes, I know, Excel is something accountants use and try to avoid. But it’s ideal for fast, visual analysis of data sets, and has some of the most advanced algorithms available as add-in libraries. Why would you start programming in Python when you can move quickly mostly without programming using a tool?

The important thing isn’t the tool. The important thing is the activity – look at the data. Seriously look at it! Don’t just scan and move on. Understand it! What you’ll almost always find is that there are errors. Mistakes. Important stuff that’s missing. Graph it and look for basic correlations, and see if it makes sense. Make sure a true subject-matter expert is by your side.

Then the fun begins. You have to fix the data. It’s the foundation of everything else. Without a solid foundation, nothing of value can be built on it.

It’s also important to understand that this isn’t something mechanical, like spell-checking. What you often find is that really crucial data is missing, or that real important data can be added. This simple-sounding fact can be a project-maker. I have been involved with several projects in which the mundane-sounding effort of adding more relevant data has been the difference between failure and incredible, world-changing success.

OK, your data’s pretty good. Time to dive into ML? No way!! Way too soon! We’re going to go in sequence here, applying the very most basic techniques to the data first.

The point of all this is simple: you squeeze all the value you can out of a given “level” of technique before advancing to the next one. There are lots of reasons why this makes sense. The simpler techniques yield results more quickly than fancier ones. They tend to be larger and more obvious, which means the impact will be big. People will understand them, and so are more likely to buy into the changes needed to apply them in real life.

I’m not going to spell out all the techniques you should apply and the proper sequence, but generally speaking, the order is the same as the order in which the techniques were discovered historically, which is the roughly the same order in which the techniques are taught in school. So you try simple linear regression before multi-variate, for example. And you always look and use visual methods, because a surprising number of the advances are often ones that people in the field know or expect, or that at least “make sense” to them.

Finally, at long last, you get to use the fancy stuff you’ve been itching to use all along. But by then … your data’s in great shape. Your system is already up and running. People are already accustomed to change and the improvements that result from applying math. And interestingly, there may not be that much “juice” to be squeezed out of the system by then. Depending on your scale, that remaining juice may be tasty indeed, but it’s the icing on the cake.

In applying AI, the pattern is the same, except that in addition to applying simpler analytic techniques, you may be writing common-sense-understandable rules by hand. Why not? It gets the job done, it’s simple and direct, and the AI can focus on that yummy icing.

Other Issues

We’re not done! There are a couple other major, overriding issues to be considered in order to get great results from these advanced methods. I’ll cover them in future posts. They are:

• Most people have a favorite method, in which they have expertise and experience. That’s wonderful, except that there is a world of different methods, and many of them are simple inapplicable to certain kinds of problems – but great on others. Picking the right method (or methods) is absolutely key to success.
• Closed loop. All too many projects run open loop. In my descriptions above, I sneakily assumed closed loop – that’s where the best feedback comes from.

Conclusion

Wonderful results can be obtained by applying modern analytic methods to real-world problems. But you have to choose: do you want an academic prize or do you want real-world improvements? Sadly, those goals don’t have loads of overlap. If you want real-world results, you should build your effort on a solid foundation of accurate and complete data, and move from simple to increasingly refined as you apply algorithms to it. If you do, you’ll see positive results fairly quickly, and those results will get better and better as you climb up the mountain of sophistication.

Are Cashless Payments a Good Thing?

Cashless payments have been around for quite a while in the form of checks and credit cards. But even while physical cards are being re-issued with chips, there is huge innovation going on to bypass checks and cards, and go all-electronic. There is Apple Pay, remote sensing, QR codes and more. Lots of people are excited about a new wave of consumer convenience, most of it based on credit card technology. It's all good!

Is it really ALL good? You never seem to hear about the downside of all this new technology. But there's actually lots of downside. Payments are a huge field. So I'm going to just point out a couple issues with credit cards, the basis of most cashless payments by consumers.

The Volumes

Let's start by getting a sense of how much money we're talking about. Look at this:


That's more than $5 TRILLION dollars in 2016 just in the US. How much is that? Well, for starters, it's more than the entire US Federal budget!! In fact, it's over a quarter of the US GDP.

Merchant Discount rates

This is an incredibly important subject -- of which most consumers are blissfully unaware. The concept is simple. Suppose you buy something from a merchant for $100. When you pay cash, the merchant ends up with ... $100! what a shocking concept! Well, it is shocking when you realize that when you pay with a credit card, the merchant will end up with less than $98, and often less than $97.

That may not sound like a big deal. But what if you're running on thin margins, like many merchants, and are under intense competitive pressure to cut prices, have sale prices, etc. You can be doing well to end up with an overall margin of 10% on your business. This means that collecting $97 instead of the full $100 amounts to a margin reduction of ... 30 percent!

Merchants grit their teeth and take it. They have no choice. The giants don't care. But next time you go to a small shop or farmer's market and see a sign that says "cash only, please," you'll understand.

This is a complicated subject, since the discount rate includes more than a dozen charges for various things, the largest of which is called "interchange." But it easy to boil it down. Consumers put about $5 trillion on their cards, which they have to pay -- undiscounted. The discount is about 2-3%, which is between $100 and $150 BILLION dollars. A year.

Some combination of issuing banks, acquiring banks and networks (like VISA) are splitting that money.

Credit card debt

Leading thinkers in payments love to talk about how consumer-centric they are, and how they're investing heavily to make using cards even more convenient for consumers. Aren't they just wonderful people?

The magic of credit cards is that when you want to buy something, and you use a card to buy it, you can get what you want even if you can't afford it. Great concept, huh?

Consumers in the US alone are burdened with about $1 TRILLION dollars of credit card debt. And it's not rich people who have the debt. It is overwhelmingly people with little to no net worth.

What's worse is what they pay. Today's average of interest rates is around 19%. It hasn't been less than 15% in quite a while. So that means that consumers have to pay well over $150 BILLION dollars a year just to keep the debt current, not counting paying it off. Almost all that money goes to the banks that issue the cards, though various lawyers and collection agencies get their cut. 

What cards are worth

None of the institutions involved with cards care a whit about whether the card is physical or electronic. What they care about is protecting and preserving their more than $250 BILLION dollars a year in revenue. How much is that? Here is an excerpt of a list of the world's governments' revenue:

As you can see, the revenue up for grabs with today's credit cards in the US is about the same as the revenues of the government of Sweden, 15th out of more than 190 countries. Card revenues are worth, well, quite a bit...

What this means

What all this means is simple: the companies involved are living high on this revenue, and will go to great lengths to protect it. They will spend big on fancy consumer features -- so long as those features are based on the card infrastructure.

Why this matters is that there are incredibly inexpensive and fast methods of transferring money that are not based on the card infrastructure. Some of these are in widespread use outside the US. In the US, the least expensive such infrastructure in wide use is ACH, which is a government-run electronic form of checks.

There is no technical barrier to upgrading this network to be near-instantaneous. Technically, it could have been near-instant 20 years ago! But no one involved felt like doing it somehow, and even today, it's getting faster at an incredibly slow pace.

There are a couple modern peer-to-peer payment systems that are based on this technology. While they're growing, they are tiny compared to card volumes, and the card insiders are working madly to control and contain this threat to their massive revenues.

Conclusion

I'm strongly in favor of consumer convenience. My goal here is simply to point out that the convenience comes at an astounding cost that is hidden from consumers who don't carry a balance on their cards, and is a constant temptation and on-going burden to those who can't afford debt, particularly the expensive kind you get with revolving credit cards.

Blockchain 1.01

The post explains some of the most basic things about Blockchain. For more detail and analysis, see this and the links therein. For a basic definition, see the one at the end of this post.

Blockchain

“Blockchain” refers to widely varying bodies of code ripped from the Bitcoin source code, always leaving out the currency, and usually varying other key aspects of the code. These subsets of the base code are typically described as an "Immutable distributed ledger" or some such. Blockchain implementations are often created to respond to some crucial defect of the base system for a particular application. Such implementations wander far from the features that Blockchain is supposed to have in the minds of most people.

Ledger

Blockchain is most often described as a distributed ledger. The fact that it’s a “ledger” refers to the fact that blockchain is an extremely primitive DBMS that lacks a query language. As a “ledger,” about all it can do is support data writing and reading. Not only does it lack a query language, but it’s effectively a key-value store. This means that you better know the key of the ledger entry you want to read or update; without this, you are totally without recourse. Some anguished early investors in Bitcoin discovered this when they saw that their original investments had grown into the equivalent of millions of dollars, but they couldn’t access the Bitcoin because they have lost their key. Many others have lost their investments due to other security disasters.

As a result of this glaring deficiency, a couple resourceful people I have met who are building blockchain applications use a relational DBMS such as Postgres side-by-side with the Blockchain, copying all the data put into the Blockchain into the DBMS, along with the additional information required to make it a practical system. Why keep the Blockchain? Simple: who would invest if the venture didn't incorporate the hot new solves-all-problems miracle technology?

Distributed

Many Blockchain applications depend on the fact that it’s a “distributed” ledger to justify using the technology. Unfortunately, as implemented for Bitcoin, the fact that the ledger is distributed is a painful but necessary consequence of the design goal of having no one in charge of the system. It's a necessary evil! The distribution is due to an unknown number of “miners” who enforce the simple but computationally intense rules of the system; there may be over tens of thousands of miners, maybe even more than 100,000, no one knows. In the Bitcoin implementation, distributing the ledger in this way leads directly to the ten minute time typically needed to complete a transaction, and an unsatisfactory implementation of standard database ACID properties and the problematic CAP theorem for distributed databases.

Many Blockchain implementations attempt to overcome these problems by using common-sense measures that effectively eliminate the key features of Bitcoin. The most typical compromise puts the Blockchain under the control of a single organization; that opens it to the same kind of insider hacking that has broadly affected large commercial and government organizations. See this, for example.

Many users seem to think that being “distributed” is an advantage that makes Blockchain applicable to solving long-intractable problems involving related data in different locations managed by different organizations and systems. The fact that Blockchain can be implemented as a distributed ledger is completely irrelevant to solving problems of this kind. Blockchain's implementation of data distribution is vastly less effective and efficient than proven methods, and its general deficiencies make it far less useful than standard DBMS’s in such applications.

Immutable

Blockchain gets sold based on the notion that it’s “immutable,” implying highly secure. In the context of Bitcoin, it’s true that the only way an outsider could come in and change the ledger would be to organize a conspiracy of the miners. However, even with the incredible security provided by an army of miners, people have still lost their investments in Bitcoin by a variety of prosaic means, such as corruption or dysfunction in the wallet or system that represents you for holding or buying/selling bitcoin. There have been a number of such incidents that have been publicized, and more that have not.

The second you move from Bitcoin to Blockchain, the level of immutability plummets to little more than rhetoric. This is because the main security in Bitcoin is provided by miners, who are incented to keep the system secure. Once the miners are gone or under the control of a central organization, the usual methods for subverting software come into play, only without the layers of security that have prevented central bank systems from being hacked, at least so far. The typical Blockchain implementation is less immutable than a properly managed RDBMS with an appropriately replicated and secured log.

Given the widespread computer security breaches that have taken place, and will continue to happen, it's comforting to imagine that having an "immutable" distributed ledger would solve the problem. However, what the Blockchain does is replace the database in a normal system. So the question is, is the pervasive lack of security in today's enterprise systems due to vulnerable databases, or some other reason? The answer is clear: the database is the MOST secure aspect of today's systems, and security in it is rarely breached, and in no case of a properly administered one of which I'm aware. The security breaches are always "top down," i.e., the hacker gets into the system as though he were a legitimate user by one of a wide variety of means, and sometimes is an authorized but crooked user. Then he sucks data out of the system. The DBMS has NOT been breached in this kind of attack, and Blockchain is equally vulnerable to such exploits. And that's not even getting into the ultimate attack, which is an insider modifying source code, which has been the source of some of the most inventive banking attacks. SInce the database is not the problem, Blockchain is not the solution.

Smart contract

A “smart contract” is a key feature of the Etherium cryptocurrency and several Blockchain implementations, and is the foundation of many of the proposed applications of Blockchain. A “smart contract” is the exact equivalent of a DBMS stored procedure. In other words, it’s a body of code buried inside the blockchain (DBMS) that goes beyond storing and retrieving, implementing any logic the creator chooses. Most “smart contracts” today are written in newly created software languages; for example, Solidity is the one used in Etherium. These languages and systems are relatively new, and have flaws that have already been exploited by hackers. Using the name “smart contract” is effective rhetoric, but that’s all it is. It’s an immature programming language running inside a seriously deficient database.

If the version of Blockchain is run like a typical DBMS, with master/slave – in other words, going against all the principles that are supposed to distinguish Blockchain – then a Smart Contract could work like a stored procedure, though crippled by lack of functionality in the DBMS. Worse, there are serious unaddressed issues maintaining the basic ACID properties that any normal person would expect – things like either the money is removed from account 1 AND added to account 2, an absolute guarantee that BOTH happen or NEITHER happens, even in the face of machine failure. If it’s run like a “distributed ledger,” then you’ve got even worse problems.

Computer History and Context

Blockchain is experiencing a huge uptick in interest, involvement and investment. Many people and organizations are convinced that it is the key to solving many long-standing problems in various industries. Generally, the fact that there are huge problems of long standing and no clear path to solving them is correct. The thought is that, with so many obviously smart and hard-working people involved with software, the solution must be elusive because of a fundamental technology barrier. The solution must hinge on the arrival of some transformative new technology that will change the game. Blockchain!

Sadly, the long-standing barriers have nothing to do with technical break-throughs, and everything to do with incentives and standard industry practices that inevitably yield such results. A study of computer history and the way things work in various industries -- something that is rarely done -- would make this clear. While it's a huge subject, here are some posts (including the many embedded links) that can provide a starting point:

• The awful state of software development
• The AI trend in healthcare, the relevance of hot new technology
• Computer security issues that don't go away, and reporting vs. reality for security
• Concentrating on things that matter
• Tech fashion trends, for example Innovation

In spite of all the above, there's a way that Blockchain can be a big win in certain applications, when applied in smart ways by top-notch teams. There's a clear historical pattern to follow. Here's how.

Who Owns your Health Data?

You own all the data about yourself, right? It's your blood pressure, your date of birth and your test results, after all.

Forget about owning your data -- just try to get your hands on it! You may think you own it, but the sad fact is that you don't possess it! (Remembering hearing about how possession is nine tenths of the law? It applies here in spades.)

Health systems like to appear to be great folks who really care about you. These days, they all talk about how great their patient information portal (or whatever) is. Hah. Just try to use it!

I've talked about how essential it is to have a personal EMR. I've sarcastically described a big break-through in EMR interchange with details from a personal example. And most recently why portability is essential for the EMR.

I've just had another MRI and want to get my hands on it, so I can see the details. So I went to the hospital system website, and found right on the first page that they offer a patient portal that lets you get your information. Great! Here's what they say:

I took the next step and immediately ran into a bit of a problem:

Hmmm. I wonder which of these my doctor uses? And which one the MRI will be under?

After lots more work, I finally got to the right portal that had my stuff. I dove right in. I found my list of test results:

Hmmm, something is wrong here. There's got to be more than that! And what's that second item, the path report?

The path report, somehow dated both 11/23/2016 and 1/06/2017 is actually a copy of a report that was done at my first hospital, Mount Sinai, shortly after the biopsy procedure on Feb 21, 2014:

Bad data!

The first item is the most recent MRI I had done at Northwell. When you read the report:

You find that Northwell has a prior MRI they did dated 9/14/2016, and two earlier "outside examinations" dated 9/24/15 and 1/22/2015, in other words, MRI's done at some other hospital they choose not to name. But they had those MRI's in their system! They just chose not to show them to me.

Even worse, they admit that they performed a prior MRI themselves on 9/14/2016, and somehow it isn't made available to me, though it was in the system and available to the person who wrote this report less than a month ago.

They've thrown away or (more likely) withheld from me, either maliciously or incompetently or some combination, a report they did and two reports sent to them by Mount Sinai, helped in part by the miraculous digital system I described here.

Well, at least I should be able to go to the Mount Sinai portal and get the two missing MRI's, right? Let's see. Here is the top of the Mount Sinai patient portal test results page:

Score: one out of two. The MRI from 1/22/2015 is available, but Mount Sinai has decided I don't deserve to have the latest one, 9/24/2015. The reason is clear. That is the very MRI that was transferred to Northwell by the amazing digital process I described. When I removed the MRI from Mount Sinai on the DVD, I guess it was no longer there, right? That's how computer data works, right, when you take it from someplace it's no longer in the original place? How can it be???

Just in case, I decided to put in a request to the portal to supply the missing information. I was constructive, polite and provided the facts. Here was the cheerful, non-helpful response:

In other words, not my problem. As though the providers had anything to do with what information in the EMR makes it into the patient portal.

The only possible explanation for all this madness is that the hospitals involved are using some cheap new software written by a bunch of hacks. They can't possibly be using any of the most famous, widely used, expensive enterprise software systems, can they?

Let's see. Here's what you see at Northwell:

And here's what you see at Mount Sinai:

Oops. Two of the premier, widely used, non-cheap vendors. Each of whom is committed to modern, state-of-the-art EMR's with rich portals to enable patients to access their own data. Except on days that end in "y." Or when it's too hot or too cold outside. Or something.

Oh, I know! Their backs must be against the wall, humping to get all those wonderful features into their EMR's so that doctors can spend even more time staring at screens instead of being bothered by patient contact. They must be stretched so thin, they just can't afford to get the work done, and all my sarcasm is just nasty and uninformed!

Let's see. Here's AllScripts:

Lack of money is not the issue at Allscripts. Epic is privately held, so there's no way of knowing their profitability, but by every indication, they're doing just fine. 

I guess it all comes down to the simple fact of who owns the data: possession being 9/10's of the law, the fact that they have it means that we patients can have what we fantasize to be our own data when we are able to pry it out of their cold, dead hands.

 

Blockchain: a Sailboat without Sails

The interest and investment in blockchain continues its exponential growth. It seems there's no end to the long-intransigent problems it will solve!

There are just a couple little issues. First and foremost is that the virtues of blockchain, such as they are, have little to do with the problems it is supposed to solve. Second is that other technologies are better than blockchain, often by a factor of 1,000 or more. Finally, the most-discussed blockchain virtues solve problems that don't happen to exist in the real world. Other than that, blockchain is great!

Why then all the interest in blockchain, you might reasonably ask? You might as well ask lemmings why they are following the lead lemmings as they run off a cliff -- it's what everyone is doing!

Let's dig into just one aspect of this widespread delusion.

What does everyone say blockchain is? Some combination of the following:

• A distributed, immutable ledger
• The foundation of Bitcoin, only without the currency

We all think we know that Bitcoin involves cryptography, somehow making it safe, and that it electronically transfers data and value between distant parties. It's kind of like email in that regard, only so solid and safe that it works for money as well as a bank. Since blockchain is the "foundation" of Bitcoin, let's get blockchain to solve the many problems that involve getting data between places, and getting information exchanged. Let's go!

The widespread picture is that Bitcoin is just an application built on top of blockchain, kind of like how Oracle Financials is just one of many applications that are built on top of the Oracle DBMS. The idea is that blockchain is a platform on which applications can be built, like an operating system, DBMS, etc. That's what everyone is pushing, but it's a bogus idea!

The reality is that what we call "blockchain" is an artificial slice of an amazing piece of software that was artfully designed to solve a VERY hard problem -- and each piece of it depends on the others in order to work properly. Once you take the currency away, the whole thing falls apart. "Blockchain" is part of a whole, just like legs are great transport mechanisms, but only work as an integral part of a whole body.

Here is a sailboat:

The sailboat is like Bitcoin. It's got a hull, rudder, masts and sails. It's moving. It's an amazing invention. All the parts are designed with the others in mind.

Here is the same boat at the dock:

The sailboat at the dock would be like blockchain if you removed the mast, the furled sail and all related ropes. There is no doubt that the sailboat is "built on" the hull.

But what a ridiculous thing! What good is a sailboat without its mast and sail? There's no motor. It isn't even set up to be rowed with oars!

The reason is simple: the hull of the sailboat wasn't designed in isolation. It was designed as an integral part of a sailboat. You know, one of those things with sails. And without those sails it's nearly useless!

That's blockchain. A sailboat without the part that makes it useful.

The currency is what drives the blockchain forward.

Let's start by reviewing the capabilities banks have for normal currency. Let's remember that banks do DDA's (the thing we draw checks on and deposit checks into) just fine.

While we often hear people complain about not having enough money, we never hear it's because the bank has somehow lost it -- that their ledger has been falsified, so that money we deposited and thought we had disappeared. Bank ledgers are already "immutable." It's just not a problem.

Paper checks can take a couple days to clear. But certified checks can be turned into cash immediately, and you can get cash from any ATM. You can also use a PIN debit card to instantly access or transfer money. Fast access to and transfer of money is widely available. That's not a problem.

Banks have their electronic ledgers in multiple places, so that nothing is lost when there's a computer failure. Money can be moved between accounts in a bank in seconds. Banks already have distributed ledgers, and every modern DBMS supports replication of various kinds.

So what about Bitcoin? Has it invented things that aren't needed or already exist? Bitcoin represents an amazing way of implementing the following:

• A virtual currency
• which no single entity controls
• which incents "miners" to do the hard work of assuring that transactions are consistent and secure by rewarding them with newly-issued Bitcoin

It is not an advantage that Bitcoin has a "distributed ledger" for much of anything: it's got a "distributed ledger" so that no one entity is in control of Bitcoin. That's it! The "mining" is an extremely clever mechanism to pay groups to keep the ledger, stored in thousands of copies in many locations, up to date and consistent. That's why there's a consensus mechanism in Bitcoin, so that new transactions go into the ledger only when most of the miners agree they should.

In a normal bank DBMS, money is moved from one account to another in a fraction of a second. The transfer conforms to the traditional ACID properties of a DBMS, which assure for example, that the money is taken from one account and added to the other -- either both transactions take place or neither takes place. Any widely-used DBMS can do this many thousands of times a second. In Bitcoin, it takes on average 10 minutes for a single transfer to take place, with total throughput a tiny fraction of any modern DBMS. This is the cost of having a "distributed ledger," which is required to meet the key design goal of Bitcoin of having no single entity controlling the currency.

If you take away the Bitcoin, you take away the reward mechanism, and all you're left with is an insecure ledger that performs worse than a database on 50 year old computers. Worse.

The fact that the ledger is "distributed" is supposed by blockchain advocates to solve the problem of resolving data in different places, like between stock trade systems and hospital EMR's. Ridiculous. Blockchain is distributed only to meet the requirement of having no one in control. Any modern DBMS supports replication, which can keep remote DBMS's in sync while performing thousands of transactions a second. This works today. Blockchain is not an advance in this regard. In the end, all parties to a transaction have to get their data to a single place, and then have a couple copies of that single place for fault tolerance. DBMS technology is optimized for this use case. Blockchain accomplishes the same thing ... eventually.

Each case in which blockchain supposedly solves long-standing problems can be proven false, which is why you always hear about the wonders of blockchain:

• using the future tense, or
• in a proof-of-concept, or
• in an application that could have been built faster, cheaper and with better performance using modern DBMS technology.

When will the blockchain insanity end? Here are my earlier thoughts on why blockchain is so hot. Here is my analysis on blockchain applied to the stock transfer problem. Here's the upside, how blockchain will in the end deliver great value (heh).

I realize I haven't covered all the issues here, but when you're confronted by an entire mental hospital's worth of insanity, covering a single floor's worth of problems feels like a lot.

The IRS Anti-fraud Contract with Equifax is Good

First there was the furor that Equifax was hacked, putting millions of confidential consumer records in criminal hands. Next there was the furor about Equifax's response. Now, our in-bred elites are outraged that the IRS would award a sole-source contract to Equifax for, of all things, anti-fraud! Outrageous! Equifax can't protect itself, and now our genius IRS awards them millions of dollars?!

Sadly, this is yet another example of pathetically ignorant people expressing outrage about a perfectly normal and sensible action by the IRS that has nothing to do with Equifax's inexcusable malfeasance in protecting consumer data.

Here's the story in a nutshell.

Equifax

Equifax is one of a handful of companies that gathers and sells information about consumers, much of it confidential. It is a public company that provides an essential service to its customers, which are predominantly credit-granting businesses. The core of their business is receiving detailed transaction data from banks, aggregating it and selling it.

The Equifax breach and follow-ons

As usual with breaches, it happened long before the company became aware of it. Also typically, the company waited a long time before making an announcement. Equifax executives added an extra unsavory twist to the events by selling stock before the breach was announced. The response of Equifax to the event, which included a bogus offer of consumer protection against identity theft, was awful. Extremely little hard-core information about the breach has been released.

With this breach, Equifax joins the ranks of large institutions, private and government, that demonstrate their inability to keep their data assets safe. This is an ongoing scandal for which there are solutions, but none that major institutions care to use. I have written extensively about this.

The IRS contract

The IRS awarded a sole-source contract to Equifax for access to confidential consumer credit data -- exactly the same kind of service that Equifax provides to most of its customers. Public figures were outraged!

If the IRS contracted with Equifax to help apply its expertise to keeping IRS data secure, the outrage might have been justified. But Equifax does not sell those services. What they sell is data, data that the buyer can use for many purposes -- often for credit-worthiness, but sometimes to help verify consumer identity. The data was valuable for this purpose before the breach, and remains valuable today.

The data that was stolen was, of course, a snapshot of what Equifax had at the time of the theft. Since then, data has continued to pour into Equifax, updating and augmenting the data it already had. By using this additional data in special ways, the IRS could improve its ability to prevent identity thieves stealing taxpayer refunds, for example. I have no idea if the IRS will be smart enough to do this (I suspect not), but in any case they need the data! Without it, the IRS will be even more vulnerable to theft and fraud than it already is.

For the Senators to castigate the IRS for buying data from Equifax shows that they don't have a clue about computers and software -- they don't care to know the difference between services and data, for example. But we already knew that.

Clueless about Technology

What this is really about is that most people, including business, government and media elites, are clueless about technology. Which doesn't stop them from pronouncing about it with great confidence. As it turns out, I wrote about before, using the IRS and e-mail to illustrate the hapless opining of public figures about Bitcoin and Blockchain.

When things go wrong, "experts" are called in, and more money is spent doing the same useless things that let the problem happen in the first place. With the side effect that everything is even slower, more error-prone and vulnerable than it is today. The current round of posturing by public figures helps nothing. Sad.

 

How to Avoid Cutting off Breasts by Mistake

I think we can all agree that if a breast is not diseased, it should not be removed. High on the list of modest, achievable healthcare goals should be avoiding misdiagnosing breast cancer, and avoiding cutting off healthy breasts.

There are widely known methods and proven technologies that are available for achieving this goal. Instead of applying them, the leading minds in the healthcare technology field ignore them, and instead invest billions of dollars in exotic, cool-sounding AI software that may deliver benefits sometime in the future. Something is deeply wrong here.

Yes, breasts are cut off by mistake

Perhaps you think I'm exaggerating? Here is a recent cover from the NY Post:

The woman felt a lump and had a biopsy. The diagnosis was cancer. She was sent to another hospital for surgery.

Makes, sense, right? Someone should double-check. The rule in carpentry is "measure twice, cut once." When breasts are involved? Double-check should be the minimum; triple-check would be nice.

Checklists

Wait, hasn't someone gone into this already? For healthcare, specifically? Oh, yeah, there's that book written by that famous doctor on this very subject, Atul Gawande:

He dives real deep into checklists in many fields, medicine in particular. Here's a partial summary by Malcolm Gladwell he puts on his site:

The hospital at which the healthy breast was removed knew about checking.They wrote a procedure for it!

What was the procedure? More paperwork. Another in the long list of things doctors have to plow through before they can do their jobs. If doctors actually took all the paperwork seriously, they'd get nothing done. In other words, the "procedure" was just paperwork created by lawyers and bureaucrats that makes life harder and helps nothing. As usual.

Effective, automated workflow checklists

What the healthy-breast-removing hospital did was not what Gawande had in mind, of course. He was thinking group meetings at which everyone goes through each step and signs off. Something meaningful. But that doesn't exactly apply here, since the double-check pathology exam should have taken place before surgery was scheduled. Thinking about it, there are many similar situations in which the checklists are spread out.

There's a concept and proven technology that applies here: workflow. Think of a flowchart with worksteps, lines and conditional branches. Here's a trivial example:

Think of the workflow as being driven not by paper diagrams like this, but implemented in software that interacts with EMR's and people, and tells them what to do.

In this case, when the patient was referred for breast surgery, a proper workflow would have scheduled the pathologist to examine the cancer biopsy data before anything else happened. If the results were negative, as they would have been, the patient would have been notified and that would be the end of it. Only if the results were positive would the system allow breast surgery to be scheduled.

Workflow systems assure that all the work that should be done is done. They eliminate paperwork by putting the checking and doctor sign-offs into the system. No lawyers. No bureaucrats. Near-zero errors. Corrections and enhancements can be made to the workflow without massive re-training and human error. Workflow is like a system-wide, automated checklist, with all the checklist benefits and more.

Let's do AI instead!

Workflow software exists. It's implemented in many industries, even sometimes in healthcare. It works. It reduces human labor and improves outcomes. Why isn't it ubiquitous in healthcare?

Workflow works, solves problems and is understandable. AI, by contrast, is cool, the coming thing and few of the people yakking about it understand it. AI is exploding:

I see a recurring pattern here: people jumping on the new trend, the leading edge, the cool new stuff that "everyone says" is the coming thing, that none of them really understands. And ignoring relatively prosaic technology that works and can provide real benefits today.

Conclusion

Cutting off healthy breasts tells us everything we need to know about exotic new AI, which is at the far end of the innovation spectrum I've described for healthcare here, and illustrated here. Workflow is at the simple end of the spectrum: proven, works, understandable and understood, delivers immediate benefits. I guess that's why the leading thinkers ignore it and all the "smart money" avoids it. 

Government Cyber-Security tops the Oxymoron List

Some cynics keep an ordered list of the "top" oxymorons. Long-term members of the list include "business ethics," "military intelligence," "northern hospitality" and "southern efficiency." While "government efficiency" has a permanent hold in the top ten, "government cyber-security" has leaped to the top.

Cyber-security is a huge problem for large organizations in general, for deep systemic reasons, as I explain here. The US federal government is making a big effort to get ahead of the problem. By doing so, it is embarrassing itself and making things worse.

The government is engaged in a giant effort to systematize cyber-security and train professionals in how to do it. Various claims are made about salaries over $100,000 and a need for over a million qualified people. I first encountered this effort in an article about a local community college that had received certification for their program:

This sounds prestigious: both the NSA and DHS are jointly behind it! This must represent the best training there is!

So I looked into it a bit. Here is what the NSA has to say:

Here is what the DHS has to say:

I noticed a couple things:

• Each organization has its own page to describe the joint program. Of course.
• Each organization lists itself first in the description. At the NSA, the program was created by the NSA and DHS, while at the DHS, DHS is listed first. Of course.
• Each describes the programs and its goals differently. Of course.

This is bureaucracy as usual.

Then I decided to find out about the program itself, so I clicked on the link at the bottom of the NSA site. Here's what I got:

No kidding! That's why in the image above, I clipped to the top of my browser, so you could see the URL and see that I wasn't fooling around. This is exactly what I got by clicking on the NSA site shown above! Maybe it's just the NSA that's screwed up. DHS probably has a better link, since their website was updated less than two months ago. Nope! Same result!

Makes perfect sense. The NSA can't keep itself secure. We already knew this from the Edward Snowden problems, and more recently from their role in the world-wide Wannacrypt virus attack. The DHS? Even government investigators have concluded that its cybersecurity efforts are worthless. So why shouldn't their joint website fail the most elementary security test?

I dug and dug, trying to find what was actually taught, and what the cybersecurity standards and practices actually were. In particular, I was curious to find if there was any mention of the NSA's role in supplying the weaponry for the Wannacry attack by means of gross deficient internal cybersecurity. I was also curious to see what level of acknowledgement there might be of their problems.

Result: a couple hours of digging resulted in amazingly little of substance. I'll just end with an interesting comparison.

Remember the world-wide outcry about the guy being dragged off a United airplane? The CEO stepped up quickly and defended his employees. Then he took it back and abjectly apologized, and there followed a stream of discussions about how other airlines did it and specifically how United was going to change to prevent a repetition of the drag event.

Compare this to Wannacry. Not just one guy, but tens of thousands of organizations, including most of the UK's NHS -- resulting in massive patient issues, many of which were far worse than United's dragging event. Who was put on the carpet? Who apologized? Any word from the NSA about their security breach that greatly magnified the problem? Of course not! Don't be silly! This is an august government organization: no one apologizes, no one loses their job, and nothing changes. Got that?

Software Myth-Conceptions: Building Materials

An amazing amount of discussion about software is governed by metaphors. This is not surprising, since software is so abstract, and invisible to most people. So a shared physical metaphor helps everyone communicate.

The trouble comes when the metaphor doesn't match with the reality of software, as it often doesn't. In those cases, what usually seems to happen is that what everyone understands about the metaphor is assumed to be true of the software. Bad things then happen.

One of the most common metaphors for software is a physical building. Buildings normally have architectural plans that are determined before ground is broken, with the architect being in charge. Then some digging happens and a foundation is created. From then on the framing is built, the electrical and mechanicals are installed, and the finishes are applied inside and out. Now the building is ready for use. Something everyone can understand, right? It makes sense that in software we also have architects, carefully planned and laid foundations, workers who fill things out according to plan, and finally finishes applied.

Software is usually built according to the physical building metaphor. Things like Agile methodology don't change things very much -- they just mean that instead of a complete plan of everything from the beginning, things happen in phases. The physical building metaphor for software is unacknowledged, unchallenged ... and pernicious. Ignoring the metaphor is one of the main things that smart software people who follow Wartime Software practices do to beat the competition and win.

Temporary physical buildings

When we think of software in terms of the building metaphor, we nearly always think of "permanent" buildings, ones with poured foundations and all the rest. Of course, you darned well better have it all planned out right -- altering a foundation is no easy job!

What about the technology used to build temporary buildings? These aren't just tents for camping; some of them are extensive. A great example is the Frieze Art Fair, an international art fair held in NYC every year. It's huge; over 200 galleries exhibited last year. Here is a view of the extensive building from the air:

Here's a view of part of the exhibition area on the inside:

There were even a couple of restaurants:

All this was assembled in under a week, used for the time of the fair, and then torn down and carted away. No poured foundations. No big deal!

Architectural programs

Architects use software to do their work today. There are varying levels of programs they can use. I used a semi-professional one to design a house. The program was amazing. It helped me with everything. Here's part of a drawing from my project:

See the steps in the lower left? The software made it easy. I didn't draw every step. There was a staircase module which did most of the work for me. Same thing with roofs, walls, windows, doors, appliances and pretty much everything. At any time, it could generate a list of the building materials required.

In the physical world, I would need those building materials. But what if I were designing a building that existed only in a virtual reality world? I wouldn't have to go farther than the design -- the design itself is all that would be needed! Now let's turn to software. Suppose I'm designing a UI and I want a footer on every page, and a logo on the top right corner. If I'm reasonably competent with software, I don't have to count the pages and place the logo and footer as many times as there are pages -- I do it once, in a page master!

In the plans for the physical house, while it's easy for me to pop 10 identical windows into the drawing, when it gets built, someone has to build and install 10 physical windows. In the software world, the realization is done by more software. In the physical world, if I want a change to all 10 windows after they're installed ... ugh! Go back to all 10 and re-do them. In software world, I go to the footer or logo definition, change what I want, and boom! It's done!

Yes, you can design software that doesn't work this way. Yes, all too many people do it the wrong way. They're stupid. What's important is that software can be designed the smart way, and it's not hard, so often enough, it is.

The building metaphor

So what does this mean? A couple things.

Using the building metaphor encourages software to be built the wrong way. It encourages people to think about it inappropriately. The smarter you are in building software, the farther away from the physical building metaphor you get. The metaphor even gets you to think about roles badly -- the "architect" is someone who does lots of planning but no real work. Lots of workers are needed to build, workers who follow exact instructions. Then checkers who walk around with the plans making sure the workers did their job. All of this is counter-productive! It leads to bad process and bad software.

Let's knock the building metaphor down, bury it and move on.