I promise I didn't plan it this way, but when I looked on Amazon for a product to help me deal with an infestation of bugs, I encountered a major ... yes, bug. I described the bug in detail here, and at the time thought it might be isolated -- after all, in my all-too-extensive use of Amazon, I had never before encountered such a bug.
Not long after, as the issue continued to "bug" me, I went onto Amazon again, and found more interesting bugs. This could be just some scamming or corruption. I'm taking the trouble to describe what I discovered because it's exactly the kind of data that modern systems are based on, and by playing with the data, people with bad intentions can cause the systems that depend on that data to come out with the results the bad people want. It's a way of causing destruction or stealing that's indirect, effective, and likely to grow as evil-doers catch onto the technique.
The new bug
I went into Amazon looking for a pest repeller again.
This one looked pretty close to the first product I saw -- same vendor, same product appearance, nearly the same language. But the price was different, and the number of reviews, while huge by pest repeller standards, was about half that of the original listing.
I went on, and the questions & answers looked like they were real, and for the product itself. Maybe Amazon has fixed things!
Let's scroll down more and see.
Whoops!
Still an astounding number of positive reviews, but look at the categories and things mentioned. Children's books again, this time with religion and a magic tree.
But the pictures are more creative. Look at the second one above -- clearly snitched from some product far, far away from children's books! One of the leading reviews makes absolutely clear that some serious messing with Amazon's data has taken place:
The only way this is about pest repellers is if you consider the devil to be a pest. No, I don't think so.
Implications
This is a bug. And some serious messing with data inside Amazon's systems. And a GAPING hole in Amazon's security apparatus, showing us yet again that effective cybersecurity is NOT about getting certifications and passing tests devised by government lawyers and bureaucrats.
The direct implications of this are pretty small to the average person. But if best-of-breed Amazon can be hacked this way, what about the labs where data is collected and stored concerning our health? What if, instead of just stealing the data to make a couple bucks selling it to scammers, someone decides to mess with the data to achieve an outcome, like happened here? In cases like this, the data is manipulated to get buyers to see a product based on the large number of positive reviews. The same thing could be done for a drug, a therapy or a health center! Going deeper, the data could be manipulated to impact "knowledge" of the kind that AI/ML discovers and implements. Even more deviously, it could have a devastating impact on "personalized medicine," the most data-driven of all, with a screwed-with set of data that says that a certain innocent-sounding treatment would be just right for someone like you -- except, for someone like you, it's precisely personalized disaster.
In order for super-charged AI/ML engines to actually do good things, we have to make absolutely sure that their data foundations and underpinning are sound, secure, and un-manipulated. Sadly, this is not a "what-if" issue. For details, see this. The assurances of the usual serious-sounding, credentialed experts with stentorian voices and/or soothing manners are not enough. Not even close.
Comments