While the success patterns laid out in the prior posts in this series may seem clear in the abstract, applying them in practice can be hard, because nearly everyone who thinks or talks about AI (these sets over overlap very little, sadly) takes a different approach.
http://www.blackliszt.com/2018/03/getting-results-from-ml-and-ai-1.html
http://www.blackliszt.com/2018/04/getting-results-from-ml-and-ai-2.html
http://www.blackliszt.com/2018/04/getting-results-from-ml-and-ai-3-closed-loop.html
I previously discussed the application of the principles to healthcare, with specific examples:
https://www.blackliszt.com/2018/08/getting-results-from-ml-and-ai-4-healthcare-examples.html
I've discussed the application of the principles to fintech, with a focus on anti-fraud:
https://www.blackliszt.com/2019/12/getting-results-from-ml-and-ai-5-fintech-fraud.html
In this post, I'll show how things can play out with a stellar example in fintech chatbots.
Computers talking with People -- Using People Language
The idea that computers should talk with us in our language rather than people struggling to learn computer talk has been around nearly as long as computers have. The earliest "high level languages" like FORTRAN and COBOL were each attempts to let people use English-like language for telling a computer what to do. They were baby steps, and people quickly decided that computers can and should do better. This thought was one of the earliest practical drivers towards Artificial Intelligence (AI) in general, and Natural Language Processing (NLP) in particular.
One of the acknowledged milestones towards a computer that can talk like people was the work of Terry Winograd at MIT during 1968-1970. He created a talking robot, SHRDLU, that could talk about and act in a special world of blocks:
SHRDLU could have conversations about the block world that amazed people at the time. Here's an excerpt, see this for more.
I was at college at the other end of Cambridge at the time, heard about SHRDLU, and got even deeper into the AI work I was engaged in at the time as a result, for example this project. SHRDLU was wonderful, but I wondered about and worked on how to represent knowledge and actions internally. After I graduated, "everyone" was convinced that "talking computers" were going to burst on the scene any day now. A couple decades passed, and nothing.
Talking Computers Today
Here we are, 50 years later, and things have definitely advanced. We have Alexa and Siri, and in banking we have Bank of America's much-promoted Erika. But in many ways, these programs remain primitive, and are far from being able to understand context and sequence the way people do.
One of the reasons for this is that human language understanding and generation is really hard. Another is that the vast majority of people who work on the problem are thoroughly immersed in the other-worldly vapors of academia, in which publishing papers and gaining esteem among your fellows are the all-consuming goals -- to the exclusion of building products that do things that normal people value and, you know, work.
The State of the Art in Chatbots
The founders of the Oak HC/FT portfolio company Kasisto come from that world, and also from the industrial labs of Stanford and IBM that try hard to commercialize such fancy stuff, with products like Siri as results. The Kasisto people are obviously exceptional not just in their field, but in their drive to make code that does real things in the real world. If that's your goal, there is exactly one standard for measurement: what people do and say.
That's the background of the first remarkable thing about Kasisto, which I learned when I probed details of their process. Pretty much everyone who deals with AI/ML builds and builds in the lab, until they have achieved amazing things that they're ready to roll out ... which promptly belly-flops in the real world. We'll do better next time, promise! Kasisto works the way everyone should work, but practically no one does -- people first!
This means they get in the middle of a huge number of human-to-human chat sessions, building and tuning their software first by human judgment, but increasingly by machine judgment. They "try" to answer the human's question, and just as important, they rate their ability to answer the question appropriately. After tens of thousands of tests, their ability to respond like a human would improves -- and just as important, their self-rating of how well they're likely to do improves as well.
As their answers and the associated self-ratings get good, they start actively playing a role in the live question-answer flow. For exactly and only the human questions for which Kasisto is "confident" (using an adjustable threshold) it will answer well, its answers go to the human -- but are still routed to a human for double-checking, with a frequency that goes down over time.
There will always be questions that are beyond the capability of the Kasisto bot, but so long as it is "self-aware" of which ones, the customers continue to get great service, with an ever-shrinking fraction of the questions being routed to humans for answers. If Kasisto can handle 95% of the questions, this might means that instead of a staff of 100 to respond to customers, a staff of 5 could do the job.
This way of thinking is far removed from the typical academic head-set -- and I've just given a couple highlights, there's actually much more where that came from!
Beating the Pack
What I've already described enables Kasisto to deliver chat results to customers that are far superior to anyone else in the field. But the overall head-set and people-first technique, combined with some true tech smarts, leads to further things.
The first thing is context. All the other chatbots are lucky if they can give reasonable answers to isolated questions. The combination of human-first, good tech and practical methods enables Kasisto to not only answer isolated questions far better than others, but follow-on questions as well -- questions that make no sense by themselves, but perfect sense when taken in the context of an interactive sequence. As a simple example, consider this:
"How much did I spend last month?"
"How much of it was on restaurants?"
"How about the previous month?"
That's a simple one for humans, but way beyond what other chatbots can do. Why? The computer has to figure out that "how much of it" means "How much did I spend on restaurants last month?" Not to mention, handling multiple languages; performing transactions; making changes and selling products.
While Kasisto started with consumer banking, it's already added high-value functionality for treasury, business banking and more. The key is: Kasisto, unlike everyone else in the field, didn't just start selling into those applications -- they followed the laborious but effective, bottoms-up, people-first method of exhaustive training and seamless integration with humans doing chat.
Conclusion
Kasisto is an excellent example of the incredible results that can come from following the path briefly outlined in these blog posts. You would think with all the talk about AI and ML, and the all efforts and announcements, that highly capable systems would be popping out all over. They're not! That makes the few that follow the success patterns I've discussed here all the more impressive.
Comments