/blog
Natural Language is the Future
Intuit sees natural language interactions as a key part of their future. They believe enabling customers to get questions answered and complete financial tasks with their own words greatly simplifies and speeds up their experience. Intuit built their chatbot with rich financial responses to give users the power of natural language for these reasons.
The Quickbooks Assistant chatbot is designed to help users get the most out of Quickbooks in three ways:
- Help users understand what features are available, how to use them, and how to troubleshoot issues, if needed.
- Help users understand their financial standing by providing instant access to any question they have about their money or taxes.
- Answer general tax and business questions for which users often search.
The chatbot is available in the native iOS and Android versions of Quickbooks, as well as Quickbooks Online on the web.
Conversational interfaces require specialized analytics
When Intuit first started working on chatbots, they learned traditional analytics tools do not work well for conversational interfaces. They first tried to make use of traditional web analytics tools and ran into issues. They then built their own in-house dashboards making use of third party tools like Splunk but were not able to traverse the conversations. After that, they performed a comprehensive search for conversational analytics tools and believed Dashbot provided the most robust offering for their needs.
Intuit saw the value in Dashbot’s conversational specific analytics. With Dashbot, users can traverse conversational paths, better understand users Intents, and optimize Natural Language Processing (NLP) response effectiveness.
One of the features Intuit makes heavy use of is Dashbot’s Message funnels – via the top messages in and out from a chatbot. Through navigating the flow of users messages and the bots responses, Intuit can better understand users interests and Intents, as well as see how well the chatbot is responding and make improvements.
Improving NLP Response Effectiveness
One area that Intuit is particularly interested in is improving the NLP response effectiveness of their chatbot. Intuit currently uses Google’s DialogFlow for NLP.
Intuit looks closely at their fallback percentage, the percentage of user inputs that don’t match an Intent in their NLP engine. A lower fallback percentage would mean their NLP is matching at a higher rate – handling more of the users’ messages. However, it’s not just whether there is a match; it needs to be a correct match. It’s important to look at mishandled Intents in addition to unhandled.
To improve the process, Intuit makes use of Dashbot’s Phrase Clustering and Unhandled Intent reports.
With Phrase Clustering, Dashbot groups similar messages together based on proprietary data science techniques. The Phrase Clusters can help identify unhandled and mishandled Intents. For example, a cluster for “help” may include a hundred different ways users ask for help – some of which might map correctly to the help Intent, some which are mapped incorrectly to a different Intent, and then others not mapped at all – the unhandled messages. Intuit can quickly view these clusters and update their NLP engine to support the various ways users ask for help.
“Dashbot phrase clusters are by far the best. We tried a competitor who had a similar tool that was useless – they were not accurate at all. We’re really impressed with the accuracy of Dashbot. Even our own internal team’s clusters weren’t as tight or accurate,” said Spencer Uttley, Senior Product Manager, Intuit Quickbooks.
Phrase Clustering saves a tremendous amount of time and is much more accurate. Previously, Intuit had to manually pour through spreadsheets of messages, trying to identify all the similar messages on their own. This was a time-consuming, laborious task that could result in overlooked messages as well. The goal is to prioritize Intents and identify which improvements can have the biggest impact on reducing the fallback percentage.
“Dashbot enables us to accelerate our NLP optimization through Machine Learning,” said Uttley.
Handling Fallbacks Gracefully
When a fallback does occur, Intuit prompts users with a series of related articles that may help the user, based on their internal search algorithm.
Intuit is able to track the success of these article suggestions using Dashbot’s event tracking. Event tracking enables Intuit to capture and report on which external articles were clicked on from the bot, as well as the user message that lead to the suggestions being displayed.
In addition, Intuit prompts the user for feedback on the suggestions. All of this information can be used to improved the response effectiveness.
User Satisfaction is Important
Intuit prompts users for a thumbs up or down if a response was helpful.
Intuit can quickly see the success of the responses through the Dashbot Message Funnels. One can traverse the flow from the chatbot response to the thumbs up, down, exit, or other response to better understand the users’ satisfaction.
In addition, Dashbot’s sentiment analysis tools help identify users’ overall feelings toward the chatbot as well as individual transcripts based on sentiment. The Intuit team often looks at the positive sentiment transcripts for inspiration on how the chatbot has been able to help users.
Dashbot is helping enable Intuit to improve the chatbot user experience through automated NLP optimization to reduce the fallback percentage, and reduce the overall manual labor involved. This in turn is enabling Intuit to provide higher quality answers and an overall better user experience for their customers.
About Intuit
Intuit’s mission is to power prosperity around the world. Innovation is nothing new at Intuit. It’s been our heartbeat for nearly a quarter century. As the world evolves, so do we. Yet we remain driven by our passion for inventing solutions to solve important problems, perfecting those solutions and delighting our customers.
We started small in 1983 with Quicken personal finance software, simplifying a common household dilemma: balancing the family checkbook. Little more than two decades later, our revenue tops \$3 billion, we’re publicly traded on the Nasdaq Stock Market, and recognized as America’s most-admired software company and one of the country’s best places to work.
About Dashbot
Dashbot is an analytics platform for conversational interfaces that enables enterprises to increase satisfaction, engagement, and conversions through actionable insights and tools.
In addition to traditional analytics like engagement and retention, we provide chatbot specific metrics including NLP response effectiveness, sentiment analysis, conversational analytics, and the full chat session transcripts.
We also have tools to take action on the data, like our live person take over of chat sessions and push notifications for re-engagement.
We support DialogFlow, Alexa, Google Assistant, Facebook Messenger, Slack, Twitter, Kik, SMS, web chat, and any other conversational interface.