Continuous Improvement: Analytics and Insights to Monitor, Measure, and Improve Intelligent Virtual Assistants

Brands have widely started using conversational AI-powered intelligent virtual assistants (IVAs) to communicate with their customers and employees. However, simply deploying it is not enough. To offer the optimal conversational experience, brands must regularly check the performance of their intelligent virtual assistant (IVA) and make necessary improvements.

Signs That Intelligent Virtual Assistants Need Continuous Improvement

What metrics indicate a poorly performing Intelligent Virtual Assistant? If you see these signs in your analytics, then it’s time to consider actively managing your intelligent virtual assistants.

• Lower first-call resolution and increased churn rate
• Customer dissatisfaction increases as the number of false positives increases
• Customer retention is declining
• Poor customer adoption

To combat this, IVA developers must keep track of various IVA metrics, data points, and KPIs and utilize this knowledge to make modifications that improve performance.

You Need More Than Just Analytics - You Need Insights

Many conversational AI tools in the market have out-of-the-box analytics features. However, we realized that the majority of them only provide quantitative metrics (interactions and user count) with little meaning. This places an additional strain on the analyst to churn data and draw relevant insights. 

There is a need for an analytics module that provides metrics as well as thorough insights, validations, and suggestions such as intent identification, conversation history, customer KPIs, and other ready-to-use information.

What We Learned From Our Global Customers  

Kore.ai has worked with large corporations in a variety of industries including banking, healthcare, insurance, retail, and manufacturing. Many of our customers have massive deployments that handle millions of voice and text transactions with customers and employees. By teaming up with them, we are able to effectively monitor the tremendous amounts of data available. This allows us to understand IVA behavior and what customers actually need from their experience.

With this experience, we developed the Kore.ai Continuous Learning Framework. The framework aims to continuously monitor all aspects of the IVA. Intents, entities, NLP data, integrations, use cases, dialog tasks, conversation flows, and user trends are all part of it. We believe that by analyzing these variables and making the appropriate changes, we can significantly improve IVA performance.

What does Kore.ai XO Platform Analytics offer?

The Analyze 2.0 module of the Kore.ai XO Platform provides multiple out-of-the-box Dashboards that display extensive IVA analytics including real-time metrics, insights, validations, and metrics. Listed below are some indicators:

• Trends in Conversation Sessions
• Analysis of Engagement
• Returning Users vs. New Users
• Intents (both Identified and Unidentified)
• Rate of Containment
• Drop-offs
• Customer Key Performance Indicators
• Conversation History, and more!

Let’s explore this further - 

Overview Dashboard

The dashboard provides a bird's-eye view of key virtual assistant parameters such as conversation sessions, users, messages, goal completion rate, and so on. You may restrict your search by using the filters. You may learn more about each metric just by clicking on it. The data is useful for leadership teams to monitor the efficiency of the virtual assistant.

Conversations Dashboard

Understand when and how your customers converse with virtual assistants and track their success. Monitor the conversations managed by the IVA, including those transmitted to live agents, self-service, and drop-offs. Get insights into your consumers' interaction channels, average daily conversations per user, and other crucial analytics.

User Dashboard

Discover who interacts with your virtual assistant. The dashboard provides detailed user trends, such as the overall number of users with a split of new and returning users during a certain time period. It can assist you in estimating the retention rate and making improvements to the virtual assistant.

Performance Dashboard

The mantra for providing a memorable experience while engaging virtual assistants is continuous improvement. You should closely monitor the VA's performance and make incremental changes as needed. The dashboard categorizes information into four categories to aid you in determining the performance of the virtual assistant.

Custom Dashboards

We've seen businesses use IVAs to solve key problems in a variety of situations. As we constantly say, the applications of IVAs are limitless. In such cases, the metrics offered by the platform may be insufficient.

This issue is addressed with the Custom Dashboard. You can create a dashboard to measure specific company KPIs. Add one or more widgets associated with pre-built datasets. Create a dashboard by extracting messages, sessions, user information, and meta tags from these sources.

A good example of this is wanting to get information on purchases made by your premium clients during the last three months.

NLP Insights 

Continuous learning is essential for virtual assistants in order to deliver the best experience to the end user. Training can be either supervised or unsupervised. However, it is always recommended to provide continuous training and to review IVA replies on a regular basis.

The NLP Insights dashboard provides insight into IVA performance, including detected and unidentified intents, unhandled utterances, and unsuccessful jobs. You can also review and improve performance by implementing corrective measures.

Conversations History

Intelligent virtual assistants handle thousands of conversations every day. It is impractical for VA analysts to go over each utterance and assess it. However, estimating and improving IVA responses is crucial.

The Kore.ai XO Platform provides a conversations history dashboard to assist you in this regard. It provides a consolidated view of all virtual assistant conversations, together with a thorough synopsis of the conversation flow and key occurrences. You can immediately review them and collaborate with your teammates to make changes.

Conversation Flows

Tracking every touchpoint of your consumer is crucial if you want to invest in an experience-led business. It includes what they are doing, when they are doing it, how they are doing it, and various conversation paths they are taking. Tracking this data provides a treasure of insights.

With the session flow visualization tool, the Conversations Flow dashboard maps popular user paths and exit points. It provides a detailed view of messages and tasks at a certain point in a conversation.

These dashboards give valuable information and insights that help you evaluate IVA performance, understand your customers, and develop plans to improve customer experiences. The XO Platform not only gives analytics but also validations and recommendations to improve the training experience. You can inspect and change the purpose and entity information by directly clicking on these interactive dashboards at any time.

Now is your chance to try out the industry-leading Kore.ai XO Platform for free!

The Kore.ai XO Platform Analytics Brief is available for download.