About the Customer
Our client is one of the leading vision-benefits company in the United States, providing comprehensive eye care solutions to millions of members. The company offers a wide range of vision plans, including medical eye exams, prescription eyewear, and contact lenses. The company is committed to providing high-quality eye care services and ensuring access to affordable vision care for its members. Through its network of eye care providers, the company aims to improve the vision and overall well-being of its members. With the firm belief that sight is the window to wellness, the company is committed to bringing together the best people, products, and services to deliver greater access to high-quality, affordable eye care and eyewear. With a vision of providing access to high-quality, cost-effective eye care to the world, the company has been instrumental in consistently delivering on its promise of providing world-class products and services to eye care professionals, employers, and more than 80 million members worldwide. The company is a major practitioner of Corporate Social Responsibility and supports local communities around the globe through initiatives that bring eye care, eyewear, education, and disaster relief to places where they’re needed most. It’s a true global leader that fosters social, cultural, and economic diversity in a creative and collaborative environment, based on inclusion.
Customer Business Challenge
Our client has a Conversational AI solution that is being used as a chatbot solution for engaging with their end customers/patients. For example, the patients interact with the chatbot for various intents like ‘Book an Appointment’, ‘Cancel an Appointment’, ‘Check order status’ and so on. There are a few inefficiencies in the overall patient experience which the client would like to correct/fine tune mainly in the areas of natural language understanding. Currently the approach is for a manager to read through various chat transcripts for indicators of an ineffective patient experience and flag these for corrections in the Conversational AI solution. The manager then works with the IT team/ Conversational AI provider to fine-tune the chat bot intents for accuracy, this process is repeated across the various chatbot intents. The manual approach of reviewing chat transcripts for any natural language understanding issues is error prone, open to interpretation, takes up valuable time from a full-time employee, does not guarantee 100% coverage and risks employee dissatisfaction due to the repetitive nature of this work.
Since our customer had a system based on manual review of chat transcripts for addressing natural language understanding issues, this system presented several risks under the prevailing scenario. Some of the notable risk included occurrence of human error, subjectivity, time-consuming processes, limited scalability and delayed feedback. Manual review is prone to human error, leading to inconsistent and inaccurate analysis. Different reviewers may have varying interpretations of the same transcript, leading to inconsistency. Manual review is time-consuming, especially for large volumes of chat transcripts. As the number of interactions increases, manual review becomes increasingly challenging. Manual review can lead to delays in identifying and addressing issues with the chatbot’s natural language understanding capabilities.
The impacts of above-mentioned risks are far-reaching with poor user experience, reduced efficiency, increased costs, damaged brand reputation and missed opportunities amongst the most notable ones. Ineffective natural language understanding can result in frustrating user experiences, leading to dissatisfaction and churn. Manual review can significantly reduce the efficiency of the chatbot and the support team. The manual review process can be costly, both in terms of labor and time. Negative user experiences can damage the brand’s reputation. Delays in addressing issues can lead to missed opportunities for customer engagement and sales. In order to address these challenges and mitigate such risks, the customer team contacted Sincera and our teams stepped up to help customer with the resolution with a view to provide lasting customer success.
Sincera Solution
Sincera team solution to our customer for addressing chatbot analytics scenario focuses on leveraging GenAI as an NLP accelerator in order to alleviate the shortcomings, streamline the processes and automate the manual activities. This solution identifies natural language understanding issues in installed conversational AI solutions and earmarks certain transcripts as being more worthy of being submitted to a correction and redressal process than others. This is being achieved by a rating mechanism whereby the system evaluates each conversation, assigns certain scores to them and all such conversations which exceed a configurable score threshold are marked and sent to the appropriate customer team for correction. Our solution helps in scoring chat transcripts to facilitate fine tuning of conversational AI solutions using entity extraction, topic modeling and sentiment analysis.
Solution Architecture Diagram
The first step is to retrieve the chat transcript file by conversation ID. The chat transcript is fed in to the system in the form of text files. The system consists of an embedding model which performs the functions of embedding and chunking which allows the system to understand the semantics of the language and record the details in the form of vectors which eventually help us determine and perform sentiment analysis. The output is stored in the vector database and then Amazon Comprehend is queried to get the final chat transcript scores and sentiment details.
The entire solution can be connected to customer website or a mobile app.
API Gateway exposes endpoint as a customer endpoint where customer reviews are entered.
The workflow includes the following steps:
- When a customer enters a review (for example, from the website), it’s sent to an API Gateway that is connected to an Amazon SQS queue. The queue acts as a buffer to store the reviews as they are entered.
- The SQS queue triggers an AWS Lambda function. If the message is not delivered to the Lambda function after a few retry attempts, it’s placed in the dead-letter queue for future inspection.
- The Lambda function invokes the AWS Step Functions state machine and passes the message from the queue.
Step Functions analyzes the full sentiment of the message by invoking the detect_sentiment API from Amazon Comprehend. It writes the results to an Amazon DynamoDB table. It analyzes the targeted sentiment of the message by invoking the detect_targeted_sentiment API from Amazon Comprehend. It writes the results to a DynamoDB table using the Map function (in parallel, one for each entity identified in the message).
For downstream systems, the DynamoDB tables use Amazon DynamoDB Streams to perform change data capture (CDC). The data inserted into the tables is streamed via Amazon Kinesis Data Streams to Amazon Kinesis Data Firehose in near-real time. Kinesis Data Firehose deposits the data into an Amazon Simple Storage Service bucket. Amazon QuickSight analyzes the data in the S3 bucket. The results are presented in various dashboards that can be viewed by sales, marketing, or customer service teams (internal users). QuickSight can also refresh the dashboard on a schedule.
Execution and Project Management
With regards to the execution phase of our project, we adopted a phased approach to implement the system while taking due care to ensure business continuity and not disrupt the incumbent processes of customer.
From the outset, we employed a structured approach, beginning with a comprehensive project plan that outlined clear objectives, timelines, and resource allocations. Our team adhered to best practices in project management, including meticulous risk assessment, proactive communication, and continuous monitoring of progress.
Key milestones were achieved through collaborative efforts and adaptive strategies, ensuring that we remained aligned with our goals despite any challenges. By utilizing tools such as Gantt charts, Agile methodologies, and regular stakeholder meetings, we maintained transparency and accountability throughout the project lifecycle.
We used Amazon Quicksight to display the dashboards with insightful information for our customer purpose. Amazon QuickSight is a powerful business intelligence (BI) service that enables organizations to analyze data and create interactive dashboards. These dashboards provide a comprehensive view of your data, allowing you to visualize key metrics and trends in real-time.
QuickSight dashboards are designed to be user-friendly and highly customizable. Overall, Amazon QuickSight dashboards empower organizations to transform raw data into actionable insights, enhancing efficiency and enabling better decision-making across the board.
We have used these dashboards to provide following information to our customer teams:
- Scores assigned to chat transcripts
- Chat transcripts bifurcated according to sentiment category
- Chat transcript sentiment slice and dice according to age, gender, and location.
- Sentiment analysis by entity
Results
The project delivered extensive and significant benefits for the end consumers and the internal client teams consisting of managers and technical staff members.
- Improved Customer Satisfaction: Our project provided a systematic rating mechanism for the team so that chats can be identified for improvement leading to significant growth in chat transcript quality.
- Patient Issues Redressal: Our differentiated service coupled with a fully automated system that evaluates chat transcripts and presents the scores in an engaging format, helps our clients to easily focus on chats that need attention, thus contributing to patient issues redressal, lowering MTTR by 21%.
- Identifying trends and Issues: Our system provided the customer with an intuitive and scalable platform to analyse patient interactions closely, apply sentiment analysis in order to highlight recurring issues and trends, allowing support teams to address common problems more proactively.
- Customer Support Productivity: This project makes a variety of analytics available to the client using techniques such as entity extraction and topic modelling which has helped immensely not only in prioritizing and resolving issues but also in tailoring responses to be more empathetic and effective, thereby improving the overall customer satisfaction (CSAT) score by 24%.
- Better resource utilization: Our system minimizes the time client managers spend manually reviewing chat transcripts, freeing them up for other value-added tasks within the organization, thereby resulting in much better resource utilization.
These results demonstrate not only the successful implementation of our advanced technological solutions but also a substantial return on investment through increased efficiency, cost reduction, and enhanced service delivery throughout the organization and highlights our commitment to serving our customers with a collaborative spirit and strategic foresight in order to bring projects to successful completion.
For more information, write to: sales@sincera.com