Blog Home

AllTop Stories
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
AI & Technology in CX

Important Factors to Consider when Exploring Sentiment Analysis in Customer Support QA: A CX Community Discussion

August 27, 2023
0 minute read

Sentiment analysis has emerged as a pivotal topic in the realm of customer support interactions, sparking intense conversations among industry professionals. As CX (Customer Experience) and QA (Quality Assurance) practitioners, it's crucial for us to approach this technology with a discerning eye, understanding both its potential benefits and challenges. We recently hosted a webinar discussion with CX leaders aiming to shed light on these important questions, fostering an environment of critical thinking and innovative problem-solving. In this blog, we delve into the key takeaways from this discussion, offering practical insights to effectively integrate sentiment analysis into QA practices.

"Sentiment analysis is something that's coming up so much in our conversations with customers & potential customers. It's the type of topic that is being thrown around so much on the internet and by different folks and it's a very sexy idea." - Vasu Prathipati, CEO & Co-Founder, MaestroQA

Wide Adoption and Implicit Detection 

Sentiment analysis has gained popularity due to its ability to analyze sentiment across a high volume of conversations. Unlike traditional customer satisfaction (CSAT) surveys, sentiment analysis offers implicit detection of customer sentiment without the need for explicit surveys. This wide adoption provides valuable insights into customer interactions.

"Sentiment analysis could be on 100 percent of conversations and it's implicitly detecting what is the customer experience sentiment versus asking for an explicit survey." - Vasu Prathipati, CEO & Co-Founder, MaestroQA

Enhancing Quality Assurance Efforts 

One of the main motivations for exploring sentiment analysis is to augment QA programs without significant increases in individual grader headcount. By leveraging sentiment analysis, companies can gain deeper insights into customer support interactions, even when QA resources are limited. It offers the potential to scale QA efforts and derive valuable analytics from a larger volume of data.

With MaestroQA, teams can comprehensively analyze 100% of interactions for negative sentiment using an omni-channel transcription tool. Through transcribing all voice interactions, teams can construct AI classifiers and Smart attributes based on keywords. These conversation analytics are completely customizable to accommodate the specific needs of the business.

"The biggest pushback that I hear from our customers is, 'Hey, we love Maestro. We plan on scaling the team, but we can't scale the QA program in terms of headcount proportionately. What kind of technology and AI and stuff can you use to help us get more in analytics without having to scale the QA teams headcount?'" - Vasu Prathipati, CEO & Co-Founder, MaestroQA

Third-Party Tools and Open Source Solutions 

Many companies, (including MaestroQA, at the moment) utilize third-party tools to power their sentiment analysis capabilities. Industry giants like Amazon and Google offer pricey sentiment analysis APIs that leverage machine learning models. Additionally, open-source technologies have emerged as cost-effective and potentially improved options for sentiment analysis.

"Most people are using third party technologies to power their sentiment. We're evolving our sentiment analysis at MaestroQA. We use a hybrid of multiple tools to provide our sentiment analysis and we're working on transitioning towards an open-source model. The benefit of that is we can do it at a tenth of the cost, so we can make it a lot more affordable for our customers." - Vasu Prathipati, CEO & Co-Founder, MaestroQA

Use Cases and Nuances 

Sentiment analysis can be applied to various use cases, such as measuring positive or negative sentiment in agent and customer emails, chats, or phone calls. Each use case has its own nuances, and sentiment analysis should be tailored accordingly. As every business is different, "positive sentiment" can mean one thing for one company and something completely different for another. Once you've determined what type of sentiment you are looking for, positive sentiment may be more relevant in evaluating agent performance, while negative sentiment can help identify areas for product improvement or potential process & policy adjustments.

For example, using MaestroQA, Novo, a leading mobile banking platform for small businesses, was able to uncover an issue that was occurring with mobile check deposits, where specific checks from the California State Government were being repeatedly rejected by their banking software. With the use of AI Classifiers, Marcus will be able to build reports using keywords associated with the "check rejection" issue to determine its frequency, identify which agents are skilled at handling these situations, and assess the negative sentiment resulting from these customer interactions. MaestroQA's highly accurate transcriptions will even enable the Classifier to listen for key phrases such as "check rejected, California, or deposit error" to identify phone calls discussing this particular issue. Novo’s QA team can also pull in other metadata from his HelpDesk to identify tickets where a customer started the conversation upset but was satisfied with the agent's response or conversations where a customer initially felt fine but became upset due to the agent's service or policy.

"You have to think about your actual conversation flow and where does it make sense to actually try and detect it. When applied critically, sentiment analysis can uncover hidden trends you wouldn't see." - Vasu Prathipati, CEO & Co-Founder, MaestroQA

Accuracy and Actionability 

Ensuring the accuracy of sentiment detection is essential. If your sentiment analysis is yielding high overall positive sentiment scores, its actionable value should be questioned. Metrics that provide actionable insights should be prioritized, enabling targeted coaching and informed decision-making. 

Our customers are harnessing the power of our AI-assisted sentiment analysis to develop more refined and actionable metrics, which in turn provide a detailed overview through a Targeted QA heatmap. For instance, one company identifies tickets with a "low empathy response," a customized Key Performance Indicator (KPI) established using MaestroQA's sentiment analysis and AI classification tool known as AI Classifiers. The concept of "low empathy response" involves detecting instances where a customer expresses negative sentiment and receives a poorly tailored macro-like response from the agent. When agents fail to provide personalized or appropriate reactions to customers' negative sentiments, they miss the opportunity to connect effectively with the customer, potentially exacerbating the negative experience.

By unearthing this particular metric, the team managed to delve deep into targeted QA, uncovering valuable learning and coaching prospects for their agents. This proactive approach aims to mitigate such interactions in the future and enhance overall customer experiences.

"Did you get a new perspective that you can consider internally to have smarter discussions internally?" - Vasu Prathipati, CEO & Co-Founder, MaestroQA

Machine Learning and Agent KPIs 

During the discussion, opinions were divided on using machine learning-based sentiment analysis as a key performance indicator (KPI) for measuring agent performance. While some participants (47% according to a poll) saw potential in leveraging machine learning for identifying lower-performing agents or areas of concern, others expressed concerns about the explainability and transparency of such metrics (41% said no and 11% were undecided).

“I think a really important consideration when we're talking about, ‘Hey, do we wanna use this as a way to measure agent performance?’ Well, it's machine learning. If someone on your team says, ‘Hey, why did I get this score? Why is this my score?’ How comfortable do you feel explaining this?” - Vasu Prathipati, CEO & Co-Founder, MaestroQA

The Black Box vs the Transparent Box

At MaestroQA, we adopt a cautious approach to Sentiment Analysis and AI implementation to maximize accuracy and reliability of results. Amidst the numerous claims of software offering 100% automated QA for customer interactions, a crucial detail often remains unaddressed. The source of this data remains ambiguous: Is it derived from an undisclosed black box of sentiment analysis or an internally developed Auto QA system that's uniformly applied across all QA clients?

At MaestroQA, we are committed to dispelling this opacity. Through our MaestroQA AI Classifiers, QA teams gain the ability to establish tailored customer Auto QA KPI metrics (a transparent box). This empowers you and your team to precisely define positive and negative customer sentiment within the context of your unique business and outline benchmarks for superior agent performance that align with your customer expectations.

The development of these AI classifiers occurs collaboratively with your designated MaestroQA Customer Success Manager. By deconstructing the specific customer pain points you intend to address, you and your representative collaboratively construct a logical framework within the MaestroQA tool. This framework facilitates the comprehensive scanning of 100% of support tickets to identify qualifying attributes. The outcome is then presented to you on a Performance Dashboard, displaying the percentage or total count of tickets that adhere to this established criteria.

What sets our sentiment analysis and auto QA apart is its genuine relevance to your business. Through customization, it becomes a tool that resonates meaningfully within your unique operational context.

Bridging the Divides 

In conclusion, sentiment analysis shows promise in enhancing customer support quality assurance efforts by providing deeper insights and analytics. However, it is crucial to carefully evaluate use cases, choose appropriate tools, and ensure the accuracy and actionability of sentiment analysis results. Additionally, using sentiment analysis as a key performance indicator (KPI) for agent performance should be approached cautiously, taking into consideration the transparency and explainability of machine learning-based metrics or the value of sentiment analysis as a tool for targeted manual quality assurance.

The community discussion on sentiment analysis, as part of our bi-monthly CEO/CX: Strategy Chats, sparked valuable insights and discussions among participants, enabling them to consider new perspectives and challenges. As the field continues to evolve, staying informed about the latest developments, evaluating the suitability of sentiment analysis for individual businesses, and making informed decisions based on specific use cases and requirements are essential.

Want to learn more?

For a deeper dive into these topics, we recommend watching the full recording of our Strategy Chat with MaestroQA CEO & Co-Founder, Vasu Prathipati, or joining our upcoming webinars on the future of customer experience. Your participation is greatly appreciated, and we look forward to engaging in more insightful discussions in the future. If you're not already familiar with MaestroQA, don't hesitate to request a demo today!

Previous Article

Mastering Customer Interactions in the Age of DSAT

The Essential Guide to Chatbot Quality Assurance: Ensuring Excellence in Every Interaction

Navigating AI Implementation Strategy in Customer Experience: Risks and Strategies

Elevating Call Center Performance with Six Sigma and MaestroQA

Elevating Business Excellence Through Non-Customer-Facing QA: A Strategic Imperative

Elevating Trust and Safety through QA: How TaskRabbit Sets the Standard

Unlocking Superior CX: The Bombas Blueprint for Quality and Coaching

Unleashing the Power of Customer Conversations: Top 6 Tech Trends Revealed at the CX Summit

Important Factors to Consider when Exploring Sentiment Analysis in Customer Support QA: A CX Community Discussion

Agent Empowerment: 5 Tactics for Customer Retention from Industry Leaders

Mastering Agent Onboarding: Quality Assurance Lessons from ClassPass

Driving Business Impact with Targeted QA: Insights from an Expert

The Transformation of QA: Driving Business Results - Key Takeaways from MaestroQA’s CX Summit

How Angi Unlocked Growth and Continuous Improvement with QA

De-Villainizing QA Scorecards with Hims & Hers Customer Service

How to Revamp QA Scorecards for Enhanced Quality Assurance

The Art of Outsourcing Customer Support: Lessons from Stitch Fix's BPO Partnership

Writing the Auto QA Playbook & Transforming Customer Support

Advancing Customer Service Metrics with AI Classifiers

MaestroQA Named One of Comparably’s 2023 Best Workplaces in New York for the Second Consecutive Year

How to Maximize Call Center & BPO Performance | MaestroQA

MaestroQA Named on Comparably’s Best Workplaces in New York

CX Strategy: The Future of AI in Quality Assurance

Elevating Customer Satisfaction with Visibility & Coaching

Champion-Challenger Model: Improve Customer Service In BPOs

5 Key Strategies to Supercharge Your BPO Partnership

How Customers Collaborate with Their BPO Partners Today

Kick Start Your Customer Service BPO Partnership Successfully

BPO Call Centers: Best Practices for Quality Assurance

Empathy in Customer Service: Everything You Need to Know

Call Calibration: What is It & What are the Benefits?

Increase QA Team Alignment with Call Calibration & GraderQA

Measuring An Organization's 3 Ps: People, Process and Product

How to Onboard Your Customer Service Team to a New QA Program

Average Handle Time (AHT): How to Calculate & Reduce It

Should You Have Dedicated Quality Assurance Specialists?

The Top 4 CX Books Recommended by Our QA Community

How Top eCommerce Brands Ensure Exceptional Customer Service in a Remote World

21 Key Customer Experience Definitions for QA Professionals

5 Key Components of a Remarkable Customer Service Experience

Customer Service Management 101: Everything You Need to Know

A Guide to Customer Service Quality Assurance Programs

The Ultimate Guide to Improving First Call Resolution (FCR)

The Key to Customer Service Coaching Is More Data (and Fewer Opinions)

How to Refresh Your Call Center Quality Monitoring Scorecard

How to Update Your QA Scorecard

The 9 Customer Service KPIs Needed To Improve CX

3 Ways to Test Your Call Center Quality Assurance Scorecard

Leveraging Customer Sentiment to Improve CX in Call Centers

This Is What an Effective Customer Service Coaching Session Looks Like

What is DSAT and 5 Steps to Improve It

Customer Experience Management and Quality Assurance Jobs

How Deeper CX Analytics Lead to Better CSAT | MaestroQA

Achieving Effortless Customer Experiences (CX) with QA

How to Create an Omnichannel Call Center Quality Assurance Scorecard

Beyond Low CSAT Scores: Finding the Root Cause of Poor CX

Customer Service Coaching 101: Improve Agent Performance

Build the Ultimate QA Scorecard Process for Email and Chat

MaestroQA's Aircall Integration: Bring Your Calls to Life

Call Center Quality Assurance with Zola and Peloton

Auto-Fail in Call Center QA: What It Means and When to Use It

Why Poor Agent Experiences Happen (and How to Fix Yours)

20 Call Center Coaching Tips to Boost Agent Performance

Why Getting Buy-in for Quality Assurance is Essential

Setting Up a Grading Cadence for Your QA Scorecard

How to Avoid Bad Customer Service as you Scale your Business

Building a New Call Center Quality Assurance Scorecard

What CX Leaders Need to Know About Ecommerce Industry Trends

Call Center Cost Per Call: How to Calculate & Reduce It

6 Tips to Automate Your Customer Service Management Process

Quality Assurance and Training with Seismic Learning & MaestroQA

11 Customer Service Training Ideas and Skills for Your Agents

The Past, Present, and Future of Quality Assurance

고객센터 품질관리를 위한 QA전문가가 꼭 필요할까요?

Improve CSAT Scores: Understanding Your Experience Blindspot

What CX Leaders Need to Know About Security and Compliance

Streamline Your Call Center's QA Program With 4 Key Features

How Customer Experience Teams Can Impact a Company's Brand

Customer Loyalty vs. Customer Retention: Which Matters More?

Empathy & Authenticity: Customer Service Skills to Improve CX

How High-Performing CX Teams Build Accountability

Five Questions to Jumpstart your QA Scorecard Research Process

How to Improve Call Center Agent Performance: 6 Key Tips

3 Ways to Improve Your CSAT Score through Quality Assurance

3 Strategies on How to Increase Customer Loyalty

A Guide to Net Promotor Score (NPS) for Customer Service

Reasons for Call Center Attrition Rate and How to Reduce It

Voice of the Customer (VOC): A Guide for Great CX Teams

What’s Really Behind Your CSAT Scores? Diving Deeper

How CX University Improves Brooklinen’s Agents Performance

Understanding Customer Effort Score (CES) & How to Measure It

Improving the Customer Experience with DSAT Scores

5 Tips for Customer Service Coaches in Call Centers

Your Most Important CX Metric Is Your QA Score - Here's Why

How Agents Can Make the Most of Customer Service Coaching

How to Grade Customer Service Calls

Why Top-Performing CX Teams Focus on Workforce Engagement

Improve Customer Satisfaction with a CX Quality Management Program

Customer Service Training and Quality Assurance – How Lessonly and MaestroQA Close the Loop

How to Calculate CX Quality Assurance Scores