Blog Home

AllTop Stories
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Quality Assurance

Writing the Auto QA Playbook & Transforming Customer Support

April 19, 2023
0 minute read

As the customer support industry evolves into an experience center, agents are no longer simply support representatives. They also serve as brand ambassadors and are critical to customer retention. However, the introduction of AI tools is once again transforming the CX landscape. This blog draws heavily on insights from a recent webinar with Vasu Prathipati, CEO & Co-Founder of MaestroQA. We will explore how an Automated Quality Assurance (Auto QA) program can speed up QA processes to revolutionize customer support, improve key performance indicators such as CSAT and AHT, and ultimately boost your bottom line. By sharing Vasu’s valuable frameworks, techniques, and tips, we aim to equip you with the knowledge needed to navigate this transformation and stay ahead of the competition. 

So, read on to learn more about how to write your own Auto QA to transform your customer support experience and drive sustainable business growth.

What is Auto QA?

First, let’s define auto QA. 

Automated QA (Quality Assurance) for call centers provides automated quality metrics on 100% of your support interactions. 

By automating the QA process, companies can customize and run precise QA metrics to surface actionable insights from all your customer interactions -- voice, chat, text, and screen capture.

Shifting from Manual Call Scoring to Automated Scoring

The Shift from Manual QA to Pairing Auto QA with Targeted Deep Dives

Today’s CX leaders want their quality assurance teams to execute a vision (see image above) that involves a gradual shift from manual QA to pairing Auto QA with targeted deep dives. “We’re seeing our customers come to us with this vision of how they want quality teams to operate going forward. Quality assurance teams want to do this. They see this as a huge opportunity to unleash their creativity. The reality, however, is that they come into this role of quality with the promise of being an analyst and coming up with insights, but they end up feeling and spending a lot of their time doing more data collection. The word auditor comes to mind.

"The hope is that there's a possibility with new technologies and new companies like MaestroQA that we can help quality teams transform into the promise that they've always set out for themselves, which is becoming a data analyst or a customer experience data analyst in particular.

"So, that’s the shift we’re working on with our customers, and that’s exciting because by allowing Auto QA workflows to handle tasks such as identifying low-scoring DSAT tickets to perform targeted QA or automated transcription to analyze 100% of interactions to get a more accurate picture of what’s causing negative feedback from customers, Quality Assurance teams can increase their strategic value to the organization significantly.” They can now shift their focus on “conducting targeted deep dives to determine where training gaps exist, uncover actionable insights, identify process and policy improvements, and enhance overall quality like never before.” With three simple steps, CX teams can elevate their Quality Assurance strategy to drive business success.

Step 1: Map Out Support Organizational Structure

A customer service organizational structure refers to the hierarchical framework and arrangement of roles within a customer service department or team. It outlines how positions and responsibilities are organized, defining reporting lines and communication channels. 

This structure typically includes roles such as customer service representatives, team leads or supervisors, managers, and may extend to include specialized teams like technical support or escalation departments. The structure aims to streamline operations, facilitate efficient customer interactions and support, and ensure effective coordination and collaboration among team members. It helps distribute tasks, establish accountability, and promote smooth communication to deliver high-quality customer service consistently.

“When CX teams want to transition to this vision of shifting their QA teams to a more data analyst role,” shared Prathipati, “it’s important that they have a clear understanding of their current state. I recommend mapping out your support work structure. A support work structure is where there are not only just Tier 1, Tier 2, and Tier 3, but there are also different lines of business.” For instance, “an Airbnb may have a line of business for renters and another line of business for hosts. But they also may have a line of business for the people who host Airbnb experiences. Each of these lines of business has tiers like Tier 1, Tier 2, and Tier 3.”

Step 1 Worksheet: The Auto QA Playbook

Step 2: List Current KPIs Used

This next step is crucial in enabling you to identify and prioritize the metrics like CSAT, AHT (average handle time), or TTR (total time to resolution) that matter most to each line of business and within each tier. “You don’t want to include something about SLAs (service-level agreements) unrelated to agents here, but if it's an SLA related to agent performance, amazing. But if it's first response time, that's a reflection on volume and staffing; you don’t want to include that here,” detailed Prathipati. “We often see some metrics that are not relevant to all the groups, and that's an important thing to note.”

Here’s an example: 

Step 2 Worksheet: The Auto QA Playbook

Lyn Miller-Bright, Director of Workforce Management and Quality Assurance at Zendesk, agrees that not all metrics are of equal importance. “While some metrics, like FCR, may be crucial for Tier 1 because they handle a wide range of tasks, they may not be as critical for Tier 2, which deals with a narrower range of responsibilities.” In other words, different tiers prioritize different metrics based on their specific roles and objectives.

“For the sake of this exercise,” said Prathipati,  you want to focus on the metrics impacting your frontline agents (the people speaking to customers) so you can make better decisions about how you manage your teams, your BPOs and how you do root cause analysis on how you coach. This exercise is ideally something you can do if you have a data person on your team or a more data-savvy person, or a metrics-oriented person.”

Step 3: Identifying Trusted Metrics vs. Misleading Metrics

In our final step, you want to understand the trust level that your teams have in metrics such as CSAT since customer satisfaction scores are a combination of support, service, product, and/or other market feedback. And like any voluntary-response survey, they’re subject to problems like sampling and response bias. 

Step 3 Worksheet: The Auto QA Playbook

Take this all too common, real-world example:

A customer leaves a low CSAT score immediately after a lengthy call with a customer service agent. The reviewing manager looks at that low score, assumes the agent needs additional coaching, and sends them training resources to work on their soft skills.

The truth of the matter is, without additional context, that low CSAT score could have been the result of several things: an ineffective company appeasement policy, an outstanding product issue, or even prior service frustrations. In this case, CSAT didn’t reflect how (well) your agent handled the situation but reflected the customer’s larger company dissatisfaction that the agent would never be able to solve by themselves.

Therefore, CSAT can be misleading.

“We use the labeling term ‘misleading’ as an indication that the team feels that the metric doesn’t accurately reflect reality or that they have no trust in that metric.”

For example, furthered Prathipati, “When we complete this worksheet with our customers, we may discover that the total time to resolution metric may not be applicable to a particular tier. This is due to the fact that when an issue is escalated, it can often sit in the queue for an extended period of time, which can skew the average and make it an inaccurate measure of an agent's performance. Therefore, it may not be effective in coaching agents on what they need to improve.” 

Completing this exercise can bridge the gap between crucial decisions that leadership teams have to make, such as choosing “who to promote, reward, or let go, and the actual ground reality. By exposing the misleading indicators, you can make more informed decisions that can impact teams, culture, and customers positively,” said Prathipati. The ability to make better-informed decisions that align with the real-world situation ultimately leads to better outcomes for your organization.

MaestroQA Webinar Poll Results

One-dimensional metrics often lead to misinterpretations, which can have a significant impact on customer experience. “MaestroQA aims to bridge this metrics gap in a new and innovative way that empowers quality teams to be more strategic and influential in their role. With MaestroQA, people-powered quality teams can become more impactful and make a meaningful difference in how customers perceive the quality of their experience.”

The Two-Step Prioritization Technique: Shift the Vision into a Reality

Now that you have identified the challenges that misleading metrics can bring by doing steps one through three and drilled down to identify your problem statement, “it’s important to remember,” said Prathipati, “that you can’t tackle all the challenges that you uncover at once.” He recommended that you follow a two-step prioritization technique:

Prioritize #1:  Pick one team within a single tier to focus on

This team might have the most misaligned metrics, or they represent the biggest areas of customer dissatisfaction and require immediate attention to improve customer experience.

Prioritize #2: Within that team, determine which metrics are truly important

During this step, the worksheet will help you identify your metric alignment process, which metrics should be kept, and which ones can be discarded. According to Prathipati, it's also a great opportunity to discuss your ideal scenario. For instance, your team may aspire to access metrics like customer request escalations, sentiment escalation in conversations, chat-to-email transfer rates, or the number of agents tickets are handed off to. These discussions not only provide insights into what's happening at the ground level but also open up a world of opportunities, such as identifying Auto QA partners like MaestroQA, that fit into this vision.

Executing the Auto QA Playbook

This shift in vision and the ability to execute that vision has inspired some quality teams to resonate more with team names such as Performance Excellence Team or Quality as a Service Team.

“We’re not saying everyone needs to change their quality assurance team’s name,” said Prathipati, “but we're trying to highlight this real shift in mindset and how Auto QA from Maestro can create value for teams, companies, and customers, plus, help understand agent performance and manage BPOs. With MaestroQA, teams can see which BPOs are hitting performance expectations and which are not. Again, this type of real, actionable data helps make better decisions about people and their livelihood and, ultimately, customer experience.”

Speaking about quality teams…

The Role Of Team Leads And Quality Teams In Change Management

It's crucial to factor in change management. This means taking into account the role of a quality person in the future, as well as the evolving responsibilities of the quality team. 

Here's how a real-world example of how a MaestroQA customer has made adjustments in their change management.

Team Leads use MaestroQA to monitor the performance of agents by analyzing aligned metrics and incorporating them into their coaching sessions. Using the MaestroQA dashboard, Team Leads can extensively review tickets based on these newly identified metrics and take a hands-on approach to coaching, providing personalized one-on-one sessions based on the metrics to ensure their team is meeting their targets.

The quality team, on the other hand, uses MaestroQA to look at things from a slightly higher level, such as how the BPOs are performing. They also look at how multiple teams are performing across a particular line of business. 

MaestroQA allows this team to look at patterns at the BPO level or across multiple BPOs to create ideas for training, new processes, or policies. “They can determine if they want metrics they don’t have today that allow them to monitor and inspect based on what they are seeing in customer interactions.,” said Prathipati. “The QA team shifts to a more targeted role where they can be hyper-focused on an insight level. “That’s why it’s important to flush through this and adjust your change management. Technology helps, but the way teams operate today is different.”

Pick the Right Auto QA Partner 

In the previous sections; we've covered the essential steps needed to identify misaligned metrics, create a clear problem statement, and explore the benefits of redefining the roles of your QA teams. The ultimate objective is to develop an Auto QA playbook and put it into action. Now, the crucial question is, how can you choose a trusted partner to help you bring things to fruition and doesn’t overpromise on AI

Auto QA Playbook: Design the Vision

“This is such a massive shift, so finding a partner who is going to keep it real, who is an expert in the Auto QA space, and can help you execute the vision that you have defined, this is the type of partner you want to seek out,” said Prathipati. 

If you would like to learn more about what MaestroQA can do for your business, please request a demo today.

Previous Article

Mastering Customer Interactions in the Age of DSAT

The Essential Guide to Chatbot Quality Assurance: Ensuring Excellence in Every Interaction

Navigating AI Implementation Strategy in Customer Experience: Risks and Strategies

Elevating Call Center Performance with Six Sigma and MaestroQA

Elevating Business Excellence Through Non-Customer-Facing QA: A Strategic Imperative

Elevating Trust and Safety through QA: How TaskRabbit Sets the Standard

Unlocking Superior CX: The Bombas Blueprint for Quality and Coaching

Unleashing the Power of Customer Conversations: Top 6 Tech Trends Revealed at the CX Summit

Important Factors to Consider when Exploring Sentiment Analysis in Customer Support QA: A CX Community Discussion

Agent Empowerment: 5 Tactics for Customer Retention from Industry Leaders

Mastering Agent Onboarding: Quality Assurance Lessons from ClassPass

Driving Business Impact with Targeted QA: Insights from an Expert

The Transformation of QA: Driving Business Results - Key Takeaways from MaestroQA’s CX Summit

How Angi Unlocked Growth and Continuous Improvement with QA

De-Villainizing QA Scorecards with Hims & Hers Customer Service

How to Revamp QA Scorecards for Enhanced Quality Assurance

The Art of Outsourcing Customer Support: Lessons from Stitch Fix's BPO Partnership

Writing the Auto QA Playbook & Transforming Customer Support

Advancing Customer Service Metrics with AI Classifiers

MaestroQA Named One of Comparably’s 2023 Best Workplaces in New York for the Second Consecutive Year

How to Maximize Call Center & BPO Performance | MaestroQA

MaestroQA Named on Comparably’s Best Workplaces in New York

CX Strategy: The Future of AI in Quality Assurance

Elevating Customer Satisfaction with Visibility & Coaching

Champion-Challenger Model: Improve Customer Service In BPOs

5 Key Strategies to Supercharge Your BPO Partnership

How Customers Collaborate with Their BPO Partners Today

Kick Start Your Customer Service BPO Partnership Successfully

BPO Call Centers: Best Practices for Quality Assurance

Empathy in Customer Service: Everything You Need to Know

Call Calibration: What is It & What are the Benefits?

Increase QA Team Alignment with Call Calibration & GraderQA

Measuring An Organization's 3 Ps: People, Process and Product

How to Onboard Your Customer Service Team to a New QA Program

Average Handle Time (AHT): How to Calculate & Reduce It

Should You Have Dedicated Quality Assurance Specialists?

The Top 4 CX Books Recommended by Our QA Community

How Top eCommerce Brands Ensure Exceptional Customer Service in a Remote World

21 Key Customer Experience Definitions for QA Professionals

5 Key Components of a Remarkable Customer Service Experience

Customer Service Management 101: Everything You Need to Know

A Guide to Customer Service Quality Assurance Programs

The Ultimate Guide to Improving First Call Resolution (FCR)

The Key to Customer Service Coaching Is More Data (and Fewer Opinions)

How to Refresh Your Call Center Quality Monitoring Scorecard

How to Update Your QA Scorecard

The 9 Customer Service KPIs Needed To Improve CX

3 Ways to Test Your Call Center Quality Assurance Scorecard

Leveraging Customer Sentiment to Improve CX in Call Centers

This Is What an Effective Customer Service Coaching Session Looks Like

What is DSAT and 5 Steps to Improve It

Customer Experience Management and Quality Assurance Jobs

How Deeper CX Analytics Lead to Better CSAT | MaestroQA

Achieving Effortless Customer Experiences (CX) with QA

How to Create an Omnichannel Call Center Quality Assurance Scorecard

Beyond Low CSAT Scores: Finding the Root Cause of Poor CX

Customer Service Coaching 101: Improve Agent Performance

Build the Ultimate QA Scorecard Process for Email and Chat

MaestroQA's Aircall Integration: Bring Your Calls to Life

Call Center Quality Assurance with Zola and Peloton

Auto-Fail in Call Center QA: What It Means and When to Use It

Why Poor Agent Experiences Happen (and How to Fix Yours)

20 Call Center Coaching Tips to Boost Agent Performance

Why Getting Buy-in for Quality Assurance is Essential

Setting Up a Grading Cadence for Your QA Scorecard

How to Avoid Bad Customer Service as you Scale your Business

Building a New Call Center Quality Assurance Scorecard

What CX Leaders Need to Know About Ecommerce Industry Trends

Call Center Cost Per Call: How to Calculate & Reduce It

6 Tips to Automate Your Customer Service Management Process

Quality Assurance and Training with Seismic Learning & MaestroQA

11 Customer Service Training Ideas and Skills for Your Agents

The Past, Present, and Future of Quality Assurance

고객센터 품질관리를 위한 QA전문가가 꼭 필요할까요?

Improve CSAT Scores: Understanding Your Experience Blindspot

What CX Leaders Need to Know About Security and Compliance

Streamline Your Call Center's QA Program With 4 Key Features

How Customer Experience Teams Can Impact a Company's Brand

Customer Loyalty vs. Customer Retention: Which Matters More?

Empathy & Authenticity: Customer Service Skills to Improve CX

How High-Performing CX Teams Build Accountability

Five Questions to Jumpstart your QA Scorecard Research Process

How to Improve Call Center Agent Performance: 6 Key Tips

3 Ways to Improve Your CSAT Score through Quality Assurance

3 Strategies on How to Increase Customer Loyalty

A Guide to Net Promotor Score (NPS) for Customer Service

Reasons for Call Center Attrition Rate and How to Reduce It

Voice of the Customer (VOC): A Guide for Great CX Teams

What’s Really Behind Your CSAT Scores? Diving Deeper

How CX University Improves Brooklinen’s Agents Performance

Understanding Customer Effort Score (CES) & How to Measure It

Improving the Customer Experience with DSAT Scores

5 Tips for Customer Service Coaches in Call Centers

Your Most Important CX Metric Is Your QA Score - Here's Why

How Agents Can Make the Most of Customer Service Coaching

How to Grade Customer Service Calls

Why Top-Performing CX Teams Focus on Workforce Engagement

Improve Customer Satisfaction with a CX Quality Management Program

How to Calculate CX Quality Assurance Scores

Customer Service Training and Quality Assurance – How Lessonly and MaestroQA Close the Loop