Blog Home

AllTop Stories
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Quality Assurance

Increase QA Team Alignment with Call Calibration & GraderQA

October 20, 2022
0 minute read

What is the foundation of a robust quality assurance program? It starts with well-defined CX standards for your organization. For a fantastic customer experience, well-defined CX standards represent the characteristics and process your customer-facing teams should adhere to and excel in. 

Two Key Workflows To Increase QA Grading Team’s Alignment

Quality assurance (QA) is crucial to ensure that your agents maintain this standard set out by the organization and give them opportunities to improve their performance. The CX scorecard is the cornerstone of any successful QA program, but one of the often overlooked aspects of QA is whether or not your QA team is aligned with how they interpret the standards in the scorecard. If your rubric is perfect, but your graders interpret it differently and have different scoring strategies, how much can you trust the results of your quality program? What insights can you draw if your graders are not using the same standards? 

This article will provide the framework for two proven workflows that help measure and improve alignment amongst your grading team! 

There are two widely accepted workflows for measuring and improving alignment:

  1. Call Calibrations
  2. Grade the Grader (in the MaestroQA platform, we call this GraderQA)

Both of these workflows ensure your QA team or team leads are aligned with how they interpret the CX standards in the scorecard. As we dig deeper into these workflows, they also have stark differences intended to drive different results. Let’s define each process before deciding which workflow suits your team: 

What is Call Calibration? 

Call calibrations allow your QA team to grade agent performance and collaborate on the same ticket. The purpose of call calibrations is to have your team grade (or calibrate) the interaction individually while utilizing the same scorecard. 

Performing Call Calibration Sessions for Quality Assurance

After each member of the QA team has submitted their scores, the team holds a call calibration session to discuss why they landed on the scores they did. Your team will probably disagree on what the “right” grade was - and that’s okay! The purpose is to have a constructive conversation to collectively decide how to grade. 

Once the call calibration process has been completed, Maestro will generate an “alignment score” to showcase how aligned each QA team member is with the final calibration score. The higher the alignment score is, the more closely aligned they are to the agreed-upon CX standards. 

Tips

  • Designate one person to be the final calibrator. This person will be the final decision maker and will submit the Final Calibration. This is your source of truth to measure alignment against! There is a lot of back and forth when going through the call calibration process, so it’s important to designate someone as the decision-maker. 
  • As you become more aligned, select harder, more complex, or more ambiguous tickets so you can continue to drive value from these conversations and raise the bar. 
  • Consider recording the sessions to share within your CX organization to build a culture of transparency around how you think about and talk about quality.  

Other benefits of Call Calibration sessions

In addition to improved team alignment, call calibration sessions can help refine training processes and identify issues, including soft skills and technical behaviors in customer interactions. Ensuring agents meet evaluation standards leads to a more consistent customer experience.

How often should my team do Call Calibrations?

If your quality program is new, we see teams go through the call calibration process as frequently as once a week! Once your call calibration process runs smoothly, and alignment is high, consider moving to once a month and focusing on more complex tickets. When you launch a new rubric, it’s a good idea to increase frequency again until all graders are comfortable and aligned with the new criteria. 

MaestroQA Grade the Grader (aka GraderQA) 

The key difference with this workflow is that it’s designed to provide feedback to an individual grader on their grading performance rather than engaging in a team-wide discussion about the standards. Similarly, there will be a member of the team designated as the “source of truth” (in Maestro, we call this a Benchmark Grader. In your program, this should be your most experienced grader or perhaps your QA Manager. Whoever is an expert on your grading standards!) who grades a ticket that another grader already completed in AgentQA. 

The goal is to provide feedback to the original grader on how they answered the questions and shared feedback with the agent. Ultimately, the work done by the Benchmark Grader is then shared back with the grader so they can review their feedback about how to grade more consistently with the CX standards set out. 

Tips: 

  • Graders want feedback as much as your agents do, as it helps them to perform at a higher level and ensure they’re keeping within your team's standards. If your team has the bandwidth, providing bi-monthly feedback on a few graded tickets should give them confidence moving forward that they are grading consistently to the team standard. 
  • Focus on tickets where the grader left the agent with qualitative feedback (comments or selecting additional checkboxes). As the “Benchmark Grader,” this will give you ample opportunity to share feedback with the grader and see how they deliver feedback to your agents. 

How often should my team run GraderQA?

Similarly to call calibrations, we typically see teams run GQA at least a few times a month to share regular feedback with your graders. Depending on internal bandwidth, it’s a good idea to include consistent feedback for your graders alongside the feedback shared for agents, so everyone can continue performing at the expected organizational standards. 

Which workflow is right for me? 

Ultimately, it depends on your ideal end state! Let’s walk through a few scenarios to help demonstrate the use cases of each:

  1. Your agents benefit from QA feedback, and you want to provide that same level of feedback to your graders to improve their grading skills.  

Our Suggestion: MaestroQA GraderQA! This gives you the tools to give qualitative feedback to individual graders. 

  1. You have a few graders that you think are underperforming. Their QA scores are high, or maybe they take a long time to grade a single ticket and are not completing their weekly assignments. Any of these are indicators that a grader may need some more help. 

Our Suggestion: GraderQA! This gives you the tools to grade these graders’ tickets, uncover which criteria they are not grading correctly, and coach the grader toward success. 

  1. You just launched a new rubric. The scoring strategy changed, and a few questions were added that focused on customer satisfaction. 

Our Suggestion: Definitely Call Calibrations! This will allow you to see what questions graders have, which areas of the rubric are unclear, and perhaps even make changes before launching. 

  1. You just hired a class of new graders. They are getting ramped on your rubric and processes. 

Our Suggestion: Both Call Calibrations and GraderQA! Call Calibrations will be great to teach the new graders how you think about the rubric and work through customer problems. GraderQA will give you a specific alignment score, so you know how quickly your graders are ramping and what parts of the rubric they need coaching on. 

Key Takeaway

GraderQA and Call Calibration workflows are valuable tools in your QA toolkit to unlock additional insights and better align your grading team. Whether pursuing just one of these workflows or implementing both, the conversations and learnings that come from these are endless. They should help create a more robust experience for your customers and agents! 

Interested in learning more about MaestroQA GraderQA?

Request a demo.

Previous Article

Mastering Customer Interactions in the Age of DSAT

The Essential Guide to Chatbot Quality Assurance: Ensuring Excellence in Every Interaction

Navigating AI Implementation Strategy in Customer Experience: Risks and Strategies

Elevating Call Center Performance with Six Sigma and MaestroQA

Elevating Business Excellence Through Non-Customer-Facing QA: A Strategic Imperative

Elevating Trust and Safety through QA: How TaskRabbit Sets the Standard

Unlocking Superior CX: The Bombas Blueprint for Quality and Coaching

Unleashing the Power of Customer Conversations: Top 6 Tech Trends Revealed at the CX Summit

Important Factors to Consider when Exploring Sentiment Analysis in Customer Support QA: A CX Community Discussion

Agent Empowerment: 5 Tactics for Customer Retention from Industry Leaders

Mastering Agent Onboarding: Quality Assurance Lessons from ClassPass

Driving Business Impact with Targeted QA: Insights from an Expert

The Transformation of QA: Driving Business Results - Key Takeaways from MaestroQA’s CX Summit

How Angi Unlocked Growth and Continuous Improvement with QA

De-Villainizing QA Scorecards with Hims & Hers Customer Service

How to Revamp QA Scorecards for Enhanced Quality Assurance

The Art of Outsourcing Customer Support: Lessons from Stitch Fix's BPO Partnership

Writing the Auto QA Playbook & Transforming Customer Support

Advancing Customer Service Metrics with AI Classifiers

MaestroQA Named One of Comparably’s 2023 Best Workplaces in New York for the Second Consecutive Year

How to Maximize Call Center & BPO Performance | MaestroQA

MaestroQA Named on Comparably’s Best Workplaces in New York

CX Strategy: The Future of AI in Quality Assurance

Elevating Customer Satisfaction with Visibility & Coaching

Champion-Challenger Model: Improve Customer Service In BPOs

5 Key Strategies to Supercharge Your BPO Partnership

How Customers Collaborate with Their BPO Partners Today

Kick Start Your Customer Service BPO Partnership Successfully

BPO Call Centers: Best Practices for Quality Assurance

Empathy in Customer Service: Everything You Need to Know

Call Calibration: What is It & What are the Benefits?

Increase QA Team Alignment with Call Calibration & GraderQA

Measuring An Organization's 3 Ps: People, Process and Product

How to Onboard Your Customer Service Team to a New QA Program

Average Handle Time (AHT): How to Calculate & Reduce It

Should You Have Dedicated Quality Assurance Specialists?

The Top 4 CX Books Recommended by Our QA Community

How Top eCommerce Brands Ensure Exceptional Customer Service in a Remote World

21 Key Customer Experience Definitions for QA Professionals

5 Key Components of a Remarkable Customer Service Experience

Customer Service Management 101: Everything You Need to Know

A Guide to Customer Service Quality Assurance Programs

The Ultimate Guide to Improving First Call Resolution (FCR)

The Key to Customer Service Coaching Is More Data (and Fewer Opinions)

How to Refresh Your Call Center Quality Monitoring Scorecard

How to Update Your QA Scorecard

The 9 Customer Service KPIs Needed To Improve CX

3 Ways to Test Your Call Center Quality Assurance Scorecard

Leveraging Customer Sentiment to Improve CX in Call Centers

This Is What an Effective Customer Service Coaching Session Looks Like

What is DSAT and 5 Steps to Improve It

Customer Experience Management and Quality Assurance Jobs

How Deeper CX Analytics Lead to Better CSAT | MaestroQA

Achieving Effortless Customer Experiences (CX) with QA

How to Create an Omnichannel Call Center Quality Assurance Scorecard

Beyond Low CSAT Scores: Finding the Root Cause of Poor CX

Customer Service Coaching 101: Improve Agent Performance

Build the Ultimate QA Scorecard Process for Email and Chat

MaestroQA's Aircall Integration: Bring Your Calls to Life

Call Center Quality Assurance with Zola and Peloton

Auto-Fail in Call Center QA: What It Means and When to Use It

Why Poor Agent Experiences Happen (and How to Fix Yours)

20 Call Center Coaching Tips to Boost Agent Performance

Why Getting Buy-in for Quality Assurance is Essential

Setting Up a Grading Cadence for Your QA Scorecard

How to Avoid Bad Customer Service as you Scale your Business

Building a New Call Center Quality Assurance Scorecard

What CX Leaders Need to Know About Ecommerce Industry Trends

Call Center Cost Per Call: How to Calculate & Reduce It

6 Tips to Automate Your Customer Service Management Process

Quality Assurance and Training with Seismic Learning & MaestroQA

11 Customer Service Training Ideas and Skills for Your Agents

The Past, Present, and Future of Quality Assurance

고객센터 품질관리를 위한 QA전문가가 꼭 필요할까요?

Improve CSAT Scores: Understanding Your Experience Blindspot

What CX Leaders Need to Know About Security and Compliance

Streamline Your Call Center's QA Program With 4 Key Features

How Customer Experience Teams Can Impact a Company's Brand

Customer Loyalty vs. Customer Retention: Which Matters More?

Empathy & Authenticity: Customer Service Skills to Improve CX

How High-Performing CX Teams Build Accountability

Five Questions to Jumpstart your QA Scorecard Research Process

How to Improve Call Center Agent Performance: 6 Key Tips

3 Ways to Improve Your CSAT Score through Quality Assurance

3 Strategies on How to Increase Customer Loyalty

A Guide to Net Promotor Score (NPS) for Customer Service

Reasons for Call Center Attrition Rate and How to Reduce It

Voice of the Customer (VOC): A Guide for Great CX Teams

What’s Really Behind Your CSAT Scores? Diving Deeper

How CX University Improves Brooklinen’s Agents Performance

Understanding Customer Effort Score (CES) & How to Measure It

Improving the Customer Experience with DSAT Scores

5 Tips for Customer Service Coaches in Call Centers

Your Most Important CX Metric Is Your QA Score - Here's Why

How Agents Can Make the Most of Customer Service Coaching

How to Grade Customer Service Calls

Why Top-Performing CX Teams Focus on Workforce Engagement

Improve Customer Satisfaction with a CX Quality Management Program

How to Calculate CX Quality Assurance Scores

Customer Service Training and Quality Assurance – How Lessonly and MaestroQA Close the Loop