3 Ways to Test Your Call Center Quality Assurance Scorecard

No matter if you’re putting together your first QA scorecard, or updating an old scorecard to reflect changes in brand values or policy, properly testing your new QA scorecard is a great way to ensure a successful rollout. 

‍

Through user acceptance testing, and by involving stakeholders in your testing phase, you build up trust and gain buy-in from the CX team—the very people who will use and benefit from these scorecards the most.

After helping teams reformat and create new scorecards, we’ve identified 3 best practices making sure your scorecard test runs smoothly and leads to amazing results 📈

‍

Assemble a Small User Group To Help Test

One of the best ways to test a QA scorecard is to involve multiple folks from your call center/support team that will help you carry out the test. 

When selecting team members, look for stakeholders who are familiar with your CX goals and are able to provide constructive feedback—you’ll want a good mix of agents, graders, and managers who’ll be using these scorecards on a daily basis.

Once you have a team of testers in place, it’s time to set up a testing structure for your scorecard that fits your test group.

If it’s your first time using a scorecard, graders can immediately begin using it to assess all tickets during the test period. However, if you’re replacing an existing QA scorecard, you will need to decide on how many tickets should be graded with the new rubric vs. the old rubric.

This is known as an A/B test—we cover it in much greater detail in our QA scorecards ebook, but the general gist is this: making only one change at a time, apply that change to half of your new tickets, while grading the other half with the old scorecard. This serves as a benchmark to compare the new scorecard against, and allows you to attribute any positive (or negative!) change to the singular change you made in your new scorecard.

Remember to keep agents in the loop, too. Reassure them that any tickets graded with the new rubric are for testing purposes only and will not influence their KPIs.

Make It Easy for Team Members to Share Feedback

It’s one thing to test your scorecard—it’s a whole other thing to receive (and implement!) the feedback you receive throughout the process.

Having a transparent, open, and honest feedback process is critical to ensuring that the new scorecard hits your goals. 

We recommend creating a dedicated Slack channel or shared document to collect real-time feedback from graders.

Your team can quickly and easily post anything about the new scorecard that seems unclear, time-consuming, or misaligned with your values and policies. 

Calibration sessions—meetings that get all of your graders talking about the new scorecard—are another great way to solicit feedback and identify potential issues.

To maximize the impact of each calibration session, use your new scorecard to grade one or more tickets prior to the meeting. 

Compare your grades to those of other graders and then use the meeting as a forum for overcoming misalignment. ‍

Scribd used calibrations sessions and GraderQA (our latest feature!) to get their graders aligned on their scorecards.

‍

Test Your QA Scorecard

Now...it’s time to test! 

We’ve seen tests successfully run for 3-4 weeks, but every team is different. 

But what are you supposed to test for? 

The primary objective here isn’t testing for a specific amount of time—it’s making sure that your new scorecard highlights critical data points about the support experience so that your team can start understand more about their experience blindspot. Ask your team these questions during the testing phase:

• Are we highlighting critical areas of improvement at the team level? And at the individual level?
• Is this scorecard surfacing process improvements that can help improve our (insert team’s target metric here)?
• Are agents able to easily interpret their results from it?
• Is this scorecard user-friendly? 
• Is it easy/efficient to grade with it? 
  

A tip to get you started—workflow automations in MaestroQA can accelerate the testing process by simplifying the assignment and grading of tickets—meaning our scorecards will always check the box on the last question for you 😎

If you have other questions about QA scorecards, look no further than our Ultimate Guide to CX QA Scorecards! We’ve combined the examples and experience from working with hundreds of support teams into this comprehensive guide.