We don't like spam either. We'll only send you content that Vasu would read, and he hates reading so...it's extremely selective content.

How To Build Your First QA Scorecard

October 29, 2019

The Romans favored a gradual, structured process when it came to city-building: they identified and established a strong central area of the city (usually an agora, or marketplace), then added to that core over time by following a pre-established pattern/framework. Very cool 😎.

But before I lose everyone who isn’t a history buff, let me get to the point—building a QA scorecard isn’t that different from planning an Ancient Roman city: you start from a strong core that reflects your company values and goals and follow a framework to add to as you go.

Keep reading to learn more about Roman cities, QA scorecards, and everything in between.

What is a QA Scorecard

A Quality Assurance Scorecard is a rubric against which a QA analyst, team lead, or manager grades an agent’s interactions with a customer. These scorecards are based on a company’s values and standards for agent-customer interactions and form the foundation of the QA process.

Why You Need a QA Scorecard

Some Customer Experience/Customer Support teams that have grown to a certain size want to implement a more formal framework for giving feedback to their CX teams. 

For others, the trigger might not be size-related. Rather, these teams want to move past qualitative methods of giving feedback and reporting, and have some quantitative data to guide their coaching and training processes.

Getting the Lay of the (QA Scorecard) Land

Back to my Roman city analogy: to build a QA scorecard, start with what’s important to your company, and build from there.

I spoke to our Customer Success Managers Matt and Laura to figure out the best way to build a QA scorecard from scratch:

To which they replied:

Companies building a QA program often consider their company values first, and this makes a lot of sense – your customer-facing team is the most frequent human point of contact a customer has with your brand, so it is essential that they embody your brand’s values.

Step one in the scorecard building process is to take a scrap piece of paper or open a Google Doc (save a 🌲 ppl), write “Company Values” on the left, and fill it up with the values that drive your customer interactions.

On the right, write: Operational KPIs and Metrics.

Is First Contact Resolution an important part of your customer experience? Add that to the list. Are you aiming for your agents to deliver a certain number of interactions per hour? Pencil it in. Are there important regulations that your agents have to follow in their interactions (PII, HIPAA-compliance, etc)? Gotta have that.

To round it off, keep listening in on customer calls and speak with agents. These interactions will help you to identify the pain points both customers and agents are currently facing, as well as potential areas of opportunity to improve. Make note of these areas as well – this could become a question on your QA scorecard.

By cross-referencing these insights with the list you’ve made, you will already have gathered the main ingredients to build your first scorecard.

Mapping out your C̶i̶t̶y̶ Scorecard

Once you’ve mapped out the most important brand values, KPIs, and areas of opportunity for your team, you can take these and affix them to a framework. Here are two suggestions from our experts:

Jeremy Watkins, Director of Customer Experience at FCR, suggests a 4C framework to build your scorecard around: 

  1. Communication, 
  2. Customer Connection, 
  3. Compliance and Security, and 
  4. Correct and Complete Content.

(I know, that’s way more Cs than we promised, but Jeremy has a point.)

Each of these represents a section in your rubric, which you can build out with questions based on the list that you’ve created previously.

Another way to do it is something that Laura suggests:

These pillars are: 

Soft skills: things like tone, understanding context, empathy go here. Other companies have also added elements like humanity or use this section to suggest having unique interactions like casually swearing at a customer (with love, of course).

Issue Resolution: this section can be as simple as asking a Yes/No question: “did the agent resolve the issue for the customer?”, but can also involve a linear sliding scale (from 1 point to 5 points, for example) to better capture the nuances in each customer interaction, or to reflect a particularly technical or complex interaction.

Procedural: did the agent properly follow internal procedures? We often see checkboxes for this section, where a QA agent can easily tick off each requirement as it gets met.

Both of these frameworks are a great place to start (and there’s plenty of overlap between them!). You can also look for patterns in the notes you’ve made, identify the main categories that emerge, and use those as the primary sections of your rubric.

Gathering your Tools

Many teams start their QA process on spreadsheets. Through a complex combination of formulas, Google Sheets and reference cells, it's possible to set up a QA program where you can figure out what works for your team without paying for a tool.

But eventually, spreadsheets and formulas become unwieldy and hard to use for even the greatest Excel whizzes ✨ out there. Imagine having to manage the QA scores and analytics of hundreds of agents on a massive spreadsheet. Just the thought of maintaining a spreadsheet with 100 Excel tabs has me quaking in my chair. It’s safe to say that what works for a team of 10 does not scale to a hundred agents

Take a moment to evaluate your company’s current needs and goals, pick a tool (cough* this CX quality management software is pretty fab *cough), and get to building!

When in Rome - QA as these Companies do.

At MaestroQA, we are uniquely positioned and privileged to have insights on how hundreds of CX teams do QA. We picked three success stories to give you a little bit of inspiration:


Fullstory regards their CX team as defenders of their brand. They started the QA planning process with their product  “watchwords” like Empathy, Clarity and Bionics – concepts around which their product was built. They then built their first scorecard with the goal of validating what their CX team was already doing, and to help them to keep improving over time.

With their watchwords in hand, they crafted detailed descriptions of the experiences a customer could expect, and included them as Yes/No questions for the scorecard, with the belief that a slow, thoughtful, qualitative QA process would present more value than a quantitative QA score.

Learn more about how to defend your brand through QA here.


MeUndies are a great case study of how to maintain quality in the face of a seasonal workforce, as well as a great example of a brand voice-oriented rubric.

Their QA program allows agents to track their own growth over time and lets the team to pay attention to the needs of each agent and rally behind them if they needed it.

In their scorecard, MeUndies included parameters like brand voice, personalization, empathy, the use of macros and (get this) use of emojis. I 😍 them already.

To learn more about how they weigh each parameter in their scorecard, as well as their philosophy behind scorecard design, read this case study.


Our friends at SeatGeek started looking at a QA program to give their CSAT scores a little boost. They’re like that kid in class who gets an A and wonders out loud how they can get an A+ 🤦‍♀️ (but SeatGeek does it for their customers!).

Their first scorecard was based on their operational needs: agent thoroughness, tone, and resolution. To keep it simple, they used a linear 10 point scale on spreadsheets. 

As their CX team grew in number, more senior CX agents were incorporated into the QA team. SeatGeek realized that the time was right to implement a QA platform to automate parts of the QA workflow and improve their efficiency. 

This new-found efficiency also let them implement more parameters to measure their agent’s performance while maintaining their average Time-to-Grade.

Read about how they did it here.

Your QA scorecard could be built in a day.

While Rome wasn’t built in a day, your QA scorecard certainly can be! You’ve probably realized this while reading, but you already have the main elements of a scorecard on hand:

You’ve been listening to your customers’ needs and pain points, talking to your agents, and you already live and breathe your brand and its values. The customer examples we included above should also have driven home the point that most scorecards start with company values and operational requirements, and then grow from there.

Start off with something manageable that also embodies your company’s values and organizational goals. Complexity can come with experience and the ever-changing needs of your CX team, but for now, it’s a good idea to keep things simple.

To summarize, when building a scorecard:

  1. Start from your brand values and operational goals 🥅
  2. Speak to both customers and agents to identify pain points and opportunities to improve
  3. Choose a tool 🛠
  4. Get building 👷‍♀️👷‍♂️
  5. And keep it sweet 

Go forth and QA!

Recent Blogs