Every grader will have a different perspective when it comes to grading performance as one grader's 5/5 may be another grader's 3/5. This can cause agents to receive inconsistent feedback, and ultimately diminish the effectiveness of your QA scores.
By setting up calibration sessions, you can help align your team's grading guidelines to make sure that agents are receiving consistent and helpful feedback.
Creating a Team Calibration Session
First, filter and find your desired tickets to calibrated on. The best method to do this is to head over to the Tickets page. On the Tickets page, identify tickets (graded or ungraded) where graders have proven to grade inconsistently on.
After selecting a series of tickets, click on "Create/Add to Assignment".
This will bring up a form where a user starts by providing the assignment a name and defining the assignment type.
Best practice is to select the "Team Calibration" assignment so that this assignment can easily be found on the Home page as well as be accounted for properly.
In addition, it doesn't hurt to also include the word "Calibration' in the title of the assignment as well.
After setting up the name and assignment type, it is now time to specify what users you want to share this Calibration with.
Always double check that users who are supposed to calibrate on the ticket have:
Access to the agents on the tickets shared
Access to the Rubric you plan to use to calibrate on
The proper Role Permissions to conduct a calibration
After you are finished, you can always confirm if you have configured the Calibration properly by checking out the summary at the end.
Getting to the Calibration
Calibration sessions can be found on both the Home page underneath your grading assignments or on the Ticket Reviews page.
Once you have received the ticket(s) to be calibrated, you can start calibrating via two methods:
First method, is to go into the ticket review and click on the ticket to calibrate. Then click on "Add Scores".
Second method, is if the ticket is ungraded, you can hover the ticket and click "See Score".
Either way, you will be brought to the screen shown below.
Here is where teams should decide on:
Who the ticket applies to
What is the proper rubric to use
The grading process from there is similar to grading a regular ticket. Once you are done calibrating, don't forget to save the calibration. In the case where someone accidentally calibrates incorrectly, you can always delete the score as well.
For the grader handling the Final Calibration, repeat the same process as if you were calibrating regularly, except select Final Calibration.
The view will be the similar, but you now can see the initial grade as well as other calibration grades. Grade the ticket as usual, and then submit the score for the Final Calibration.
Note: Final Calibration scores do not replace the original graded score.
All Graders/Individual Calibrators will only be able to see the grades they submitted and the Final Calibration score. Once again, please keep in mind that the Final Calibration will not replace the original grade of the ticket.
Once the Final Calibration is complete, Admin with access to view the Final Calibration will have the view above. Here you can compare everyone's Individual Calibration answers in addition to the Final Calibration.
For larger teams, note the option to select specific Graders to compare to the Final Calibration for a much cleaner experience.
If you have any issues with users being unable to see grades, double-check your configuration by going to Settings > User Roles > Role Permissions. You can learn more about User Roles by clicking on the button below!
Common Questions About Calibrations
Do Team Calibration scores count towards QA scores?
Calibration scores do not contribute towards QA Scores.
Do Team Calibrations count towards QA Count?
Calibrations scores do not contribute towards QA Count.
How do I export calibration scores?
Export scores as per usual, and then while filling out the form, be sure to check [Include calibration scores]. This will then include calibration scores in the export.