Note: This article will help walk through the new Final Calibration view. This feature is currently not available for all customers and is in limited availability. If you are interested in testing this feature out, please chat in!
What is a Final Calibration?
A Final Calibration is the final step of a Team or Benchmark Calibration workflow. This process involves a Final Calibrator creating a Final Calibration score. The Final Calibration score will then serve as the benchmark for how the ticket should have been graded, and other individual calibrator responses will be compared to this benchmark.
The creation of a Final Calibration will also automatically calculate a Calibration Alignment score per each individual calibrator.
What is Calibration Alignment?
Calibration Alignment is used to help determine how aligned a specific calibrator or collective group of calibrators are aligned with a Final Calibration.
This uses the same calculation process as Grader QA, and to learn more about how the alignment score is calculated, check out this article here.
Conducting a Final Calibration
Creating a Final Calibration score is the same workflow as conducting an Individual Calibration. Answer each criteria as you would creating a regular QA score for the ticket.
While creating the Final Calibration score, you will notice that under each criteria, there is a table labeled "Calibrator Responses".
In the table below, it also groups together calibrator responses and provides a visual view, in form of a stacked bar chart. If a criteria is selected, it will list out next to each criteria:
How many times the option was selected
How many times that option was selected out of all other choices (shown in percentage)
Hovering over a specific segment of the stacked bar chart, will also show the same details.
Note: These colors are pre-set when conducting a Final Calibration and cannot be customized.
By clicking on "Show More" next to an Option, this will expand a table underneath the option that will show all users who answered the calibration with that option.
Note: Criteria Alignment and Weight will NOT populate until after the Final Calibration is created.
Viewing Alignment Scores
For Non-Admin Users
Viewing Your Own Individual Alignment Score
On the See Scores page, right before going to view the Final Calibration, you will be able to see the Alignment score for your specific individual calibration.
As a non-Admin user (Grader or Manager roles), you will only be limited to see your own alignment score underneath the "Alignment" column.
In the Final Calibration View
To gain further context into your alignment score, you will need to have permission to view the Final Calibration. If that permission is granted, you will see a Final Calibration table where you can click on the Final Calibration to view the results.
While viewing a Final Calibration, you will be able to see how each criteria was weighted and what your alignment score was per criteria. It is only possible to see your own alignment or the overall average alignment.
To filter for your own results, make sure to only have your own user profile selected in the selector highlighted in this screenshot.
For Admins and Limited Admins
Viewing All Individual Alignment Scores
On the See Scores page, Admins and Limited Admins will be able to see alignment scores related to users other than themselves. These would be available underneath the "Alignment" column.
In the Final Calibration View
When in the Final Calibration View, as an Admin you will be able to see the Criteria Alignment and Weight for all users who have participated in the session.
You can further drill into specific users by clicking on the "Grader" button and select only the specific individual calibrators you would like to see responses for.
When filtering down from this view, you will also dynamically update Criteria Alignment to represent the average alignment score from users selected. If only one user is selected, the Criteria Alignment would just display the individual user's alignment with the criteria.