EvaluAgent CX: How to use the SmartScore Change Report

Prerequisites

In order to complete this help guide, you must have the following: 

• EvaluAgent CX Plan
• You need the permission "SmartScore Reporting" enabled. This is found under the "Reporting" heading within roles and permissions
• For the report to be populated with any data, you must have evaluated at least one conversation with SmartScore

Why use the SmartScore Change Report?

The purpose of the SmartScore Change Report is to help you monitor the accuracy of your Automated Line Items and keep track of how frequently Evaluators are overruling the automated outcome. 

By regularly reviewing the accuracy of the automation and any associated feedback from Evaluators, you may choose to review the underlying logic/prompt/guideline of the Automated Line Item in question. 

To view the SmartScore Change report, follow the steps below.

How can the SmartScore Report help me monitor accuracy? 

The SmartScore Change Report is split up into three key areas.

Reporting Tiles

This area provides you with a high-level overview of usage, and accuracy

1. • The first tile counts the total number of line items that have been scored by SmartScore over the given time period and is subject to any filters that may be applied. In the screenshot, no filters have been applied and we are looking at the last 30 days.
   • The second tile provides you with the number of line items that were initially scored by SmartScore but changed by the evaluators before an evaluation was published.
   • The third tile provides you with the level of accuracy SmartScore has achieved based on the previous two tiles.

Data Table

1. The area is your working area. It enables you to review the changed scores along with any comments which have been left by the evaluator.  By clicking the checkbox next to each row of data, you are able to move rows of data back and forth between the "Under review" tab and the "Reviewed" tab as you digest this feedback to improve the underlying logic of your Automated Line Item. 

Other features of the SmartScore change report include: 

• The ability to export the data to a CSV file, by clicking the "Export" button 
• The ability to filter the data so you can get an understanding of SmartScore accuracy by Scorecard/Line Item and Evaluator over any selected data period.

How to use the SmartScore Report to improve accuracy?

By encouraging evaluators to leave feedback as to why they changed the automated score, you are helping yourselves in improving the accuracy of SmartScore over time. Whilst the report itself does not improve the accuracy of SmartScore - information from this report should be taken into account whenever you are thinking about creating or editing an Automated Line Item.

Additionally, if you find evaluators are overruling SmartScore incorrectly then it may be useful to consider running a calibration session to agree on the expected standard