EvaluAgent CX: Adding Automated Line Items to a Scorecard / Building a blended scorecard

Prerequisites

To complete this help guide, you must have the following: 

• EvaluAgent CX enabled on your account
• Permission to Manage Scorecards
• At least one "Active" Automated Line Item within the Topic Manager

Step 1: Read before continuing

Before reading this guide, it is worth familiarising yourself with the following user guides that covers:

How do I create a new Quality Scorecard

Adding an Automated Line Item to your scorecard is very similar to adding a manual line item, as described in Step 3 of the above guide and you may be able to proceed by yourself without reading the rest of this user guide.

How to add an automated line item to a scorecard

1. From within your Draft Scorecard, click the Add Line Item option to reveal the Add Line Item Popup

   
   
   

2. From the popup model, select the "Automated" tab

   
   
   

3. Click into the "Topic" dropdown to reveal a list of your "Active" automated line items which can be added to your scorecard.

   
   
   Please note: Depending on the technology type of your Automated Line Item (AI or Logic) you may see slightly different fields in your view.
   
   For example, if your Automated Line Item uses AI, you will see a preview of the GPT Prompt, Scoring Templates and Guidelines that are being used however these cannot be edited from within the Scorecard Builder. 
   
   

4. Complete the remainder of the form:

    

   1. Title = This is unique to the Line Item on your Scorecard and can be changed if you wish.
   2. Allow Line item Feedback = Decide if the evaluator can add Feedback to this specific line item.
   3. Categories = Assign any categories to the line item. Categories can be used to filter reports and identify the performance of line items that have been categorised under a specific heading. For example, you may wish to understand the performance of all line items categorised as Risk. Line item categories can be configured in Quality Settings.
   4. Set line item as an Auto-fail = Decide whether or not the line item is an auto-fail line item whereby the evaluation FAILS if this line item is marked as a "fail" during evaluation.
   5. Weighting: If this is a weighted scorecard, then you will need to add a weighting for the line item. Don’t worry about making the weighting of all line items add up to 100% at this point. When you save the scorecard, the system will sum the weightings for all line items and then divide each individual line item weighting by this sum to derive a percentage weight for each line item. In this way, the sum of the system-derived percentages will always add up to 100%. Of course, if your scorecard has percentages already assigned and you’re comfortable that they all add up to 100%, just add each line-item percentage as an integer into the field.

5. Once you're happy with your selections, click Add line item and you'll be taken back to the scorecard screen where you can continue adding line items.

   

Editing/Deleting Automated Line Items from Scorecards

Editing an Automated Line Item takes place in two places within the EvaluAgent App.

I want to edit an Automated Line Item I've just added to my Scorecard (eg. Weightings/Categories etc.) 

Once a scorecard is published and becomes active, to ensure reporting integrity, it is not possible to edit most parts of the scorecard. By the same token, inactive scorecards that have evaluations saved against them cannot be edited.

This remains true for Automated Line Items, however, when the Scorecard is in "Draft" mode, you can edit the following from within the Scorecard Builder by clicking the EDIT button next to a line item. The opens a model where you can edit: 

1. Title
2. Allow Line item Feedback
3. Categories
4. Set line item as an Auto-fail
5. Weighting

If you wish to delete an automated line item from your scorecard, press the "Delete" button instead.

I want to edit the underlying logic of an Automated Line Item

If you wish to edit the underlying logic or guidelines of a Topic, this is done from within the Topic Manager. A user guide for creating and managing Automated Line Items can be found here. If an Automated Line Item is updated within the Topic Manager, these changes are automatically applied to your existing Line Items as Scorecards will always use the latest Active version when deciding on the outcome. 

To edit the underlying logic/guidelines/prompt, you must have the SmartScore AddOn and access to Topic Manager. 

We therefore recommend if you are materially changing the guidelines, you create a new line item and scorecard.