EvaluAgent CX Testing Console: How to test Insight Topics and Automated Line Items

Prerequisites

To complete this help guide, you must have the following: 

• You must have Smart Intelligence, SmartScore or AutoQA Feature enabled on your account. 
• You must have permission to access the Testing Console. These permissions could be:
  • Build, test and manage custom insight topics
  • Manage SmartScore
  • Configure analytics. Build & test topics & auto-QA scorecards
• You must have at least 1 imported conversation available to build a datasource.

Why use the testing console?

Before applying your Custom Insight Topics or using your Automated Line Items within actual evaluations it is recommended you test your newly created topics against existing conversations within the Testing Console. This allows you to establish the accuracy of your Insight Topic or Line Item and ensure your prompts/guidelines are returning the expected outcomes (eg: pass/fail) across multiple conversations. 

If the testing console shows an unexpected result, such as marking a line item as a fail, when you would expect it to pass, you can make edit to Draft versions of Topics or create a new version of your Topic to ensure you are making things better. 

Any changes made within the testing console are saved to your Insight Topics or Automated Line Items. 

How to access the testing console?

There are three ways in which you can access the Testing Console.

1. From the Main Menu via the Robot Icon then click Testing Console
2. From the Topic Manager 

All navigational routes take you to the same place. As a result, this user guide will focus on how to use the Testing Console once you arrive there. 

How to use the testing console?

Step 1: Selecting the source of the conversation

If you have more than one integration, you first need to choose the source of conversations. This can be done by selecting one of your active integrations from the dropdown list in the top right of the screen

Step 2: Choose what Insight Topic or Automated Line Item you would like to test

You can use the dropdown to select up to 3 topics to test.

This could be testing:

• Different versions of the same Topic against themselves
• The performance of an AI Line Item vs. Logic Based Line Item
• Simply testing a few different Insight Topics against the same list of calls.
  
  
  
  

Therefore in the list you will see and be able to search among: 

• All versions of a Logic-Based Insight Topic
• All versions of an AI-Based Insight Topic
•  All versions of a Logic-Based Line Item
• All versions of an AI-Based Line Iten

If the list appears cluttered, then you can use the "Exclude Archived" toggle on the righthand side to only show active Insight Topics and Line Items within the list. 

In Step 6 we will cover what the "View" and "Edit" options mean 

Step 3: Select a datasource

Once you have chosen a topic(s) you can select the conversations you want to test against.

Please note that if you are using the Testing Console for the first time, follow the instructions provided in the section below

Creating your first "Datasource"

Step 4: Click the test button 

Depending on the number of line items, and the length of the conversations being used - this process may take several minutes to complete. We are continually working hard to improve performance in this area but in the meantime, please be patient. 

Step 5: Review the results

Once the test is complete, you can review the results to ensure the outcome is "as expected".

As seen in the screenshot above, click "view" next to a conversation you wish to review

Please note: This results screen, and what you seen when you click "View" next to each row will differ slightly depending on what it is you are testing. For example, if you are testing an Insight Topic you will see a Pass/Fail whereas an Automated Line Item will show a representation of the Line Item Outcome and associated colour. 

This will load up a Results pane, which can be navigated as follows: 

1. This area contains the result, and examples from the transcripts used to justify the outcome SmartScore has come to.
2. This navigation allows you to scroll through each of the three line items that have been tested within this conversation 
3. This navigation allows you to scroll through each of the conversations that have been SmartScored. In the example above - 5 of the conversations.

Step 6: Review and Refine

At this point, the test has been completed. In most cases, especially in the first 90 days of using EvaluAgent, you will need to refine your Insight Topic or Automated Line Item to get a more accurate result. This could be writing a more tailored prompt so our AI can account for particular nuances or broadening out the keywords and phrases to account for mistranscription.

THE VIEW OPTION 

If the Insight Topic or Automated Line Item is "Active" then you can only click the "view" button from within the Testing Console to remind yourself of what it is looking for. If you wish to amend Active Topic, you must first create a new draft version from within the Topic Manager. Please read this user guide

THE EDIT OPTION 

If the Insight Topic or Automated Line Item is a "Draft" then you will see the option to edit from within the Testing Console. This enables you to make amendments as a result of your tests, and save those edits within the existing version of the topic with revision notes. 

Useful Information:

Creating your first "Datasource"

The first time you use Testing Console, you will need to create a list of conversations, known inside the platform as a "Datasource".

You can do this in one or two ways. 

Option 1: From within the testing Console

This can be done by clicking 'Manage test-lists' link, which opens a new screen where you can select from the following options:

• Add a specific contact (conversation) using the reference ID
• Apply a random sample of contacts matching a saved filter created in the imported contacts’ screen
• Set a random filter by entering the volume you’d like to return
• Add a random sample of contacts containing specific words or phrases 

Once saved, this test list will appear in the saved test-lists table over to the right-hand side of the screen, alternatively, you will be able to access any of these saved test-list via the drop-down selection on the main test screen as shown in Step 3.

Option 2: From within Imported Contacts

Test-lists can be created within the 'Imported Contacts' screen by using any of the below filters in conjunction with the quick word or phrase search to uncover the conversations that really matter.

1. Date 
2. Agent
3. Zendesk fields
4. Survey fields
5. Conversation Insights

Please follow this user guide if you wish to build a data source from within the Imported Contacts page.