By following the navigation prompt, you'll be directed to the Manage topics screen where you can see all topics available to your organisation and those provided by EvaluAgent.
To create a new GenAI-based line item topic, open the topic builder by clicking on the +New topic button.
From here, options for technology are presented, allowing you to choose between Text Analytics and GenAI;
- Text Analytics: Uses Evaluagent’s rules engine to determine whether queries (words and phrases) are present or absent in a conversation.
- GenAI: Uses a large language model (AI) to judge whether or not context or intent is present or absent in a conversation.
Select GenAI.
Then, options for the type of topic are presented, allowing you to choose between Line Item and Insight topic;
- Line Item: Line Items are added to a scorecard and then run during the evaluation process, presenting the results to the evaluator for review.
- Insight topic: Insight topics are run across all conversations as they are imported, and the results are immediately available to the evaluator to filter and organise conversations before evaluating.
Select Line Item.
The line item topic builder will slide into view, ready for you to begin populating it with details that make the line item.
The image below is an example of a GenAI-based Line Item topic configuration.
Name: You will use the name to identify the line item in your list of topics, and when creating scorecards, keep it brief and descriptive.
Description: Expand the purpose of the line item in the description. Evaluators can read this to gain an understanding of whether or not they want this line item on the scorecards they create.
Theme: The theme is used to group data on reports. Use the same theme name if you want all GenAI line items displayed together on the report. Alternatively, find a theme that best fits where you want it on the report.
Scoring conditions: The scoring conditions section defines how the line item will be scored when evaluating a conversation. We will define the prompt and scoring template in a GenAI-based line item.
GPT prompt: Write your (AI) prompt here. For more guidance on writing prompts, see SmartScore Top Tips: Prompts, Line Items and Guidelines.
Scoring template: Select the scoring template appropriate for scoring the line item.
Example of a 2-point scoring template:
Pass: Reflecting on the prompt you have written for the line item, write an explanation for a positive outcome from the prompt.
Fail: Also reflecting on the prompt, write an explanation for a negative outcome from the prompt.
N/A: If your line item needs the not applicable option, write an explanation for why this might apply.
To add this line item to a scorecard, see the following help articles.
Comments
0 comments
Article is closed for comments.