How do I raise a dispute on an evaluation?
Required Feature Flags
The following feature flags and permissions are required to use this feature:
Feature Flag | Description |
Evaluation disputes | Lets agents and evaluators raise and review disputes (also called queries) on a published evaluation |
Agents can raise and view disputes | Lets agents raise disputes on their own evaluations |
Required Permissions:
View Completed Evaluations — to find and open the evaluation you want to dispute
If you disagree with how an evaluation was scored, you can raise a dispute (sometimes called a query). The evaluator or a designated reviewer takes another look and either keeps the original scores or changes them.
Step 1: Open the evaluation
Go to Evaluate > click Completed evaluations and open the evaluation you want to dispute.
You can only dispute an evaluation that has been published. If you can't see a dispute option, check that:
Disputes are enabled for your organisation
The query deadline hasn't passed
The evaluation hasn't already been disputed the maximum number of times
Step 2: Start a dispute
Click Query or Dispute on the evaluation.
This opens the dispute form, where you can challenge the overall outcome, specific line items, or both.
Step 3: Choose what you're disputing
You can dispute:
The overall outcome — for example, you believe the evaluation should be a Pass instead of a Fail. Select the outcome you think is correct.
Specific line items — pick the line items you disagree with. For each one, indicate the score you believe is correct.
You can dispute the outcome, line items, or both in the same dispute.
Step 4: Explain your reasoning
For each disputed line item, write a clear explanation in the verbatim field describing why you think the score is wrong. If you're disputing the overall outcome without flagging specific line items, you'll need to enter a reason in the main verbatim field.
Good reasons reference the conversation, the scorecard guidelines, or both. The reviewer uses your explanation to decide whether to change the score.
Step 5: Submit the dispute
Click Submit to send the dispute for review.
Once submitted, the dispute moves into the Pending status and the relevant people are notified:
The evaluator who scored the evaluation
The reviewer assigned to handle disputes
The agent (if someone else raised the dispute on their behalf)
What happens next
A reviewer with the right permission opens your dispute, looks at the original scores against the conversation, and decides on each disputed item. They can:
Keep the original score (the dispute closes as Unchanged)
Change the score to what you proposed, or to a different score (the dispute closes as Changed)
The reviewer adds their own notes explaining the decision, and you'll be notified once the dispute is resolved.
Tracking dispute status
Open the evaluation at any time to see its dispute history:
Status | What it means |
Pending | The dispute is with the reviewer and waiting for a decision |
Changed | The reviewer agreed and updated the evaluation |
Unchanged | The reviewer kept the original scores |
Each dispute on an evaluation is numbered (Attempt 1, Attempt 2, and so on) so you can see the full history.
Tips for a good dispute
Be specific — point to what was said, when, and why it should have been scored differently
Refer to the scorecard guidelines where you can
Raise the dispute promptly. There's a deadline (set by your administrator) after which disputes are no longer accepted
Keep it factual. The reviewer needs evidence, not frustration