How can I set a distribution of human oversight and auto publishing?
Introduction
When setting up a work queue using an automated scorecard, you have the option to:
Automatically publish evaluations, allowing for 100% automation, or
Assign auto-scored evaluations for human review, which are then manually published.
This article explains how to use the percentage slider to set a distribution between human oversight and auto-publishing, giving you more control over how your evaluations are managed.
Settings
Navigation prompt
Go to Evaluate > Click Auto Work Queues > Create assignment rules
When setting up or editing a work queue where the auto-publish feature is available, you’ll see options to determine whether human oversight is required:
Yes, publish without oversight:
100% of your evaluations will be automatically published by our copilot.No, require human oversight:
Allows you to set a percentage of evaluations to be assigned to evaluators for review.
When you choose to require human oversight, a % slider will appear.
Use this slider to specify the proportion of evaluations to assign to evaluators for manual review. The remaining percentage will be auto-published by our copilot.
For example, setting the slider to 30% means that 30% of evaluations will require human review, while the remaining 70% will be auto-published.