How can I set a distribution of human oversight and auto publishing?
Required Feature Flags
The following feature flags and permissions are required to use this feature:
Feature Flag | Technical Name | Description |
Auto Work Queues |
| Enables Auto Work Queue templates |
Auto-Publish Evaluations |
| Enables the human-oversight vs auto-publish distribution slider |
SmartScore |
| Required for the automated scorecard the queue uses |
Required Permissions:
Manage Work Queue Templates (
admin.assignmentbuilders.configure) — to create or edit Work Queue templates and configure the distribution
Introduction
When setting up a work queue using an automated scorecard, you have the option to:
Automatically publish evaluations, allowing for 100% automation, or
Assign auto-scored evaluations for human review, which are then manually published.
This article explains how to use the percentage slider to set a distribution between human oversight and auto-publishing, giving you more control over how your evaluations are managed.
Settings
Go to Evaluate > click Auto Work Queues > click Create assignment rules.
When setting up or editing a work queue where the auto-publish feature is available, you’ll see options to determine whether human oversight is required:
Yes, publish without oversight:
100% of your evaluations will be automatically published by our copilot.
No, require human oversight:
Allows you to set a percentage of evaluations to be assigned to evaluators for review.
When you choose to require human oversight, a % slider will appear.
Use this slider to specify the proportion of evaluations to assign to evaluators for manual review. The remaining percentage will be auto-published by our copilot.
For example, setting the slider to 30% means that 30% of evaluations will require human review, while the remaining 70% will be auto-published.