How can I set a distribution of human oversight and auto publishing? – EvaluAgent

Introduction

When setting up a work queue using an automated scorecard, you have the option to:

Automatically publish evaluations, allowing for 100% automation, or
Assign auto-scored evaluations for human review, which are then manually published.

This article explains how to use the percentage slider to set a distribution between human oversight and auto-publishing, giving you more control over how your evaluations are managed.

Settings

Navigation prompt

Go to Evaluate > Click Auto Work Queues > Create assignment rules

When setting up or editing a work queue where the auto-publish feature is available, you’ll see options to determine whether human oversight is required:

Yes, publish without oversight:
100% of your evaluations will be automatically published by our copilot.
No, require human oversight:
Allows you to set a percentage of evaluations to be assigned to evaluators for review.

When you choose to require human oversight, a % slider will appear.

Use this slider to specify the proportion of evaluations to assign to evaluators for manual review. The remaining percentage will be auto-published by our copilot.

For example, setting the slider to 30% means that 30% of evaluations will require human review, while the remaining 70% will be auto-published.

Introduction

Settings

Related articles