Human review

In addition to offline and online evaluations, Autoblocks provides a path for adding human review to your test results.

Human Review Scores

To get started with adding human review to your test results, Autoblocks requires that you create human review scores for your organization. Scores are setup by the user and specify what types of human review can occur.

You can configure a binary or range scores.

• Binary scores are a simple yes or no condition. For example a user may want to create a certified-by-professional score that is either passed or failed.
• Range scores require a score and a threshold to determine if the score is passing or not. For example, a user may want to create a confidence-score score, where any score greater than 0.5 is considered passing.

Start configuring human review scores

Adding human review to singular test case

After configuring scores, you will now see controls in the test run results page allowing you to enter a human review for a single test result.

Human review session

You can also review multiple runs and test results at once with a human review session.

To do so, click the "Human review" button in the top right of a test run. This will open a new page where you can select what runs you want to review, and a stepper to walk through each test case and their respective output.