> ## Documentation Index
> Fetch the complete documentation index at: https://docs.autoblocks.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> Learn about Autoblocks Evaluators and how they help you assess your AI applications.

# Evaluators Overview

Autoblocks Evaluators provide a comprehensive system for assessing the quality and performance of your AI applications. They enable you to define custom evaluation criteria and integrate them seamlessly into your development workflow.

## Key Features

### Flexible Evaluation Types

* Rule-based evaluators for simple checks
* LLM-based evaluators for complex assessments
* Webhook evaluators for custom logic
* Out-of-box evaluators for common use cases

### Integration Options

* TypeScript and Python SDK support
* UI-based evaluator creation
* CLI integration
* CI/CD pipeline support

### Rich Evaluation Capabilities

* Custom scoring logic
* Threshold-based pass/fail
* Detailed evaluation metadata
* Evaluation history tracking

## Getting Started

Choose your preferred language to begin:

<CodeGroup>
  <CodeGroupItem title="TypeScript">
    ```typescript
    import { BaseTestEvaluator, Evaluation } from '@autoblocks/client/testing';

    class HasSubstring extends BaseTestEvaluator<MyTestCase, string> {
      id = 'has-substring';

      evaluateTestCase(args: { testCase: MyTestCase; output: string }): Evaluation {
        const score = args.output.includes(args.testCase.expectedSubstring) ? 1 : 0;
        return {
          score,
          threshold: { gte: 1 },
        };
      }
    }
    ```
  </CodeGroupItem>

  <CodeGroupItem title="Python">
    ```python
    from autoblocks.testing.models import BaseTestEvaluator
    from autoblocks.testing.models import Evaluation
    from autoblocks.testing.models import Threshold

    class HasSubstring(BaseTestEvaluator):
      id = "has-substring"

      def evaluate_test_case(self, test_case: MyTestCase, output: str) -> Evaluation:
          score = 1 if test_case.expected_substring in output else 0
          return Evaluation(
              score=score,
              threshold=Threshold(gte=1),
          )
    ```
  </CodeGroupItem>
</CodeGroup>

## Core Concepts

### Evaluator Types

Different approaches to evaluation:

* Rule-based evaluators for simple checks
* LLM judges for complex assessments
* Webhook evaluators for custom logic
* Out-of-box evaluators for common use cases

### Evaluation Components

Key elements of an evaluator:

* Unique identifier
* Scoring logic
* Threshold configuration
* Metadata and documentation

### Integration Methods

Ways to use evaluators:

* SDK integration
* UI-based creation
* CLI execution
* CI/CD pipeline

## Next Steps

* [TypeScript Quick Start](/v2/guides/evaluators/typescript/quick-start)
* [Python Quick Start](/v2/guides/evaluators/python/quick-start)
* [Out of Box Evaluators](/v2/guides/evaluators/out-of-box)