> ## Documentation Index
> Fetch the complete documentation index at: https://docs.autoblocks.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Python Quick Start

> Get started with Autoblocks Evaluators in Python.

# Python Quick Start

This guide will help you get started with creating and using evaluators in Python.

## Installation

First, install the Autoblocks client:

```bash
pip install autoblocks-client
```

## Creating an Evaluator

Let's create a simple evaluator that checks if a response contains a specific substring:

```python
from typing import Dict, Any
from autoblocks.testing import BaseTestEvaluator, Evaluation

class HasSubstring(BaseTestEvaluator):
    id = "has-substring"

    def evaluate_test_case(self, test_case: Dict[str, Any], output: str) -> Evaluation:
        score = 1 if test_case["expected_substring"] in output else 0
        return Evaluation(score=score, threshold={"gte": 1})
```

## Using Out of Box Evaluators

Autoblocks provides several [out-of-box evaluators](/v2/guides/evaluators/out-of-box) that you can use directly:

```python
from typing import Dict, Any
from autoblocks.testing import BaseAccuracy

class Accuracy(BaseAccuracy):
    id = "accuracy"

    def output_mapper(self, output: str) -> str:
        return output

    def expected_output_mapper(self, test_case: Dict[str, Any]) -> str:
        return test_case["expected_output"]
```

## Running Evaluations

You can run evaluations using the test suite:

```python
from typing import Dict, Any
from autoblocks.testing import run_test_suite

async def main():
    await run_test_suite(
        id="my-test-suite",
        test_cases=[
            {
                "input": "hello world",
                "expected_output": "hello world",
            }
        ],
        test_case_hash=["input"],
        fn=lambda test_case: test_case["input"],
        evaluators=[Accuracy()],
    )

if __name__ == "__main__":
    import asyncio
    asyncio.run(main())
```

## Next Steps

* [Out of Box Evaluators](/v2/guides/evaluators/out-of-box)
* [Testing SDK](/v2/guides/testing/python/quick-start)