Autoblocks CLI Commands

The links below will take you to documentation on each subcommand of the Autoblocks CLI:

  • testing - Interact with and run your Autoblocks test suites

testing

Interact with and run your Autoblocks test suites.

exec

Execute any command that runs Autoblocks test suites. For example:

npx autoblocks testing exec -- poetry run start
npx autoblocks testing exec -- python3 my_test_script.py
npx autoblocks testing exec -- npm run start
npx autoblocks testing exec -- tsx my_test_script.ts

Options

  • Name
    --api-key
    Type
    string (required)
    Description

    Your Autoblocks API key. This can also be set via the AUTOBLOCKS_API_KEY environment variable. If running tests locally, make sure to use your local testing API key from the settings page.

  • Name
    --message
    Type
    string
    Description

    A description for the test run. For example:

    npx autoblocks testing exec -m "my first run" -- npm run start
    
  • Name
    --port
    Type
    number
    Description

    Choose the port of the internal server the CLI runs to collect test results. The default is 5555. For example:

    npx autoblocks testing exec --port 5000 -- npm run start
    
  • Name
    --exit-1-on-evaluation-failure
    Type
    boolean
    Description

    If set, the CLI will exit with code 1 at the end of the run if the score of any evaluation in any of the test suites does not meet its threshold. The default is false. For example:

    npx autoblocks testing exec --exit-1-on-evaluation-failure -- npm run start