Autoblocks

Autoblocks CLI Commands

The links below will take you to documentation on each subcommand of the Autoblocks CLI:

  • testing - Interact with and run your Autoblocks test suites

testing

Interact with and run your Autoblocks test suites.

exec

Execute any command that runs Autoblocks test suites. For example:

npx autoblocks testing exec -- poetry run start

npx autoblocks testing exec -- python3 my_test_script.py

npx autoblocks testing exec -- npm run start

npx autoblocks testing exec -- tsx my_test_script.ts

Options

  • Name
    --api-key
    Type
    string
    Required
    Description

    Your Autoblocks API key. This can also be set via the AUTOBLOCKS_API_KEY environment variable, which will be read either from your shell or from a .env file as long as it is in the same directory where you run the command.

    If running tests locally, make sure to use your local testing API key from the settings page.

  • Name
    --message
    Type
    string
    Description

    A description for the test run. For example:

    npx autoblocks testing exec -m "my first run" -- npm run start
  • Name
    --shared
    Type
    boolean
    Description

    Runs the tests and shares the results with your team. A shareable URL will be printed to the console.

    npx autoblocks testing exec --shared -- npm run start
  • Name
    --port
    Type
    number
    Description

    Choose the port of the internal server the CLI runs to collect test results. The default is 5555. For example:

    npx autoblocks testing exec --port 5000 -- npm run start

    If the port is not available, the CLI will increment the port number by 1 until it finds an available port.

  • Name
    --slack-webhook-url
    Type
    string
    Description

    A Slack webhook URL where test results will be posted. This can also be set via the AUTOBLOCKS_SLACK_WEBHOOK_URL environment variable, which will be read either from your shell or from a .env file as long as it is in the same directory where you run the command.

    See more details on Slack notifications in the Running in CI guide.

  • Name
    --exit-1-on-evaluation-failure
    Type
    boolean
    Description

    If set, the CLI will exit with code 1 at the end of the run if the score of any evaluation in any of the test suites does not meet its threshold. The default is false. For example:

    npx autoblocks testing exec --exit-1-on-evaluation-failure -- npm run start