Prompt Management

Autoblocks prompt management allows product teams to collaborate on prompts while still maintaining an excellent developer experience via type safety, autocomplete in your IDE, and backwards-incompatibility protection.

Prompt management UI

Prompts can be configured in the Autoblocks UI using a simple interface. You can change your provider and common settings used for interacting with LLMs.

Multiple templates can be used to define a prompt. The templates can be combined at runtime to create a single prompt sent to an LLM provider.

See template flexibility for more examples.

Autogenerated interfaces

The Autoblocks prompt SDKs ship with a CLI that autogenerates interfaces for all aspects of the prompts you have configured in the UI:

Prompt IDs
Prompt versions
Prompt model parameters
Prompt template IDs
Prompt template parameters

This ensures your application will fail to compile if you reference a prompt, prompt version, or prompt template that doesn't exist or pass incorrect parameters to your templates:

Autocomplete

Because we generate interfaces for all aspects of the prompt, developers benefit from autocomplete in their IDEs when integrating Autoblocks-managed prompts into their applications:

Automated versioning

No need to manually version your prompts. When updating a prompt in the UI, the Autoblocks platform will automatically determine the next version based on the changes you make.

Backwards-incompatibility protection

Prompts consist of a major version and a minor version. The major version is incremented when you make a backwards-incompatible change to your prompt while the minor version is incremented when you make a backwards-compatible change to your prompt.

Major version bump

An example of a change that would require a major version bump is changing the name of a template parameter in your prompt:

Minor version bump

An example of a change that would only require a minor version bump is changing the text of a prompt template:

Another way to think about this is: if the autogenerated interfaces will change, then the major version must be incremented.

When initializing a prompt manager, the major version must be pinned while the minor version can either be pinned or set to latest. If set to latest, the prompt manager will automatically start using the latest minor version of a prompt after it is deployed from the Autoblocks platform. This allows product teams to safely ship backwards-compatible changes to their prompts without requiring a code deploy.

Zero added latency

When the minor version on a prompt manager is set to latest, you can configure the interval at which the prompt manager refreshes the prompt in the background so that the in-memory prompt eventually reflects the latest version. By default this interval is every 10 seconds.

Each SDK exposes an exec function that starts an execution context with the current, in-memory prompt. Starting a new execution context will never make a request to Autoblocks. This means you can safely ship features that use Autoblocks prompt management without any added latency to your application.

When starting a new execution context with exec, the execution context uses a frozen copy of the in-memory prompt as it existed at the time exec was called. This ensures the prompt is stable for the duration of an execution, even if the in-memory prompt on the manager instance is refreshed mid-execution.

If the minor version is pinned to a particular version, there is no background refresh process and the prompt is fetched exactly once when the prompt manager is initialized.

from datetime import timedelta

from my_project.autoblocks_prompts import TextSummarizationPromptManager

# A prompt manager should only be initialized once throughout
# the lifetime of your application.
# The configured version is fetched once within `__init__`.
mgr = TextSummarizationPromptManager(
    # If the minor version is "latest", we start a background
    # thread that refreshes the in-memory prompt at the
    # given refresh interval.
    minor_version="latest",
    refresh_interval=timedelta(
        seconds=5,
    ),
)

# Then, throughout your application, use `exec` to start
# new execution contexts any time you want to make a request
# to your LLM model. `exec` will never block on I/O to request
# a prompt from Autoblocks; it uses whatever prompt is currently
# in memory on the manager instance.
with mgr.exec() as prompt:
    # Build prompts, send request to LLM, etc.
    ...

A/B testing in production

When initializing a prompt manager, you can specify a weighted list of minor versions instead of a single version. If configured with a weighted list, the prompt manager will choose randomly between the given versions every time you start a new execution context with exec.

This allows you to A/B test multiple versions of a prompt simultaneously in production. For example, you can use the latest version of a prompt in only 1% of requests with the following configuration:

from autoblocks.prompts.models import WeightedMinorVersion

from my_project.autoblocks_prompts import TextSummarizationPromptManager

mgr = TextSummarizationPromptManager(
    minor_version=[
        WeightedMinorVersion(
            version="latest",
            # Latest minor version will be used 1% of the time
            weight=1,
        ),
        WeightedMinorVersion(
            version="0",
            # Minor version 0 will be used 99% of the time
            weight=99,
        ),
    ]
)

# The prompt manager will use the weights given to
# randomly choose a version for each execution context
with mgr.exec() as prompt:
    ...

Template flexibility

Many prompt management providers restrict prompts to just a single template with placeholders. This forces you to create multiple "prompts" for the same task, using some sort of prefix naming convention to denote they are being used together:

completion = client.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
    {
      "role": "system",
      "content": naive_prompt_manager.get_prompt(
        id="text-summarization-system-prompt",
        params={
          "languageRequirement": naive_prompt_manager.get_prompt(
            id="text-summarization-util-language",
            params={
              "language": "Spanish",
            },
          ),
          "toneRequirement": naive_prompt_manager.get_prompt(
            id="text-summarization-util-tone",
            params={
              "tone": "formal",
            },
          ),
        },
      ),
    },
    ...
  ],
)

In this example, each prompt is considered a separate entity despite being used together to send a single request. With a provider like this, these prompts will be versioned separately, edited separately, and reliant on your team remembering to use a consistent naming convention to keep track of relationships between prompts.

Autoblocks prompts, on the other hand, can be comprised of multiple templates. These usually consist of the system and user templates, but can also include any auxiliary templates that are necessary for building the final prompt(s) sent in the request to an LLM.

There should be a 1:1 relationship between Autoblocks prompts and an LLM request.

with mgr.exec() as prompt:
  completion = client.chat.completions.create(
    model=prompt.params.model,
    messages=[
      {
        "role": "system",
        "content": prompt.render.system(
          language_requirement=prompt.render.util_language(
            language="Spanish",
          ),
          tone_requirement=prompt.render.util_tone(
            tone="formal",
          ),
        ),
      },
      ...
    ],
  )

This allows you to manage all templates related to a particular task within a single prompt entity, ensuring they are versioned together, edited together, and tracked together.

SDKs

Get started with the Autoblocks prompt SDKs for your language: