Trace Tree

The Trace Tree provides a high level overview of your LLM events and allows you to visualize the structure of each trace.

Spans

While not necessary to use the Trace Tree, you can send span information along with your events to build a tree that reflects the structure of the work done within your application. Events can be grouped into spans using the spanId (or span_id) property, and you can additionally specify parent / child relationships between your spans with the parentSpanId (or parent_span_id) property.

In the code example below, we'll recreate the structure of the tree shown above by hardcoding the span IDs and parent span IDs so that we can easily demonstrate how to use spans to build a Trace Tree. Note that the properties have also been removed for readability.

import uuid

from autoblocks.tracer import AutoblocksTracer

tracer = AutoblocksTracer(
  trace_id=str(uuid.uuid4()),
)

tracer.send_event("ai.rag.start", span_id="1")
tracer.send_event("ai.embedding.request", span_id="1-a", parent_span_id="1")
tracer.send_event("ai.embedding.response", span_id="1-a", parent_span_id="1")
tracer.send_event("ai.embedding.request", span_id="1-b", parent_span_id="1")
tracer.send_event("ai.embedding.response", span_id="1-b", parent_span_id="1")
tracer.send_event("ai.rag.end", span_id="1")
tracer.send_event("ai.completion.request", span_id="2")
tracer.send_event("ai.completion.response", span_id="2")

LLM-Specific Properties

The Trace Tree view pulls out high level, LLM-specific properties from your events to display in the tree. In general, if you include the LLM request and response objects as-is anywhere in your event properties, Autoblocks will automatically pull out the relevant properties to display in the tree.

If you are modifying the LLM request or response objects before sending them to Autoblocks, however, you can reference the table below to see which properties are pulled out of your event properties:

Property Name	Description
`provider`	The model provider (e.g. openai, anthropic, etc.)
`model`, `model_name`, `modelName`	The model name (e.g. gpt-3.5-turbo, text-davinci-003, etc.)
`max_tokens`, `maxTokens`, `max_tokens_to_sample`, `maxTokensToSample`	The maximum number of tokens to generate
`total_tokens`, `totalTokens`	Total number of tokens used in the request (prompt + completion)
`completion_tokens`, `completionTokens`	Number of tokens in the generated completion
`prompt_tokens`, `promptTokens`	Number of tokens in the prompt
`temperature`	How random the completion should be
`frequency_penalty`, `frequencyPenalty`	How much to penalize new tokens based on their existing frequency in the text so far
`presence_penalty`, `presencePenalty`	How much to penalize new tokens based on whether they appear in the text so far
`top_p`, `topP`	An alternative to temperature called nucleus sampling
`top_k`, `topK`	How many tokens the model can choose from for its next token (sorted by probability)
`function_call`, `function_call.name`	The function call requested or generated by the model
`tool_choice.name`, `tool_choice`, `function.name`	The tool choice requested or generated by the model

Autoblocks will also look for the following properties to display the latency (all are assumed to be in milliseconds):

latency
latency_ms
latencyMs
duration
duration_ms
durationMs

If a latency property isn't detected, the latency displayed will be the time elapsed between the start and end of the span if there are span IDs attached to the events.

Inputs and Outputs

Like the LLM-specific properties described above, the inputs and outputs on the right-hand side of the tree are automatically parsed out of your event properties. If you include the LLM request and response objects as-is in your events, you likely do not need to do anything else in order for the inputs and outputs to be displayed. If you are modifying these objects before sending them to Autoblocks, however, make sure your LLM inputs and outputs match the below schemas in your event properties:

Inputs

{
  "messages": [
    {
      "role": "string",
      "content": "string"
    }
  ]
}

Outputs

{
  "choices": [
    {
      "message": {
        "role": "string",
        "content": "string"
      }
    }
  ]
}

More Examples

See our examples repository for more examples of sending span information with your events.