Capture and Track Explicit User Feedback

In just a few minutes, you can send user feedback events to Autoblocks and easily monitor trends using our straightforward visualizations.

Prerequisites

This guide assumes you have already completed the Quick Start guide or Analyze and Debug LLM Requests guide.

Attach user feedback to a trace

It's a breeze to attach explicit user feedback to LLM interactions in Autoblocks. Simply send in an event using the same traceId. If you're not able to associate using the traceId property, you can alternatively send identifiers like a userId or sessionId as properties to link user feedback with LLM interactions.

import os
import uuid

from autoblocks.tracer import AutoblocksTracer

tracer = AutoblocksTracer(
  os.environ["AUTOBLOCKS_INGESTION_KEY"],
  trace_id=str(uuid.uuid4()),
)

# Send events related to LLM interaction using tracer

# feedback positive | negative
tracer.send_event(
  "user.feedback",
  properties=dict(
    feedback="positive"
  )
)

# score 1 | 2 | 3 | 4 | 5
tracer.send_event(
  "user.feedback",
  properties=dict(
    score=5
  )
)

Visualize user feedback

Once you have sent in user feedback, you can visualize it on the Explore page.

1. Filter for the user.feedback event type.
2. Open Chart Options
3. Breakdown by the feedback property
4. Select Stacked Bar or Line chart type. The stacked bar chart is great for visualizing what proportion of events are negative, while a line chart is useful for visualizing trends over time.
5. Granularity is set to hourly by default, but you can tailor this to your specific needs.

Fine tuning based on explicit user feedback

User feedback is a great property to filter by when curating a fine-tuning datasets. For example, you can use the traces API to export input-output pairs corresponding positive user feedback.