Capture and Track Explicit User Feedback
In just a few minutes, you can send user feedback events to Autoblocks and easily monitor trends using our straightforward visualizations.
It's a breeze to attach explicit user feedback to LLM interactions in Autoblocks. Simply send in an event using the same
traceId. If you're not able to associate using the traceId property, you can alternatively send identifiers like a
sessionId as properties to link user feedback with LLM interactions.
import os import uuid from autoblocks.tracer import AutoblocksTracer tracer = AutoblocksTracer( os.environ["AUTOBLOCKS_INGESTION_KEY"], trace_id=str(uuid.uuid4()), ) # Send events related to LLM interaction using tracer # feedback positive | negative tracer.send_event( "user.feedback", properties=dict( feedback="positive" ) ) # score 1 | 2 | 3 | 4 | 5 tracer.send_event( "user.feedback", properties=dict( score=5 ) )
Once you have sent in user feedback, you can visualize it on the Explore page.
- Filter for the
- Open Chart Options
- Breakdown by the
- Select Stacked Bar or Line chart type. The stacked bar chart is great for visualizing what proportion of events are negative, while a line chart is useful for visualizing trends over time.
- Granularity is set to hourly by default, but you can tailor this to your specific needs.
User feedback is a great property to filter by when curating a fine-tuning datasets. For example, you can use the traces API to export input-output pairs corresponding positive user feedback.