Labeling queues

You can always label a span right in the trace view.

It is also useful to label spans in a separate places for multiple reasons:

• You may want to only send a subset of spans to a labeler for labeling.
• You may want to label spans later, when you have more context.
• Other team members may need to review the labels, and they don’t have to look at full traces.

​

Creating a labeling queue

Navigate to the labeling queues page by clicking on the “Labeling queues” tab in the left sidebar.

Create a new labeling queue by clicking on the “New queue” button.

We recommend creating separate queues for different team members, so that no more than one person labels the same span. You can create more than one queue per team member.

​

Labeling spans in a queue

You can move back and forth between spans in a queue by clicking on “Next” and “Previous”.

​

Additional hooks

Depending on the source of the data, some additional effects may be triggered when a span is labeled.

For now, this only works for evaluations. If the spans are added to a queue from evaluations, using the HumanEvaluator class, the resulting labels will be added back to the evaluation alongside the programmatic evaluator scores. See an example here.

​

Viewing labels

Once a span has been labeled, you can view all the labels assigned to it by clicking on the span.

You will be able to see the label class, label value, and the email of the labeler.