You can always label a span right in the trace view.

It is also useful to label spans in a separate places for multiple reasons:

  • You may want to only send a subset of spans to a labeler for labeling.
  • You may want to label spans later, when you have more context.
  • Other team members may need to review the labels, and they don’t have to look at full traces.

​
Creating a labeling queue

Navigate to the labeling queues page by clicking on the “Labeling queues” tab in the left sidebar.

Create a new labeling queue by clicking on the “New queue” button.

We recommend creating separate queues for different team members, so that no more than one person labels the same span. You can create more than one queue per team member.

​
Adding data to a queue

There are three ways to add data to a queue:

  • Manually adding from the span view in the traces page.
  • Pushing from the API.
  • Registering a human evaluator in an evaluation.

​
Manually pushing from the span view

You can manually push data to a queue by clicking on the “Add to labeling queue” button in the span view.

​
Pushing from the API

You can push data to a queue by calling the POST /v1/queues/push API endpoint.

See the API reference for more information.

​
Registering a human evaluator in an evaluation

You can register a human evaluator in an evaluation by passing a HumanEvaluator object with the queue name to the evaluate function.

See the Evaluation configuration page for more information.

​
Labeling spans in a queue

You can move back and forth between spans in a queue by clicking on “Next” and “Previous”.

Labels are not saved automatically. Make sure to save them by clicking on the “Complete” button, once you are done with labeling one entry.

​
Additional hooks

Depending on the source of the data, some additional effects may be triggered when a span is labeled.

For now, this only works for evaluations. If the spans are added to a queue from evaluations, using the HumanEvaluator class, the resulting labels will be added back to the evaluation alongside the programmatic evaluator scores. See an example here.

​
Viewing labels

Once a span has been labeled, you can view all the labels assigned to it by clicking on the span.

You will be able to see the label class, label value, and the email of the labeler.

IntroductionLabeling from code