How to use human evaluators in your evaluations.
HumanEvaluator
in your evaluation:
story_quality
scores are pending, and length_check
is completed.
When you click on a single evaluation run, you’ll see the human evaluator span. You can see the data that was sent to this human evaluator and manually score it.
After you score the human evaluator, you’ll see the scores in the dashboard.
span_type = 'HUMAN_EVALUATOR'
and can be identified by their evaluator name. Here’s a basic query to find all human evaluator spans in evaluation with certain id:
input
to dataset data
fieldoutput
to dataset target
fieldtarget
field as an expected score for the LLM-as-a-judge evaluator.