data
- Required JSON input sent to the executortarget
- Optional reference data sent to the evaluator, typically containing expected outputsmetadata
- Optional metadata. This can be used to filter evaluation results in the UI after the evaluation is run.data
as inputtarget
are passed to the evaluatordata
as it is defined in the datapoints.
Evaluator accepts the output of the executor as its first argument,
and target
as it’s defined in the datapoints as the second argument.
This means that the type of the data
fields in your datapoints must
match the type of the first parameter of the executor function. Similarly,
the type of the target
fields in your datapoints must match the type of
the second parameter of the evaluator function(s).
Python is a bit more permissive. If you see type errors in TypeScript,
make sure the data types and the parameter types match.
For a more precise description, here’s the partial TypeScript type signature of the evaluate
function:
Laminar
initialization. If not specified,
Laminar will look for the key in the LMNR_PROJECT_API_KEY
environment variable.
my-first-evaluation.ts
and add the following code:config
object with instrumentModules
to evaluate
to ensure that the OpenAI client and any other instrumented modules are instrumented.lmnr eval
CLI or directly executing the evaluation file.
evaluate
function calls in your files - you don’t need to wrap them in a main
function or any special structure.
evals
directory with the naming pattern *.eval.{ts,js}
:evaluate
directly from your application code:
evaluate
function is flexible and can be used both in standalone scripts processed by the CLI and integrated directly into your application code.
evaluate
automatically initializes Laminar behind the scenes. All instrumented function calls and model invocations are traced without any additional setup.groupName
parameter to the evaluate
function.