​
Glossary

  • Span – Think of this as one step in your LLM application’s process. It could be a function call or an API request.

    • Span attributes – Extra information about the span, like input parameters or results
    • Span path – Shows where the span came from in your code. For example: get_user.validate.api_call

  • Trace – A collection of spans, where each span can have parent and child spans. For example, when function A calls function B, function A is the parent span and function B is the child span. This creates a hierarchical view showing how different parts of your code interact with each other.

  • Session – A group of related traces that belong to the same user or conversation.

​
Example trace

Tree on the left shows a trace of an example LLM application. Function answer_question calls fetch_page_and_check, which in turn calls check_presence, which calls OpenAI. Each line in the tree is a span.

​
Tracing overview

Laminar offers comprehensive tracing of your entire application. For every run, the entire execution trace is logged, so the information you can see in the traces includes:

  • Inputs and outputs of each span
  • Total execution time
  • Total execution tokens and cost
  • Span-level execution time and token counts

​
Getting started

​
Project API key

To get the project API key, go to the Laminar dashboard, click the project settings, and generate a project API key.

Specify the key at Laminar initialization. If not specified, Laminar will look for the key in the LMNR_PROJECT_API_KEY environment variable.

​
Add 2 lines to start tracing your entire application

from lmnr import Laminar
Laminar.initialize(project_api_key="LMNR_PROJECT_API_KEY")

Laminar should be initialized once in your application. This could be at the server startup, or in the entry point of your application.

This will automatically instrument all major LLM provider SDKs, LLM frameworks including LangChain and LlamaIndex, and calls to vector databases.

To disable or configure automatic instrumentation, see the section on Automatic instrumentation.

For example, calling OpenAI after initializing Laminar like this will create the following span in Laminar dashboard:

from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)

Laminar automatically records latency, cost, and tokens for LLM spans, taking into account the model and the number of tokens in the response.

​
Tracing specific functions with observe and adding children spans

You can instrument specific functions by adding the @observe() decorator. This is especially helpful when you want to trace functions, or group separate functions into a single trace.

from lmnr import observe

@observe()  # annotate all functions you want to trace
def my_function():
    res = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "What is the capital of France?"}],
    )
    return res.choices[0].message.content

my_function()

We are now recording my_function and the OpenAI call, which is nested inside it, in the same trace. Notice that the OpenAI span is a child of my_function. Parent-child relationships are automatically detected and visualized with tree hierarchy.

You can nest as many spans as you want inside each other. By observing both the functions and the LLM/vector DB calls you can have better visualization of execution flow which is useful for debugging and better understanding of the application.

Input arguments to the function are automatically recorded as inputs of the span. The return value is automatically recorded as the output of the span.

Passing arguments to the function in TypeScript is slightly non-trivial. Example:

const myFunction = observe(
  { name: 'myFunction'}, 
  async (param1, param2) => {
    // ...
  }
  'argValue1',
  'argValue2'
);

​
Sending traces to a self-hosted Laminar instance

By default, Laminar client packages send traces to the Laminar cloud at api.lmnr.ai.

If you want to send traces to your own self-hosted Laminar instance, you can set the base URL and port numbers at initialization. Default settings in our repo are gRPC port 8001 and HTTP port 8000.

from lmnr import Laminar
Laminar.initialize(
    project_api_key="<your-project-api-key>",
    base_url="http://localhost", # do not include ports or trailing slash here
    http_port=8000,
    grpc_port=8001
)

​
OpenTelemetry compatibility

Laminar’s manual and automatic instrumentation is compatible with OpenTelemetry. Our Rust backend is an OpenTelemetry-compatible ingestion endpoint and processes traces using gRPC. This ensures long-lived connections and extremely fast ingestion.

Instrumentations for LLM and vector DB libraries are provided by OpenLLMetry.

This means that you can use OpenTelemetry SDKs to send traces to Laminar, and they will be displayed in the Laminar UI.

To get started, in your application, set the OpenTelemetry exporter to the Laminar gRPC endpoint: https://api.lmnr.ai:8443/v1/traces.

Read on to the Otel section to learn more about the OpenTelemetry objects and attributes that Laminar uses in more detail.

​
Next steps

To better understand the anatomy of a trace, the terminology, and the concepts, read on to the Structure page.

InstallationAdding structure to traces