Introduction

​

Concept

Dataset is a collection of datapoints. It can be used for the following purposes:

1. Data storage for use in future fine-tuning or prompt-tuning.
2. Datasource for semantic search.
3. Provide inputs and expected outputs for Evaluations.

​

Format

Every datapoint has two fixed JSON objects: data and target, each with arbitrary keys. target is only used in evaluations.

• data – the actual datapoint data
• target – data additionally sent to the evaluator pipeline. See evaluations

For every key inside data and target, the value can be any JSON value.

​

Example

This is an example of a valid datapoint.

{
    "data": {
        "color": "red",
        "size": "large",
        "messages": [
            {
                "role": "user",
                "content": "Hello, can you help me choose a T-shirt?"
            },
            {
                "role": "assistant",
                "content": "I'm afraid, we don't sell T-shirts"
            }
        ]
    },
    "target": {
        "expected_output": "Of course! What size and color are you looking for?"
    }
}

​

Indexing

In order to be able to retrieve semantically relevant data points, we must index the dataset on a certain key.

This is done by simply specifying the key in the dataset page.

​

Indexing considerations

• Only values inside "data" of the dataset can be indexed.
• Datapoints that do not contain the specified key are skipped.
• If the value corresponding to this key is not a string or a list of chat messages, the datapoint is skipped.
• Newly added datapoints are indexed automatically according to the above rules.
• When updating the index column name, the entire dataset is re-indexed.

For the example above, if the dataset is indexed on "color", then only the first datapoint will be indexed, but not the second one, because it has an array of strings.

​

Use case: Evaluations

Datasets can be used for evaluations to specify inputs and expected outputs.

You will need to make sure the dataset keys match the input and output node names of the pipelines. See more in the Evaluations page.