Data definition

To run evaluations, you must create a Dataset object with a DataDefinition, which maps:

• Column types (e.g., categorical, numerical, text).
• Column roles (e.g., id, prediction, target).

This allows Evidently to process the data correctly. Some evaluations need specific columns and will fail if they’re missing. You can define the mapping using the Python API or by assigning columns visually when uploading data to the Evidently platform.

​

Basic flow

Step 1. Imports. Import the following modules:

from evidently import Dataset
from evidently import DataDefinition

Step 2. Prepare your data. Use a pandas.DataFrame.

Step 3. Create a Dataset object. Use Dataset.from_pandas with data_definition:

eval_data = Dataset.from_pandas(
    pd.DataFrame(source_df),
    data_definition=DataDefinition()
)

To map columns automatically, pass an empty DataDefinition() . Evidently will map columns:

• By type (numerical, categorical).
• By matching column names to roles (e.g., a column “target” treated as target).

Automation works in many cases, but manual mapping is more accurate. It is also necessary for evaluating prediction quality or handling text columns.

Step 4. Run evals. Once the Dataset object is ready, you can add Descriptors and run Reports.

​

Special cases

Working directly with pandas.DataFrame. You can sometimes pass a pandas.DataFrame directly to report.run() without creating the Dataset object. This works for checks like numerical/categorical data summaries or drift detection. However, it’s best to always create a Dataset object explicitly for clarity and control.

Working with two datasets. If you’re working with current and reference datasets (e.g., for drift detection), create a Dataset object for each. Both must have identical data definition.

​

Data definition

This page shows all the different mapping options. Note that you only need to use the relevant ones that apply for your evaluation scenario. For example, you don’t need columns like target/prediction to run data quality or LLM checks.

​

Column types

Knowing the column type helps compute correct statistics, visualizations, and pick default tests.

​

Text data

If you run LLM evaluations, simply specify the columns with inputs/outputs as text.

definition = DataDefinition(
    text_columns=["Latest_Review"]
    )
    
eval_data = Dataset.from_pandas(
    pd.DataFrame(source_df),
    data_definition=definition
)

​

Tabular data

Map numerical, categorical or datetime columns:

definition = DataDefinition(
    text_columns=["Latest_Review"],
    numerical_columns=["Age", "Salary"],
    categorical_columns=["Department"],
    datetime_columns=["Joining_Date"]
    )
    
eval_data = Dataset.from_pandas(
    pd.DataFrame(source_df),
    data_definition=definition
)

Explicit mapping helps avoid mistakes like misclassifying numerical columns with few unique values as categorical.

​

Default column types

If you do not pass explicit mapping, the following defaults apply:

​

ID and timestamp

If you have a timestamp or ID column, it’s useful to identify them.

definition = DataDefinition(
    id_column="Id",
    timestamp="Date"
    )

​

LLM evals

When you generate text descriptors and add them to the dataset, they are automatically mapped as descriptors in Data Definition. This means they will be included in the TextEvals preset or treated as descriptors when you plot them on the dashboard.

However, if you computed some scores or metadata externally and want to treat them as descriptors, you can map them explicitly:

definition = DataDefinition(
    numerical_descriptors=["chat_length", "user_rating"],
    categorical_descriptors=["upvotes", "model_type"]
    )

​

Regression

To run regression quality checks, you must map the columns with:

• Target: actual values.
• Prediction: predicted values.

You can have several regression results in the dataset, for example in case of multiple regression. (Pass the mappings in a list).

Example mapping:

definition = DataDefinition(
    regression=[Regression(target="y_true", prediction="y_pred")]
    )

Defaults:

    target: str = "target"
    prediction: str = "prediction"

​

Classification

To run classification checks, you must map the columns with:

• Target: true label.
• Prediction: predicted labels/probabilities.

There two different mapping options, for binary and multi-class classification. You can also have several classification results in the dataset. (Pass the mappings in a list).

​

Multiclass

Example mapping:

from evidently import MulticlassClassification

data_def = DataDefinition(
    classification=[MulticlassClassification(
        target="target",
        prediction_labels="prediction",
        prediction_probas=["0", "1", "2"],  # If probabilistic classification
        labels={"0": "class_0", "1": "class_1", "2": "class_2"}  # Optional, for display only
    )]
)

Available options and defaults:

    target: str = "target"
    prediction_labels: str = "prediction"
    prediction_probas: Optional[List[str]] = None #if probabilistic classification
    labels: Optional[Dict[Label, str]] = None

​

Binary

Example mapping:

from evidently import BinaryClassification

definition = DataDefinition(
    classification=[BinaryClassification(
        target="target",
        prediction_labels="prediction")],
    categorical_columns=["target", "prediction"])

Available options and defaults:

    target: str = "target"
    prediction_labels: Optional[str] = None
    prediction_probas: Optional[str] = "prediction" #if probabilistic classification
    pos_label: Label = 1 #name of the positive label
    labels: Optional[Dict[Label, str]] = None

​

Ranking

​

RecSys

To evaluate recommender systems performance, you must map the columns with:

• Prediction: this could be predicted score or rank.
• Target: relevance labels (e.g., this could be an interaction result like user click or upvote, or a true relevance label)

The target column can contain either:

• a binary label (where 1 is a positive outcome)
• any scores (positive values, where a higher value corresponds to a better match or a more valuable user action).

Here are the examples of the expected data inputs.

If the system prediction is a score (expected by default):

If the model prediction is a rank:

Example mapping:

definition = DataDefinition(
    ranking=[Recsys()]
    )

Available options and defaults:

    user_id: str = "user_id" #columns with user IDs
    item_id: str = "item_id" #columns with ranked items
    target: str = "target"
    prediction: str = "prediction"