Overview

Get your data in a table like a pandas.DataFrame. More on data requirements. You can also load data from Evidently Platform, like tracing or synthetic datasets.

Create a Dataset object with DataDefinition() that specifies column role and types. You can also use default type detection. How to set Data Definition.

eval_data = Dataset.from_pandas(
    source_df,
    data_definition=DataDefinition()
)

For text evals, choose and compute row-level descriptors. Optionally, add row-level tests to get pass/fail for specific inputs. How to use Descriptors.

eval_data.add_descriptors(descriptors=[
    TextLength("Question", alias="Length"),
    Sentiment("Answer", alias="Sentiment")
])

For dataset-level evals (classification, data drift) or to summarize descriptors, create a Report with chosen metrics or presets. How to configure Reports.

report = Report([
    DataSummaryPreset()
])

Add dataset-level Pass/Fail conditions, like to check if all texts are in < 100 symbols length. How to configure Tests.

report = Report([
    DataSummaryPreset(),
    MaxValue(column="Length", tests=[lt(100)]),
])

Add tags or metadata to identify specific evaluation runs or datasets, or override the default timestamp . How to add metadata.

To execute the eval, runthe Report on the Dataset (or two).

my_eval = report.run(eval_data, None)

• To upload to the Evidently Platform. How to upload results.

ws.add_run(project.id, my_eval, include_data=True)

• To view locally. All output formats.

my_eval
##my_eval.json()