All Presets

An overview of the evaluations you can do with Evidently.

Evidently has several pre-built reports and test suites. We call them Presets. Each preset evaluates or tests a particular aspect of the data or model quality.

This page links to the description of each preset. To see the code and interactive examples, head to example notebooks instead.

Metric Presets

Metric presets are pre-built reports that help with visual exploration, debugging and documentation of the data and model performance. You can also use them to calculate and log metrics as JSON or Python dictionary.

Data Quality Shows the dataset statistics and feature behavior. Requirements: model inputs.

Data Drift Explores the distribution shift in the model features. Requirements: model inputs, a reference dataset.

Target Drift Explores the distribution shift in the model predictions or target. Requirements: model predictions and/or target, a reference dataset.

Classification Evaluates the classification model quality and errors. Requirements: model predictions and true labels.

Regression Evaluates the regression model quality and errors. Requirements: model predictions and actuals.

Text Overview Evaluates text data drift and descriptive statistics. Requirements: model inputs (raw text data)

Test Presets

Test presets are pre-built test suites that perform structured data and model checks as part of the pipeline.

NoTargetPerformance Tests the model performance without ground truth or actuals. Requirements: model inputs, predictions, a reference dataset.

Data Drift Tests for distribution drift per column and overall dataset drift. Requirements: model inputs, a reference dataset.

Data Stability Tests if a data batch is similar to reference. Checks schema, data ranges, etc. Requirements: inputs, a reference dataset.

Data Quality Tests if the data quality is suitable for (re)training. Checks nulls, duplicates, etc. Requirements: model inputs.

Regression Tests the performance of the regression model against expectation. Requirements: model predictions and actuals.

Multi-class Classification Tests the performance of a multi-class classification model against expectation. Requirements: model predictions, true labels.

Binary Classification Tests the performance of a binary classification model against expectation. Requirements: model predictions, true labels.

Binary Classification top-K Tests the performance of a binary classification model at top-K. Requirements: model predictions, true labels.

Individual Metrics and Tests

You can also create custom test suites and reports from individual metrics and tests. You can explore 100+ available tests and metrics.