Running managed evaluations over traces on a platform.

Scheduled evals

Evidently AI - Documentation

What is Evidently?

How to install the open-source Python library.

Installation

Evidently Cloud

How to self-host the open-source Evidently UI service.

Self-hosting

Core concepts and components of the Evidently Python library.

Introduction

Available metrics, tests and how to customize them.

Evaluations

Reference page for all dataset-level evals.

All Metrics

Reference page for all row-level text and LLM evals.

All Descriptors

How to add a custom row-level text evaluator.

Custom Text Descriptor

How to run prompt-based evaluators for custom criteria.

Configure LLM Judges

How to use models from HuggingFace as evaluators.

Use HuggingFace models

How to change data drift detection methods and conditions.

Customize Data Drift

How to create a custom dataset or column-level Metric.

Custom Metric

Data drift

Open-source metrics for ranking and recommendations.

Ranking and RecSys metrics

Synthetic data

When do you need synthetic data in LLM evaluations.

Why synthetic data?

RAG evaluation dataset

Adversarial testing

Create synthetic inputs

Tutorials and guides

LLM evaluations

RAG evals

How to run regression testing for LLM outputs.

LLM regression testing

LLM as a judge

Evaluate the LLM outputs with multiple LLMs.

LLM-as-a-jury

Testing LLM outputs as part of the CI/CD flow.

Evidently and GitHub actions

Frequently Asked Questions

Why Evidently?

Open-source vs. Cloud

How to migrate to the new Evidently version?

Migration Guide

A new version of Evidently Cloud available starting April 10, 2025.

Evidently Cloud v2

What data is collected when you use Evidently open-source.

Telemetry

Contact us

Product updates

GitHub

Community

Docs

Examples

F.A.Q.

Changelog

Old docs

Sign up

Website

Talk to us

Log in

LLM Evaluation

Data and ML checks

How to capture LLM inputs and outputs and evaluate them.

Tracing

Overview

Data definition

Descriptors

Report

Tests

How to generate multiple metrics at once.

Metric generators

Output formats

Add tags and metadata

Set up an evaluation or monitoring Project.

Manage Projects

How to create, upload and manage Datasets.

Work with datasets

Set up tracing

How to run evals and log them on the platform

Run evals via API

How to evaluate your data in a no-code interface.

No code evals

Reviewing the evaluation results on the Platform.

Explore view

How to design your Dashboard with custom Panels.

Add dashboard panels (API)

Add dashboard panels (UI)

How production AI quality monitoring works.

Batch monitoring

Alerts

Text Evals

Data Drift

Data Summary

Overview of the Classification Quality Preset

Classification

Overview of the Regression Quality Preset

Get Started

Setup

Evaluation library

Platform

Scheduled evals