Search…
Command line interface
How to use Evidently from the Terminal
To start, prepare your data as csv files. Follow the same data requirements as described in the Jupyter notebook guide.
If you prefer a video version, here is 7-min Quick Start on how to use Evidently using CLI.

Generate HTML report or JSON Profile

To generate the HTML report, run the following command in bash:
1
$ python -m evidently calculate dashboard --config config.json
2
--reference reference.csv --current current.csv --output output_folder --report_name output_file_name
Copied!
To generate a JSON profile, run the following command in bash:
1
$ python -m evidently calculate profile --config config.json
2
--reference reference.csv --current current.csv --output output_folder --report_name output_file_name
Copied!
Here:
    reference is the path to the reference data,
    current is the path to the current data,
    output is the path to the output folder,
    config is the path to the configuration file,
    pretty_print to print the JSON profile with indents (for profile only).
You can choose the following Tabs:
    data_drift to estimate the data drift,
    num_target_drift to estimate target drift for the numerical target
    cat_target_drift to estimate target drift for the categorical target
    regression_performance to explore the performance of a regression model
    classification_performance to explore the performance of a classification model
    prob_classification_performance to explore the performance of a probabilistic classification model
To configure the report you need to create the config.json file or a config.yaml file. This file configures the way of reading your input data and the type of the report.

Configuration examples

Here is an example of a simple configuration, where we have comma-separated csv files with headers and there is no date column in the data.
Dashboard:
1
{
2
"data_format":{
3
"separator":",",
4
"header":true,
5
"date_column":null
6
},
7
"column_mapping":{},
8
"dashboard_tabs":["cat_target_drift"]
9
}
Copied!
Profile:
1
{
2
"data_format":{
3
"separator":",",
4
"header":true,
5
"date_column":null
6
},
7
"column_mapping":{},
8
"profile_sections":["data_drift"],
9
"pretty_print":true
10
}
Copied!
Here is an example for a more complicated configuration, where we have comma-separated csv files with headers and datetime column. We also specified the column_mapping dictionary, where we added information about the datetime, target and numerical_features.
Dashboard:
1
{
2
"data_format":{
3
"separator":",",
4
"header":true,
5
"date_column":"datetime"
6
},
7
"column_mapping":{
8
"datetime":"datetime",
9
"target":"target",
10
"numerical_features":["mean radius", "mean texture", "mean perimeter",
11
"mean area", "mean smoothness", "mean compactness", "mean concavity",
12
"mean concave points", "mean symmetry"]},
13
"dashboard_tabs":["cat_target_drift"],
14
"sampling": {
15
"reference": {
16
"type": "none"
17
},
18
"current": {
19
"type": "nth",
20
"n": 2
21
}
22
}
23
}
Copied!
Profile:
1
{
2
"data_format":{
3
"separator":",",
4
"header":true,
5
"date_column":null
6
},
7
"column_mapping":{
8
"target":"target",
9
"numerical_features":["mean radius", "mean texture", "mean perimeter",
10
"mean area", "mean smoothness", "mean compactness", "mean concavity",
11
"mean concave points", "mean symmetry"]},
12
"profile_sections":["data_drift", "cat_target_drift"],
13
"pretty_print":true,
14
"sampling": {
15
"reference": {
16
"type": "none"
17
},
18
"current": {
19
"type": "random",
20
"ratio": 0.8
21
}
22
}
23
}
Copied!

Telemetry

Telemetry is collected in Evidently starting from version 0.1.21.dev0.
When you use Evidently in the command-line interface, we collect some basic telemetry. It includes data on the environment (e.g. Python version) and usage (type of report or profile generated). You can read more about what we collect here.
You can opt-out from telemetry collection by setting the environment variable:
1
EVIDENTLY_DISABLE_TELEMETRY=1
Copied!

Sampling for large datasets

As shown in the configuration example above, you can specify sampling parameters for large files. You can use different sampling strategies for the reference and current data, or apply sampling only to one of the files.
Currently, you can choose from the following options:
    none- no sampling will be applied
    nth - each Nth row of the file will be taken. This option works together with the n parameter (see the example with the Dashboard above)
    random - random sampling will be applied. This option works together with ratio parameter (see the example with the Profile above)
If you do not specify the sampling parameters in the configuration, it will be treated as none and no sampling will be applied.
Last modified 2mo ago