latest
Search…
⌃K
Links

evidently.test_preset

Predefined Test Presets for Test Suite

class BinaryClassificationTestPreset(prediction_type: str, columns: Optional[List[str]] = None, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None, probas_threshold: Optional[float] = None)

Bases: TestPreset
Binary Classification Tests. :param threshold: probabilities threshold for prediction with probas :param prediction_type: type of prediction (‘probas’ or ‘labels’)
Contains:
  • TestColumnValueDrift for target
  • TestPrecisionScore - use threshold if prediction_type is ‘probas’
  • TestRecallScore - use threshold if prediction_type is ‘probas’
  • TestF1Score - use threshold if prediction_type is ‘probas’
  • TestAccuracyScore - use threshold if prediction_type is ‘probas’

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

class BinaryClassificationTopKTestPreset(k: Union[float, int], probas_threshold: Optional[float] = None, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None)

Bases: TestPreset
Binary Classification Tests for Top K threshold. :param threshold: probabilities threshold for prediction with probas :param prediction_type: type of prediction (‘probas’ or ‘labels’)
Contains:
  • TestColumnValueDrift for target
  • TestPrecisionScore - use threshold if prediction_type is ‘probas’
  • TestRecallScore - use threshold if prediction_type is ‘probas’
  • TestF1Score - use threshold if prediction_type is ‘probas’
  • TestAccuracyScore - use threshold if prediction_type is ‘probas’

Attributes:

stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

class DataDriftTestPreset(stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, cat_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, num_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, per_column_stattest: Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None, stattest_threshold: Optional[float] = None, cat_stattest_threshold: Optional[float] = None, num_stattest_threshold: Optional[float] = None, per_column_stattest_threshold: Optional[Dict[str, float]] = None)

Bases: TestPreset
Data Drift tests.
Contains tests:
  • TestShareOfDriftedColumns
  • TestColumnValueDrift
  • TestAllFeaturesValueDrift

Attributes:

cat_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
cat_stattest_threshold : Optional[float]
num_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
num_stattest_threshold : Optional[float]
per_column_stattest : Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]]
per_column_stattest_threshold : Optional[Dict[str, float]]
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

class DataQualityTestPreset(columns: Optional[List[str]] = None)

Bases: TestPreset
Data Quality tests.
Contains tests:
  • TestAllColumnsShareOfMissingValues
  • TestAllColumnsMostCommonValueShare
  • TestNumberOfConstantColumns
  • TestNumberOfDuplicatedColumns
  • TestNumberOfDuplicatedRows
  • TestHighlyCorrelatedColumns

Attributes:

columns : Optional[List[str]]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

class DataStabilityTestPreset(columns: Optional[List[str]] = None)

Bases: TestPreset
Data Stability tests.
Contains tests:
  • TestNumberOfRows
  • TestNumberOfColumns
  • TestColumnsType
  • TestAllColumnsShareOfMissingValues
  • TestNumColumnsOutOfRangeValues
  • TestCatColumnsOutOfListValues
  • TestNumColumnsMeanInNSigmas

Attributes:

columns : Optional[List[str]]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

class MulticlassClassificationTestPreset(prediction_type: str, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None)

Bases: TestPreset
Multiclass Classification tests.
  • Parameters
    prediction_type – type of prediction data (‘probas’ or ‘labels’)
Contains tests:
  • TestAccuracyScore
  • TestF1Score
  • TestPrecisionByClass for each class in data
  • TestRecallByClass for each class in data
  • TestNumberOfRows
  • TestColumnValueDrift
  • TestRocAuc
  • TestLogLoss

Attributes:

stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

class NoTargetPerformanceTestPreset(columns: Optional[List[str]] = None, drift_share: float = 0.5, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, cat_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, num_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, per_column_stattest: Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None, stattest_threshold: Optional[float] = None, cat_stattest_threshold: Optional[float] = None, num_stattest_threshold: Optional[float] = None, per_column_stattest_threshold: Optional[Dict[str, float]] = None)

Bases: TestPreset
No Target Performance tests.
  • Parameters
    columns – list of columns include to tests
Contains tests:
  • TestColumnValueDrift
  • TestShareOfDriftedColumns
  • TestColumnsType
  • TestAllColumnsShareOfMissingValues
  • TestNumColumnsOutOfRangeValues
  • TestCatColumnsOutOfListValues
  • TestNumColumnsMeanInNSigmas
  • TestCustomFeaturesValueDrift

Attributes:

cat_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
cat_stattest_threshold : Optional[float] = None
columns : Optional[List[str]]
drift_share : float
num_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
num_stattest_threshold : Optional[float] = None
per_column_stattest : Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None
per_column_stattest_threshold : Optional[Dict[str, float]] = None
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
stattest_threshold : Optional[float] = None

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

class RegressionTestPreset()

Bases: TestPreset
Regression performance tests.
Contains tests:
  • TestValueMeanError
  • TestValueMAE
  • TestValueRMSE
  • TestValueMAPE

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

Submodules

classification_binary module

class BinaryClassificationTestPreset(prediction_type: str, columns: Optional[List[str]] = None, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None, probas_threshold: Optional[float] = None)

Bases: TestPreset
Binary Classification Tests. :param threshold: probabilities threshold for prediction with probas :param prediction_type: type of prediction (‘probas’ or ‘labels’)
Contains:
  • TestColumnValueDrift for target
  • TestPrecisionScore - use threshold if prediction_type is ‘probas’
  • TestRecallScore - use threshold if prediction_type is ‘probas’
  • TestF1Score - use threshold if prediction_type is ‘probas’
  • TestAccuracyScore - use threshold if prediction_type is ‘probas’

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

classification_binary_topk module

class BinaryClassificationTopKTestPreset(k: Union[float, int], probas_threshold: Optional[float] = None, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None)

Bases: TestPreset
Binary Classification Tests for Top K threshold. :param threshold: probabilities threshold for prediction with probas :param prediction_type: type of prediction (‘probas’ or ‘labels’)
Contains:
  • TestColumnValueDrift for target
  • TestPrecisionScore - use threshold if prediction_type is ‘probas’
  • TestRecallScore - use threshold if prediction_type is ‘probas’
  • TestF1Score - use threshold if prediction_type is ‘probas’
  • TestAccuracyScore - use threshold if prediction_type is ‘probas’

Attributes:

stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

classification_multiclass module

class MulticlassClassificationTestPreset(prediction_type: str, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None)

Bases: TestPreset
Multiclass Classification tests.
  • Parameters
    prediction_type – type of prediction data (‘probas’ or ‘labels’)
Contains tests:
  • TestAccuracyScore
  • TestF1Score
  • TestPrecisionByClass for each class in data
  • TestRecallByClass for each class in data
  • TestNumberOfRows
  • TestColumnValueDrift
  • TestRocAuc
  • TestLogLoss

Attributes:

stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

data_drift module

class DataDriftTestPreset(stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, cat_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, num_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, per_column_stattest: Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None, stattest_threshold: Optional[float] = None, cat_stattest_threshold: Optional[float] = None, num_stattest_threshold: Optional[float] = None, per_column_stattest_threshold: Optional[Dict[str, float]] = None)

Bases: TestPreset
Data Drift tests.
Contains tests:
  • TestShareOfDriftedColumns
  • TestColumnValueDrift
  • TestAllFeaturesValueDrift

Attributes:

cat_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
cat_stattest_threshold : Optional[float]
num_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
num_stattest_threshold : Optional[float]
per_column_stattest : Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]]
per_column_stattest_threshold : Optional[Dict[str, float]]
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

data_quality module

class DataQualityTestPreset(columns: Optional[List[str]] = None)

Bases: TestPreset
Data Quality tests.
Contains tests:
  • TestAllColumnsShareOfMissingValues
  • TestAllColumnsMostCommonValueShare
  • TestNumberOfConstantColumns
  • TestNumberOfDuplicatedColumns
  • TestNumberOfDuplicatedRows
  • TestHighlyCorrelatedColumns

Attributes:

columns : Optional[List[str]]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

data_stability module

class DataStabilityTestPreset(columns: Optional[List[str]] = None)

Bases: TestPreset
Data Stability tests.
Contains tests:
  • TestNumberOfRows
  • TestNumberOfColumns
  • TestColumnsType
  • TestAllColumnsShareOfMissingValues
  • TestNumColumnsOutOfRangeValues
  • TestCatColumnsOutOfListValues
  • TestNumColumnsMeanInNSigmas

Attributes:

columns : Optional[List[str]]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

no_target_performance module

class NoTargetPerformanceTestPreset(columns: Optional[List[str]] = None, drift_share: float = 0.5, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, cat_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, num_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, per_column_stattest: Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None, stattest_threshold: Optional[float] = None, cat_stattest_threshold: Optional[float] = None, num_stattest_threshold: Optional[float] = None, per_column_stattest_threshold: Optional[Dict[str, float]] = None)

Bases: TestPreset
No Target Performance tests.
  • Parameters
    columns – list of columns include to tests
Contains tests:
  • TestColumnValueDrift
  • TestShareOfDriftedColumns
  • TestColumnsType
  • TestAllColumnsShareOfMissingValues
  • TestNumColumnsOutOfRangeValues
  • TestCatColumnsOutOfListValues
  • TestNumColumnsMeanInNSigmas
  • TestCustomFeaturesValueDrift

Attributes:

cat_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
cat_stattest_threshold : Optional[float] = None
columns : Optional[List[str]]
drift_share : float
num_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
num_stattest_threshold : Optional[float] = None
per_column_stattest : Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None
per_column_stattest_threshold : Optional[Dict[str, float]] = None
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
stattest_threshold : Optional[float] = None

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

regression module

class RegressionTestPreset()

Bases: TestPreset
Regression performance tests.
Contains tests:
  • TestValueMeanError
  • TestValueMAE
  • TestValueRMSE
  • TestValueMAPE

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

test_preset module

class TestPreset()

Bases: object

Methods:

abstract generate_tests(data: InputData, columns: DatasetColumns)