evidently.test_preset
Predefined Test Presets for Test Suite
class BinaryClassificationTestPreset(stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None, probas_threshold: Optional[float] = None)
Bases: TestPreset
Binary Classification Tests. :param stattest: statistical test for target drift test. :param stattest_threshold: threshold for statistical test for target drift test. :param probas_threshold: threshold for label calculation for prediction.
Contains:
TestColumnValueDrift for target
TestPrecisionScore
TestRecallScore
TestF1Score
TestAccuracyScore
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
class BinaryClassificationTopKTestPreset(k: Union[float, int], stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None)
Bases: TestPreset
Binary Classification Tests for Top K threshold. :param stattest: stattest for TestColumnDrift :param stattest_threshold: threshold for stattest
Contains:
TestColumnDrift for target
TestPrecisionScore
TestRecallScore
TestF1Score
TestAccuracyScore
Attributes:
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
class DataDriftTestPreset(columns: Optional[List[str]] = None, drift_share: Optional[float] = None, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, cat_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, num_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, per_column_stattest: Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None, stattest_threshold: Optional[float] = None, cat_stattest_threshold: Optional[float] = None, num_stattest_threshold: Optional[float] = None, per_column_stattest_threshold: Optional[Dict[str, float]] = None)
Bases: TestPreset
Data Drift tests.
Contains tests:
TestShareOfDriftedColumns
TestColumnValueDrift
TestAllFeaturesValueDrift
Attributes:
cat_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
cat_stattest_threshold : Optional[float]
columns : Optional[List[str]]
drift_share : Optional[float]
num_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
num_stattest_threshold : Optional[float]
per_column_stattest : Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]]
per_column_stattest_threshold : Optional[Dict[str, float]]
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
class DataQualityTestPreset(columns: Optional[List[str]] = None)
Bases: TestPreset
Data Quality tests.
Contains tests:
TestAllColumnsShareOfMissingValues
TestAllColumnsMostCommonValueShare
TestNumberOfConstantColumns
TestNumberOfDuplicatedColumns
TestNumberOfDuplicatedRows
TestHighlyCorrelatedColumns
Attributes:
columns : Optional[List[str]]
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
class DataStabilityTestPreset(columns: Optional[List[str]] = None)
Bases: TestPreset
Data Stability tests.
Contains tests:
TestNumberOfRows
TestNumberOfColumns
TestColumnsType
TestAllColumnsShareOfMissingValues
TestNumColumnsOutOfRangeValues
TestCatColumnsOutOfListValues
TestNumColumnsMeanInNSigmas
Attributes:
columns : Optional[List[str]]
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
class MulticlassClassificationTestPreset(stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None)
Bases: TestPreset
Multiclass Classification tests.
Parameters
stattest– statistical test for target drift test.stattest_threshold– threshold for statistical test for target drift test.
Contains tests:
TestAccuracyScore
TestF1Score
TestPrecisionByClass for each class in data
TestRecallByClass for each class in data
TestNumberOfRows
TestColumnValueDrift
TestRocAuc
TestLogLoss
Attributes:
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
class NoTargetPerformanceTestPreset(columns: Optional[List[str]] = None, drift_share: Optional[float] = None, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, cat_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, num_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, per_column_stattest: Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None, stattest_threshold: Optional[float] = None, cat_stattest_threshold: Optional[float] = None, num_stattest_threshold: Optional[float] = None, per_column_stattest_threshold: Optional[Dict[str, float]] = None)
Bases: TestPreset
No Target Performance tests.
Parameters
columns– list of columns include to tests
Contains tests:
TestColumnValueDrift
TestShareOfDriftedColumns
TestColumnsType
TestAllColumnsShareOfMissingValues
TestNumColumnsOutOfRangeValues
TestCatColumnsOutOfListValues
TestNumColumnsMeanInNSigmas
TestCustomFeaturesValueDrift
Attributes:
cat_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
cat_stattest_threshold : Optional[float] = None
columns : Optional[List[str]]
drift_share : Optional[float]
num_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
num_stattest_threshold : Optional[float] = None
per_column_stattest : Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None
per_column_stattest_threshold : Optional[Dict[str, float]] = None
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
stattest_threshold : Optional[float] = None
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
class RegressionTestPreset()
Bases: TestPreset
Regression performance tests.
Contains tests:
TestValueMeanError
TestValueMAE
TestValueRMSE
TestValueMAPE
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
Submodules
classification_binary module
class BinaryClassificationTestPreset(stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None, probas_threshold: Optional[float] = None)
Bases: TestPreset
Binary Classification Tests. :param stattest: statistical test for target drift test. :param stattest_threshold: threshold for statistical test for target drift test. :param probas_threshold: threshold for label calculation for prediction.
Contains:
TestColumnValueDrift for target
TestPrecisionScore
TestRecallScore
TestF1Score
TestAccuracyScore
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
classification_binary_topk module
class BinaryClassificationTopKTestPreset(k: Union[float, int], stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None)
Bases: TestPreset
Binary Classification Tests for Top K threshold. :param stattest: stattest for TestColumnDrift :param stattest_threshold: threshold for stattest
Contains:
TestColumnDrift for target
TestPrecisionScore
TestRecallScore
TestF1Score
TestAccuracyScore
Attributes:
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
classification_multiclass module
class MulticlassClassificationTestPreset(stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None)
Bases: TestPreset
Multiclass Classification tests.
Parameters
stattest– statistical test for target drift test.stattest_threshold– threshold for statistical test for target drift test.
Contains tests:
TestAccuracyScore
TestF1Score
TestPrecisionByClass for each class in data
TestRecallByClass for each class in data
TestNumberOfRows
TestColumnValueDrift
TestRocAuc
TestLogLoss
Attributes:
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
data_drift module
class DataDriftTestPreset(columns: Optional[List[str]] = None, drift_share: Optional[float] = None, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, cat_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, num_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, per_column_stattest: Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None, stattest_threshold: Optional[float] = None, cat_stattest_threshold: Optional[float] = None, num_stattest_threshold: Optional[float] = None, per_column_stattest_threshold: Optional[Dict[str, float]] = None)
Bases: TestPreset
Data Drift tests.
Contains tests:
TestShareOfDriftedColumns
TestColumnValueDrift
TestAllFeaturesValueDrift
Attributes:
cat_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
cat_stattest_threshold : Optional[float]
columns : Optional[List[str]]
drift_share : Optional[float]
num_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
num_stattest_threshold : Optional[float]
per_column_stattest : Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]]
per_column_stattest_threshold : Optional[Dict[str, float]]
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
data_quality module
class DataQualityTestPreset(columns: Optional[List[str]] = None)
Bases: TestPreset
Data Quality tests.
Contains tests:
TestAllColumnsShareOfMissingValues
TestAllColumnsMostCommonValueShare
TestNumberOfConstantColumns
TestNumberOfDuplicatedColumns
TestNumberOfDuplicatedRows
TestHighlyCorrelatedColumns
Attributes:
columns : Optional[List[str]]
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
data_stability module
class DataStabilityTestPreset(columns: Optional[List[str]] = None)
Bases: TestPreset
Data Stability tests.
Contains tests:
TestNumberOfRows
TestNumberOfColumns
TestColumnsType
TestAllColumnsShareOfMissingValues
TestNumColumnsOutOfRangeValues
TestCatColumnsOutOfListValues
TestNumColumnsMeanInNSigmas
Attributes:
columns : Optional[List[str]]
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
no_target_performance module
class NoTargetPerformanceTestPreset(columns: Optional[List[str]] = None, drift_share: Optional[float] = None, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, cat_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, num_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, per_column_stattest: Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None, stattest_threshold: Optional[float] = None, cat_stattest_threshold: Optional[float] = None, num_stattest_threshold: Optional[float] = None, per_column_stattest_threshold: Optional[Dict[str, float]] = None)
Bases: TestPreset
No Target Performance tests.
Parameters
columns– list of columns include to tests
Contains tests:
TestColumnValueDrift
TestShareOfDriftedColumns
TestColumnsType
TestAllColumnsShareOfMissingValues
TestNumColumnsOutOfRangeValues
TestCatColumnsOutOfListValues
TestNumColumnsMeanInNSigmas
TestCustomFeaturesValueDrift
Attributes:
cat_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
cat_stattest_threshold : Optional[float] = None
columns : Optional[List[str]]
drift_share : Optional[float]
num_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
num_stattest_threshold : Optional[float] = None
per_column_stattest : Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None
per_column_stattest_threshold : Optional[Dict[str, float]] = None
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
stattest_threshold : Optional[float] = None
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
regression module
class RegressionTestPreset()
Bases: TestPreset
Regression performance tests.
Contains tests:
TestValueMeanError
TestValueMAE
TestValueRMSE
TestValueMAPE
Methods:
generate_tests(data: InputData, columns: DatasetColumns)
test_preset module
class TestPreset()
Bases: object
Methods:
abstract generate_tests(data: InputData, columns: DatasetColumns)
Last updated