latest
HomeBlogGitHub
Search
⌃K
Links

evidently.test_preset

Predefined Test Presets for Test Suite

class BinaryClassificationTestPreset(stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None, probas_threshold: Optional[float] = None)

Bases: TestPreset
Binary Classification Tests. :param stattest: statistical test for target drift test. :param stattest_threshold: threshold for statistical test for target drift test. :param probas_threshold: threshold for label calculation for prediction.
Contains:
  • TestColumnValueDrift for target
  • TestPrecisionScore
  • TestRecallScore
  • TestF1Score
  • TestAccuracyScore

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

class BinaryClassificationTopKTestPreset(k: Union[float, int], stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None)

Bases: TestPreset
Binary Classification Tests for Top K threshold. :param stattest: stattest for TestColumnDrift :param stattest_threshold: threshold for stattest
Contains:
  • TestColumnDrift for target
  • TestPrecisionScore
  • TestRecallScore
  • TestF1Score
  • TestAccuracyScore

Attributes:

stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

class DataDriftTestPreset(columns: Optional[List[str]] = None, drift_share: Optional[float] = None, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, cat_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, num_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, per_column_stattest: Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None, stattest_threshold: Optional[float] = None, cat_stattest_threshold: Optional[float] = None, num_stattest_threshold: Optional[float] = None, per_column_stattest_threshold: Optional[Dict[str, float]] = None)

Bases: TestPreset
Data Drift tests.
Contains tests:
  • TestShareOfDriftedColumns
  • TestColumnValueDrift
  • TestAllFeaturesValueDrift

Attributes:

cat_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
cat_stattest_threshold : Optional[float]
columns : Optional[List[str]]
drift_share : Optional[float]
num_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
num_stattest_threshold : Optional[float]
per_column_stattest : Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]]
per_column_stattest_threshold : Optional[Dict[str, float]]
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

class DataQualityTestPreset(columns: Optional[List[str]] = None)

Bases: TestPreset
Data Quality tests.
Contains tests:
  • TestAllColumnsShareOfMissingValues
  • TestAllColumnsMostCommonValueShare
  • TestNumberOfConstantColumns
  • TestNumberOfDuplicatedColumns
  • TestNumberOfDuplicatedRows
  • TestHighlyCorrelatedColumns

Attributes:

columns : Optional[List[str]]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

class DataStabilityTestPreset(columns: Optional[List[str]] = None)

Bases: TestPreset
Data Stability tests.
Contains tests:
  • TestNumberOfRows
  • TestNumberOfColumns
  • TestColumnsType
  • TestAllColumnsShareOfMissingValues
  • TestNumColumnsOutOfRangeValues
  • TestCatColumnsOutOfListValues
  • TestNumColumnsMeanInNSigmas

Attributes:

columns : Optional[List[str]]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

class MulticlassClassificationTestPreset(stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None)

Bases: TestPreset
Multiclass Classification tests.
  • Parameters
    • stattest – statistical test for target drift test.
    • stattest_threshold – threshold for statistical test for target drift test.
Contains tests:
  • TestAccuracyScore
  • TestF1Score
  • TestPrecisionByClass for each class in data
  • TestRecallByClass for each class in data
  • TestNumberOfRows
  • TestColumnValueDrift
  • TestRocAuc
  • TestLogLoss

Attributes:

stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

class NoTargetPerformanceTestPreset(columns: Optional[List[str]] = None, drift_share: Optional[float] = None, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, cat_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, num_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, per_column_stattest: Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None, stattest_threshold: Optional[float] = None, cat_stattest_threshold: Optional[float] = None, num_stattest_threshold: Optional[float] = None, per_column_stattest_threshold: Optional[Dict[str, float]] = None)

Bases: TestPreset
No Target Performance tests.
  • Parameters
    columns – list of columns include to tests
Contains tests:
  • TestColumnValueDrift
  • TestShareOfDriftedColumns
  • TestColumnsType
  • TestAllColumnsShareOfMissingValues
  • TestNumColumnsOutOfRangeValues
  • TestCatColumnsOutOfListValues
  • TestNumColumnsMeanInNSigmas
  • TestCustomFeaturesValueDrift

Attributes:

cat_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
cat_stattest_threshold : Optional[float] = None
columns : Optional[List[str]]
drift_share : Optional[float]
num_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
num_stattest_threshold : Optional[float] = None
per_column_stattest : Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None
per_column_stattest_threshold : Optional[Dict[str, float]] = None
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
stattest_threshold : Optional[float] = None

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

class RegressionTestPreset()

Bases: TestPreset
Regression performance tests.
Contains tests:
  • TestValueMeanError
  • TestValueMAE
  • TestValueRMSE
  • TestValueMAPE

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

Submodules

classification_binary module

class BinaryClassificationTestPreset(stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None, probas_threshold: Optional[float] = None)

Bases: TestPreset
Binary Classification Tests. :param stattest: statistical test for target drift test. :param stattest_threshold: threshold for statistical test for target drift test. :param probas_threshold: threshold for label calculation for prediction.
Contains:
  • TestColumnValueDrift for target
  • TestPrecisionScore
  • TestRecallScore
  • TestF1Score
  • TestAccuracyScore

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

classification_binary_topk module

class BinaryClassificationTopKTestPreset(k: Union[float, int], stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None)

Bases: TestPreset
Binary Classification Tests for Top K threshold. :param stattest: stattest for TestColumnDrift :param stattest_threshold: threshold for stattest
Contains:
  • TestColumnDrift for target
  • TestPrecisionScore
  • TestRecallScore
  • TestF1Score
  • TestAccuracyScore

Attributes:

stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

classification_multiclass module

class MulticlassClassificationTestPreset(stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, stattest_threshold: Optional[float] = None)

Bases: TestPreset
Multiclass Classification tests.
  • Parameters
    • stattest – statistical test for target drift test.
    • stattest_threshold – threshold for statistical test for target drift test.
Contains tests:
  • TestAccuracyScore
  • TestF1Score
  • TestPrecisionByClass for each class in data
  • TestRecallByClass for each class in data
  • TestNumberOfRows
  • TestColumnValueDrift
  • TestRocAuc
  • TestLogLoss

Attributes:

stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

data_drift module

class DataDriftTestPreset(columns: Optional[List[str]] = None, drift_share: Optional[float] = None, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, cat_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, num_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, per_column_stattest: Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None, stattest_threshold: Optional[float] = None, cat_stattest_threshold: Optional[float] = None, num_stattest_threshold: Optional[float] = None, per_column_stattest_threshold: Optional[Dict[str, float]] = None)

Bases: TestPreset
Data Drift tests.
Contains tests:
  • TestShareOfDriftedColumns
  • TestColumnValueDrift
  • TestAllFeaturesValueDrift

Attributes:

cat_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
cat_stattest_threshold : Optional[float]
columns : Optional[List[str]]
drift_share : Optional[float]
num_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
num_stattest_threshold : Optional[float]
per_column_stattest : Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]]
per_column_stattest_threshold : Optional[Dict[str, float]]
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]
stattest_threshold : Optional[float]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

data_quality module

class DataQualityTestPreset(columns: Optional[List[str]] = None)

Bases: TestPreset
Data Quality tests.
Contains tests:
  • TestAllColumnsShareOfMissingValues
  • TestAllColumnsMostCommonValueShare
  • TestNumberOfConstantColumns
  • TestNumberOfDuplicatedColumns
  • TestNumberOfDuplicatedRows
  • TestHighlyCorrelatedColumns

Attributes:

columns : Optional[List[str]]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

data_stability module

class DataStabilityTestPreset(columns: Optional[List[str]] = None)

Bases: TestPreset
Data Stability tests.
Contains tests:
  • TestNumberOfRows
  • TestNumberOfColumns
  • TestColumnsType
  • TestAllColumnsShareOfMissingValues
  • TestNumColumnsOutOfRangeValues
  • TestCatColumnsOutOfListValues
  • TestNumColumnsMeanInNSigmas

Attributes:

columns : Optional[List[str]]

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

no_target_performance module

class NoTargetPerformanceTestPreset(columns: Optional[List[str]] = None, drift_share: Optional[float] = None, stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, cat_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, num_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, per_column_stattest: Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None, stattest_threshold: Optional[float] = None, cat_stattest_threshold: Optional[float] = None, num_stattest_threshold: Optional[float] = None, per_column_stattest_threshold: Optional[Dict[str, float]] = None)

Bases: TestPreset
No Target Performance tests.
  • Parameters
    columns – list of columns include to tests
Contains tests:
  • TestColumnValueDrift
  • TestShareOfDriftedColumns
  • TestColumnsType
  • TestAllColumnsShareOfMissingValues
  • TestNumColumnsOutOfRangeValues
  • TestCatColumnsOutOfListValues
  • TestNumColumnsMeanInNSigmas
  • TestCustomFeaturesValueDrift

Attributes:

cat_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
cat_stattest_threshold : Optional[float] = None
columns : Optional[List[str]]
drift_share : Optional[float]
num_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
num_stattest_threshold : Optional[float] = None
per_column_stattest : Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None
per_column_stattest_threshold : Optional[Dict[str, float]] = None
stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None
stattest_threshold : Optional[float] = None

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

regression module

class RegressionTestPreset()

Bases: TestPreset
Regression performance tests.
Contains tests:
  • TestValueMeanError
  • TestValueMAE
  • TestValueRMSE
  • TestValueMAPE

Methods:

generate_tests(data: InputData, columns: DatasetColumns)

test_preset module

class TestPreset()

Bases: object

Methods:

abstract generate_tests(data: InputData, columns: DatasetColumns)
Previous
evidently.suite
Next
evidently.test_suite
Last modified 9mo ago