evidently.calculations.stattests

Available statistical tests. For detailed information about statistical tests see module documentation.
Submodules
anderson_darling_stattest module
Anderson-Darling test of two samples.
Name: “anderson”
Import:
>>> from evidently.calculations.stattests import anderson_darling_test
Properties:
only for numerical features
returns p-value
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import anderson_darling_test
>>> options = DataDriftOptions(all_features_stattest=anderson_darling_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="anderson")
chisquare_stattest module
Chisquare test of two samples.
Name: “chisquare”
Import:
>>> from evidently.calculations.stattests import chi_stat_test
Properties:
only for categorical features
returns p-value
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import chi_stat_test
>>> options = DataDriftOptions(all_features_stattest=chi_stat_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="chisquare")
cramer_von_mises_stattest module
Cramer-Von-mises test of two samples.
Name: “cramer_von_mises”
Import:
>>> from evidently.calculations.stattests import cramer_von_mises
Properties:
only for numerical features
returns p-value
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import cramer_von_mises
>>> options = DataDriftOptions(all_features_stattest=cramer_von_mises)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="cramer_von_mises")
class CramerVonMisesResult(statistic, pvalue)
Bases: object
energy_distance module
Energy-distance test of two samples.
Name: “ed”
Import:
>>> from evidently.calculations.stattests import energy_dist_test
Properties:
only for numerical features
returns p-value
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import energy_dist_test
>>> options = DataDriftOptions(all_features_stattest=energy_dist_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="ed")
epps_singleton_stattest module
Epps-Singleton test of two samples.
Name: “es”
Import:
>>> from evidently.calculations.stattests import epps_singleton_test
Properties:
only for numerical features
returns p-value
default threshold 0.05
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import epps_singleton_test
>>> options = DataDriftOptions(all_features_stattest=epps_singleton_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="es")
fisher_exact_stattest module
Fisher’s exact test of two samples.
Name: “fisher_exact”
Import:
>>> from evidently.calculations.stattests import fisher_exact_test
Properties:
only for categorical features
returns p-value
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import fisher_exact_test
>>> options = DataDriftOptions(all_features_stattest=fisher_exact_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="fisher_exact")
g_stattest module
G-test of two samples.
Name: “g_test”
Import:
>>> from evidently.calculations.stattests import g_test
Properties:
only for categorical features
returns p-value
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import g_test
>>> options = DataDriftOptions(all_features_stattest=g_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="g_test")
hellinger_distance module
Hellinger distance of two samples.
Name: “hellinger”
Import:
>>> from evidently.calculations.stattests import hellinger_stat_test
Properties:
only for categorical and numerical features
returns distance
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import hellinger_stat_test
>>> options = DataDriftOptions(all_features_stattest=hellinger_stat_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="hellinger")
jensenshannon module
Jensen-Shannon distance of two samples.
Name: “jensenshannon”
Import:
>>> from evidently.calculations.stattests import jensenshannon_stat_test
Properties:
only for categorical and numerical features
returns distance
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import jensenshannon_stat_test
>>> options = DataDriftOptions(all_features_stattest=jensenshannon_stat_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="jensenshannon")
kl_div module
Kullback-Leibler divergence of two samples.
Name: “kl_div”
Import:
>>> from evidently.calculations.stattests import kl_div_stat_test
Properties:
only for categorical and numerical features
returns divergence
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import kl_div_stat_test
>>> options = DataDriftOptions(all_features_stattest=kl_div_stat_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="kl_div")
ks_stattest module
Kolmogorov-Smirnov test of two samples.
Name: “ks”
Import:
>>> from evidently.calculations.stattests import ks_stat_test
Properties:
only for numerical features
returns p-value
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import ks_stat_test
>>> options = DataDriftOptions(all_features_stattest=ks_stat_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="ks")
mann_whitney_urank_stattest module
Mann-Whitney U-rank test of two samples.
Name: “mannw”
Import:
>>> from evidently.calculations.stattests import mann_whitney_u_stat_test
Properties:
only for numerical features
returns p-value
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import mann_whitney_u_stat_test
>>> options = DataDriftOptions(all_features_stattest=mann_whitney_u_stat_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="mannw")
psi module
PSI of two samples.
Name: “psi”
Import:
>>> from evidently.calculations.stattests import psi_stat_test
Properties:
only for categorical and numerical features
returns PSI value
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import psi_stat_test
>>> options = DataDriftOptions(all_features_stattest=psi_stat_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="psi")
registry module
class StatTest(name: str, display_name: str, func: Callable[[pandas.core.series.Series, pandas.core.series.Series, str, float], Tuple[float, bool]], allowed_feature_types: List[str], default_threshold: float = 0.05)
Bases: object
Attributes:
     allowed_feature_types : List[str]
     default_threshold : float = 0.05
     display_name : str
     func : Callable[[Series, Series, str, float], Tuple[float, bool]]
     name : str
exception StatTestInvalidFeatureTypeError(stattest_name: str, feature_type: str)
Bases: ValueError
exception StatTestNotFoundError(stattest_name: str)
Bases: ValueError
class StatTestResult(drift_score: float, drifted: bool, actual_threshold: float)
Bases: object
Attributes:
     actual_threshold : float
     drift_score : float
     drifted : bool
get_stattest(reference_data: Series, current_data: Series, feature_type: str, stattest_func: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]])
register_stattest(stat_test: StatTest)
t_test module
T test of two samples.
Name: “t_test”
Import:
>>> from evidently.calculations.stattests import t_test
Properties:
only for numerical features
returns p-value
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import t_test
>>> options = DataDriftOptions(all_features_stattest=t_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="t_test")
tvd_stattest module
Total variation distance of two samples.
Name: “TVD”
Import:
>>> from evidently.calculations.stattests import tvd_test
Properties:
only for numerical features
returns distance
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import tvd_test
>>> options = DataDriftOptions(all_features_stattest=tvd_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="TVD")
utils module
generate_fisher2x2_contingency_table(reference_data: Series, current_data: Series)
Generate 2x2 contingency matrix for fisher exact test :param reference_data: reference data :param current_data: current data
Raises
ValueError – if reference_data and current_data are not of equal length
Returns
contingency_matrix for binary data
Return type
contingency_matrix
get_binned_data(reference_data: Series, current_data: Series, feature_type: str, n: int, feel_zeroes: bool = True)
Split variable into n buckets based on reference quantiles :param reference_data: reference data :param current_data: current data :param feature_type: feature type :param n: number of quantiles
Returns
% of records in each bucket for reference current_percents: % of records in each bucket for current
Return type
reference_percents
get_unique_not_nan_values_list_from_series(current_data: Series, reference_data: Series)
Get unique values from current and reference series, drop NaNs
permutation_test(reference_data, current_data, observed, test_statistic_func, iterations=100)
Perform a two-sided permutation test :param reference_data: reference data :param current_data: current data :param observed: observed value :param test_statistic_func: the test statistic function :param iterations: number of times to permute
Returns
two-sided p_value
Return type
p_value
wasserstein_distance_norm module
Wasserstein distance of two samples.
Name: “wasserstein”
Import:
>>> from evidently.calculations.stattests import wasserstein_stat_test
Properties:
only for numerical features
returns p-value
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import wasserstein_stat_test
>>> options = DataDriftOptions(all_features_stattest=wasserstein_stat_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="wasserstein")
z_stattest module
Mann-Whitney U-rank test of two samples.
Name: “mannw”
Import:
>>> from evidently.calculations.stattests import mann_whitney_u_stat_test
Properties:
only for numerical features
returns p-value
Example
Using by object:
>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import mann_whitney_u_stat_test
>>> options = DataDriftOptions(all_features_stattest=mann_whitney_u_stat_test)
Using by name:
>>> from evidently.options import DataDriftOptions
>>> options = DataDriftOptions(all_features_stattest="mannw")
proportions_diff_z_stat_ind(ref: DataFrame, curr: DataFrame)
proportions_diff_z_test(z_stat, alternative='two-sided')
evidently.calculations
evidently.metrics
Last modified 6mo ago