Estimators

The skfb.estimators module implements fallback meta-estimators that extend classification models with a reject option. They offer configurations allowing a model to refuse to make predictions (and return fallback labels), accept all inputs but mask rejections in background, or ignore rejections. Can be used in high-stakes environments, where instead of accepting all inputs, the model needs to delegate potential anomalies or uncertainties to other specialists.

Threshold-Based Rejectors

The following fallback estimators accept and/or learn hard rules for rejection, e.g., based on certainty thresholds.

class skfb.estimators.ThresholdFallbackClassifier(estimator, *, threshold=0.5, ambiguity_threshold=0.0, fallback_label=-1, fallback_mode='store')[source]

A fallback classifier based on provided certainty threshold.

Formally, let $p(x) = [p_1(x),\dots,p_K(x)]$ be the class probability vector returned by the wrapped estimator for a sample $x$. A sample is rejected whenever the maximum probability or the difference between top two probabilities are low:

\[\max_k p_k(x) < \tau \quad \text{or} \quad \max_k p_k(x) - \text{second\_max}_k p_k(x) < \alpha\]

where $\tau$ is threshold and $\alpha$ is ambiguity_threshold. The final output is

\[\begin{split}y(x) = \begin{cases} \arg\max_k p_k(x) & \text{if accepted},\\ \text{fallback\_label} & \text{otherwise}. \end{cases}\end{split}\]

The return format depends on fallback_mode.

Parameters:

estimator (object) – The base estimator making decisions w/o fallbacks.
threshold (float, default=0.5) – The fallback threshold.
ambiguity_threshold (float, default=0.0) – Predictions w/ the close top 2 scores are rejected.
fallback_label (any, default=-1) – The label of a rejected example. Should be compatible w/ the class labels from training data.
fallback_mode ({"return", "store", "ignore"}, default="store") –
While predicting w/ the predict method, whether to return:
- ("return") a numpy ndarray of both predictions and fallbacks;
- ("store") an FBNDArray of predictions storing also fallback mask;
- ("ignore") a numpy ndarray of only estimator’s predictions.
Calling decision_function or predict_proba is equivalent to estimator’s corresponding calls except that w/ "store", FBNDArray is returned.

Examples

>>> import numpy as np
>>> from sklearn.linear_model import LogisticRegression
>>> from skfb.estimators import ThresholdFallbackClassifier
>>> X = np.array([[0, 0], [4, 4], [1, 1], [3, 3], [2.5, 2], [2., 2.5]])
>>> y = np.array([0, 1, 0, 1, 0, 1])
>>> estimator = LogisticRegression(random_state=0)
>>> rejector = ThresholdFallbackClassifier(estimator, threshold=0.6).fit(X, y)
>>> y_pred = rejector.predict(X)
>>> y_pred, y_pred.get_dense_fallback_mask()
(FBNDArray([0, 1, 0, 1, 1, 1]),
 array([False, False, False, False,  True, False]))
>>> rejector.set_params(fallback_mode="return").predict(X)
array([ 0,  1,  0,  1, -1,  1])
>>> rejector.score(X, y)
1.0
>>> rejector.set_params(fallback_mode="store").score(X, y)
0.8333333333333334

See also

skfb.estimators.ThresholdFallbackClassifier: Fallback classification based on fixed threshold.

Methods

`decision_function`(X)	Calls `decision_function` on the estimator.
`fit`(X, y, **fit_params)	Trains base estimator and sets meta-estimator attributes.
`get_metadata_routing`()	Get metadata routing of this object.
`get_params`([deep])	Get parameters for this estimator.
`predict`(X)	Makes predictions using the base estimator and the fallback rules.
`predict_log_proba`(X)	Returns log of `predict_proba` on the estimator.
`predict_proba`(X)	Calls `predict_proba` on the estimator.
`score`(X, y)	Evaluates an accuracy score.
`set_params`(**params)	Set the parameters of this estimator.
`validate_fallback_label`(fallback_label, classes)	Checks if fallback label is compatible w/ classes from training data..

decision_function(X)

Calls decision_function on the estimator.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_score – Result of the decision function for X based on the estimator.
Return type:: ndarray of shape (n_samples,) or (n_samples, n_classes) or (n_samples, n_classes * (n_classes-1) / 2)

fit(X, y, **fit_params)

Trains base estimator and sets meta-estimator attributes.

If X and y are valid, obtains unique classes and saves in self.classes_. New self.fallback_label_ is set to be of the same type as self.classes_.

Parameters:

X ({array-like, sparse matrix}, shape (n_samples, n_features)) – The training input samples.
y (array-like, shape (n_samples,) or (n_samples, n_outputs)) – The target values (class labels in classification, real numbers in regression).

Returns:

self – Returns self.

Return type:

object

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:: routing – A MetadataRequest encapsulating routing information.
Return type:: MetadataRequest

get_params(deep=True)

Get parameters for this estimator.

Parameters:: deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns:: params – Parameter names mapped to their values.
Return type:: dict

predict(X)

Makes predictions using the base estimator and the fallback rules.

Parameters:

X (indexable, length n_samples)) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.

Returns:

y_pred – Depending on self.fallback_mode:

("return") a numpy ndarray of both predictions and fallbacks, or;
("store") an FBNDArray of predictions storing also fallback mask, or;
("ignore") a numpy ndarray of only estimator’s predictions.

Return type:

ndarray or FBNDArray of shape (n_samples,)

predict_log_proba(X)

Returns log of predict_proba on the estimator.

If fallback_mode != “ignore”, returns FBNDArray w/ fallback mask.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_pred – Predicted class log-probabilities for X based on the estimator. The order of the classes corresponds to that in the fitted attribute classes_. If self.fallback_mode == "ignore", returns an ndarray.
Return type:: FBNDArray or ndarray of shape (n_samples, n_classes)

predict_proba(X)

Calls predict_proba on the estimator.

If fallback_mode != “ignore”, returns FBNDArray w/ fallback mask.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_pred – Predicted class probabilities for X based on the estimator. The order of the classes corresponds to that in the fitted attribute classes_. If self.fallback_mode == "ignore", returns an ndarray.
Return type:: FBNDArray or ndarray of shape (n_samples, n_classes)

score(X, y)

Evaluates an accuracy score.

Parameters:

X (indexable, length n_samples) – Input samples to evaluate. Must fulfill the input assumptions of the underlying estimator.
y (array-like of shape (n_samples,)) – True labels for X (excluding fallback label).

Returns:

score – Depending on self.fallback_mode, calculate:

("return") accuracy on accepted inputs;
("store") predict_reject_accuracy_score();
("ignore") accuracy on all inputs.

Return type:

float

See also

skfb.metrics.predict_reject_accuracy_score: Combined, prediction-rejection accuracy score.
skfb.metrics.predict_reject_recall_score: Combined, prediction-rejection balanced accuracy score.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:: **params (dict) – Estimator parameters.
Returns:: self – Estimator instance.
Return type:: estimator instance

static validate_fallback_label(fallback_label, classes): Checks if fallback label is compatible w/ classes from training data..

class skfb.estimators.ThresholdFallbackClassifierCV(estimator, *, thresholds=None, ambiguity_threshold=0.0, cv=None, scoring=None, n_jobs=None, verbose=0, fallback_label=-1, fallback_mode='store')[source]

A fallback classifier based on the best certainty threshold learnt via CV.

Training evaluates a candidate set of thresholds $T$ by cross-validation. For each threshold $t\in T$ and fold, compute the validation score $s_{\text{fold}}(t)$ using the supplied scoring function. The chosen threshold maximizes the mean score:

\[t^{\ast} = \arg\max_{t\in T} \frac{1}{n_{\text{folds}}} \sum_{\text{fold}} s_{\text{fold}}(t).\]

The selected value is stored in threshold_.

Predictions then follow the same rule as ThresholdFallbackClassifier using threshold_.

Parameters:

estimator (object) – The base estimator making decisions.
thresholds (array-like of shape (n_thresholds,) or int, default=None) – Array of fallback thresholds to evaluate. If None, defaults to 10 thresholds from p = 1 / len(classes), which is about not falling back, to 0.95. Same with int except that the number of threshold equals this value.
ambiguity_threshold (float, default=0.0) – Predictions w/ the close top 2 scores are rejected.
cv (int, cross-validation generator or an iterable, default=None) –
Determines the cross-validation splitting strategy. Possible inputs for cv are:
- None, to use the efficient Leave-One-Out cross-validation
- integer, to specify the number of folds.
- CV splitter,
- An iterable yielding (train, test) splits as arrays of indices.
For integer/None inputs, if y is binary or multiclass, StratifiedKFold is used.
scoring (callable or str, default=None) – A scorer callable object supporting a reject option, such as metrics from metrics. If not from scikit-learn, make sure to wrap it with make_scorer.
verbose (int, default=0) – Verbosity level .
fallback_label (any, default=-1) – The label of a rejected example. Should be compatible w/ the class labels from training data.
fallback_mode ({"return", "store", "ignore"}, default="store") –
While predicting w/ the predict method, whether to return:
- ("return") a numpy ndarray of both predictions and fallbacks;
- ("store") an FBNDArray of predictions storing also fallback mask;
- ("ignore") a numpy ndarray of only estimator’s predictions.
Calling decision_function or predict_proba is equivalent to estimator’s corresponding calls except that with "store", FBNDArray is returned.

threshold_

Learned fallback threshold.

Type:: float

best_score_

Best CV score.

Type:: float

cv_scores_

Trace of CV evaluations.

Type:: ndarray, shape (n_thresholds, n_splits)

Examples

>>> import numpy as np
>>> from sklearn.linear_model import LogisticRegression
>>> from skfb.estimators import ThresholdFallbackClassifierCV
>>> X = np.array([[0, 0], [4, 4], [1, 1], [3, 3], [2.5, 2], [2., 2.5]])
>>> y = np.array([0, 1, 0, 1, 0, 1])
>>> estimator = LogisticRegression(random_state=0)
>>> rejector = ThresholdFallbackClassifierCV(
...     estimator=estimator,
...     thresholds=(0.5, 0.55, 0.6, 0.65),
...     cv=2)
>>> rejector.fit(X, y)
ThresholdFallbackClassifierCV(cv=2,
                              estimator=LogisticRegression(random_state=0),
                              n_jobs=2, thresholds=(0.5, 0.55, 0.6, 0.65),
                              verbose=1)
>>> rejector.threshold_
0.55
>>> rejector.best_score_
0.8333333333333333
>>> y_pred = rejector.predict(X)
>>> y_pred, y_pred.get_dense_fallback_mask()
(FBNDArray([0, 1, 0, 1, 1, 1]),
 array([False, False, False, False,  True, False]))
>>> rejector.set_params(fallback_mode="return").predict(X)
array([ 0,  1,  0,  1, -1,  1])
>>> rejector.score(X, y)
1.0
>>> rejector.set_params(fallback_mode="store").score(X, y)
1.0

Methods

`decision_function`(X)	Calls `decision_function` on the estimator.
`fit`(X, y, **fit_params)	Trains base estimator and sets meta-estimator attributes.
`get_metadata_routing`()	Get metadata routing of this object.
`get_params`([deep])	Get parameters for this estimator.
`predict`(X)	Makes predictions using the base estimator and the fallback rules.
`predict_log_proba`(X)	Returns log of `predict_proba` on the estimator.
`predict_proba`(X)	Calls `predict_proba` on the estimator.
`score`(X, y)	Evaluates an accuracy score.
`set_params`(**params)	Set the parameters of this estimator.
`validate_fallback_label`(fallback_label, classes)	Checks if fallback label is compatible w/ classes from training data..

decision_function(X)

Calls decision_function on the estimator.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_score – Result of the decision function for X based on the estimator.
Return type:: ndarray of shape (n_samples,) or (n_samples, n_classes) or (n_samples, n_classes * (n_classes-1) / 2)

fit(X, y, **fit_params)

Trains base estimator and sets meta-estimator attributes.

If X and y are valid, obtains unique classes and saves in self.classes_. New self.fallback_label_ is set to be of the same type as self.classes_.

Parameters:

X ({array-like, sparse matrix}, shape (n_samples, n_features)) – The training input samples.
y (array-like, shape (n_samples,) or (n_samples, n_outputs)) – The target values (class labels in classification, real numbers in regression).

Returns:

self – Returns self.

Return type:

object

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:: routing – A MetadataRequest encapsulating routing information.
Return type:: MetadataRequest

get_params(deep=True)

Get parameters for this estimator.

Parameters:: deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns:: params – Parameter names mapped to their values.
Return type:: dict

predict(X)

Makes predictions using the base estimator and the fallback rules.

Parameters:

X (indexable, length n_samples)) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.

Returns:

y_pred – Depending on self.fallback_mode:

("return") a numpy ndarray of both predictions and fallbacks, or;
("store") an FBNDArray of predictions storing also fallback mask, or;
("ignore") a numpy ndarray of only estimator’s predictions.

Return type:

ndarray or FBNDArray of shape (n_samples,)

predict_log_proba(X)

Returns log of predict_proba on the estimator.

If fallback_mode != “ignore”, returns FBNDArray w/ fallback mask.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_pred – Predicted class log-probabilities for X based on the estimator. The order of the classes corresponds to that in the fitted attribute classes_. If self.fallback_mode == "ignore", returns an ndarray.
Return type:: FBNDArray or ndarray of shape (n_samples, n_classes)

predict_proba(X)

Calls predict_proba on the estimator.

If fallback_mode != “ignore”, returns FBNDArray w/ fallback mask.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_pred – Predicted class probabilities for X based on the estimator. The order of the classes corresponds to that in the fitted attribute classes_. If self.fallback_mode == "ignore", returns an ndarray.
Return type:: FBNDArray or ndarray of shape (n_samples, n_classes)

score(X, y)

Evaluates an accuracy score.

Parameters:

X (indexable, length n_samples) – Input samples to evaluate. Must fulfill the input assumptions of the underlying estimator.
y (array-like of shape (n_samples,)) – True labels for X (excluding fallback label).

Returns:

score – Depending on self.fallback_mode, calculate:

("return") accuracy on accepted inputs;
("store") predict_reject_accuracy_score();
("ignore") accuracy on all inputs.

Return type:

float

See also

skfb.metrics.predict_reject_accuracy_score: Combined, prediction-rejection accuracy score.
skfb.metrics.predict_reject_recall_score: Combined, prediction-rejection balanced accuracy score.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:: **params (dict) – Estimator parameters.
Returns:: self – Estimator instance.
Return type:: estimator instance

static validate_fallback_label(fallback_label, classes): Checks if fallback label is compatible w/ classes from training data..

class skfb.estimators.MultiThresholdFallbackClassifier(estimator, *, thresholds, ambiguity_threshold=0.0, fallback_label=-1, fallback_mode='store')[source]

A fallback classifier based on local (class-wise) thresholds.

Formally, let $p(x) = [p_1(x),\dots,p_K(x)]$ be the class probability vector returned by the wrapped estimator for a sample $x$ and $\tau = [t_1,\dots,t_K]$ local thresholds. A sample is rejected whenever the maximum probability or the difference between top two probabilities are low:

\[\max_k p_k(x) < \tau_{k^*} \quad \text{or} \quad \max_k p_k(x) - \text{second\_max}_k p_k(x) < \alpha\]

where $k^* = \arg\max_k p_k(x)$, $\tau$ is thresholds, and $\alpha$ is ambiguity_threshold. The final output is

\[\begin{split}y(x) = \begin{cases} \arg\max_k p_k(x) & \text{if accepted},\\ \text{fallback\_label} & \text{otherwise}. \end{cases}\end{split}\]

The return format depends on fallback_mode.

Parameters:

estimator (object) – The best estimator making decisions w/o fallbacks.
thresholds (array-like, shape (n_classes,)) – Local thresholds, a threshold for each class.
ambiguity_threshold (float, default=0.0) – Predictions w/ the close top 2 scores are rejected.
fallback_label (any, default=-1) – The label of a rejected example. Should be compatible w/ the class labels from training data.
fallback_mode ({"return", "store", "ignore"}, default="store") –
While predicting w/ the predict method, whether to return:
- ("return") a numpy ndarray of both predictions and fallbacks;
- ("store") an FBNDArray of predictions storing also fallback mask;
- ("ignore") a numpy ndarray of only estimator’s predictions.
Calling decision_function or predict_proba is equivalent to estimator’s corresponding calls except that with "store", FBNDArray is returned.

Examples

>>> import numpy as np
>>> from sklearn.linear_model import LogisticRegression
>>> from skfb.estimators import MultiThresholdFallbackClassifier
>>> X = np.array(
...     [
...         [-3, -3], [-3, -2], [-2, -3],
...         [2, 3], [2, 4], [3, 4], [3.1, 3], [3, 2.9],
...         [4, 3], [3, 2], [4, 2], [2.9, 3], [3, 3.1],
...     ])
>>> y = np.array(["a"] * 3 + ["b"] * 5 + ["c"] * 5)
>>> estimator = LogisticRegression(C=10_000, random_state=0)
>>> thresholds = [0.99, 0.8, 0.75]
>>> r = MultiThresholdFallbackClassifier(estimator, thresholds=thresholds)
>>> r.set_params(fallback_label="d", fallback_mode="return").fit(X, y).predict(X)
array(['a', 'a', 'a', 'b', 'b', 'b', 'd', 'd', 'c', 'c', 'c', 'd', 'd'],
      dtype='<U1')

See also

skfb.estimators.ThresholdFallbackClassifier: Fallback classification based on fixed threshold.

Methods

`decision_function`(X)	Calls `decision_function` on the estimator.
`fit`(X, y, **fit_params)	Trains base estimator and sets meta-estimator attributes.
`get_metadata_routing`()	Get metadata routing of this object.
`get_params`([deep])	Get parameters for this estimator.
`predict`(X)	Makes predictions using the base estimator and the fallback rules.
`predict_log_proba`(X)	Returns log of `predict_proba` on the estimator.
`predict_proba`(X)	Calls `predict_proba` on the estimator.
`score`(X, y)	Evaluates an accuracy score.
`set_params`(**params)	Set the parameters of this estimator.
`validate_fallback_label`(fallback_label, classes)	Checks if fallback label is compatible w/ classes from training data..

decision_function(X)

Calls decision_function on the estimator.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_score – Result of the decision function for X based on the estimator.
Return type:: ndarray of shape (n_samples,) or (n_samples, n_classes) or (n_samples, n_classes * (n_classes-1) / 2)

fit(X, y, **fit_params)

Trains base estimator and sets meta-estimator attributes.

If X and y are valid, obtains unique classes and saves in self.classes_. New self.fallback_label_ is set to be of the same type as self.classes_.

Parameters:

X ({array-like, sparse matrix}, shape (n_samples, n_features)) – The training input samples.
y (array-like, shape (n_samples,) or (n_samples, n_outputs)) – The target values (class labels in classification, real numbers in regression).

Returns:

self – Returns self.

Return type:

object

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:: routing – A MetadataRequest encapsulating routing information.
Return type:: MetadataRequest

get_params(deep=True)

Get parameters for this estimator.

Parameters:: deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns:: params – Parameter names mapped to their values.
Return type:: dict

predict(X)

Makes predictions using the base estimator and the fallback rules.

Parameters:

X (indexable, length n_samples)) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.

Returns:

y_pred – Depending on self.fallback_mode:

("return") a numpy ndarray of both predictions and fallbacks, or;
("store") an FBNDArray of predictions storing also fallback mask, or;
("ignore") a numpy ndarray of only estimator’s predictions.

Return type:

ndarray or FBNDArray of shape (n_samples,)

predict_log_proba(X)

Returns log of predict_proba on the estimator.

If fallback_mode != “ignore”, returns FBNDArray w/ fallback mask.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_pred – Predicted class log-probabilities for X based on the estimator. The order of the classes corresponds to that in the fitted attribute classes_. If self.fallback_mode == "ignore", returns an ndarray.
Return type:: FBNDArray or ndarray of shape (n_samples, n_classes)

predict_proba(X)

Calls predict_proba on the estimator.

If fallback_mode != “ignore”, returns FBNDArray w/ fallback mask.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_pred – Predicted class probabilities for X based on the estimator. The order of the classes corresponds to that in the fitted attribute classes_. If self.fallback_mode == "ignore", returns an ndarray.
Return type:: FBNDArray or ndarray of shape (n_samples, n_classes)

score(X, y)

Evaluates an accuracy score.

Parameters:

X (indexable, length n_samples) – Input samples to evaluate. Must fulfill the input assumptions of the underlying estimator.
y (array-like of shape (n_samples,)) – True labels for X (excluding fallback label).

Returns:

score – Depending on self.fallback_mode, calculate:

("return") accuracy on accepted inputs;
("store") predict_reject_accuracy_score();
("ignore") accuracy on all inputs.

Return type:

float

See also

skfb.metrics.predict_reject_accuracy_score: Combined, prediction-rejection accuracy score.
skfb.metrics.predict_reject_recall_score: Combined, prediction-rejection balanced accuracy score.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:: **params (dict) – Estimator parameters.
Returns:: self – Estimator instance.
Return type:: estimator instance

static validate_fallback_label(fallback_label, classes): Checks if fallback label is compatible w/ classes from training data..

class skfb.estimators.RateFallbackClassifierCV(estimator, *, fallback_rate=0.1, cv=None, n_jobs=None, verbose=0, fallback_label=-1, fallback_mode='store')[source]

Fallback classifier learning a threshold based on the desired fallback rate.

Let $r$ denote the requested fallback rate. For each cross-validation fold, the base estimator is fitted on the training split and the maximum class probabilities $p_{\max}(x)$ are computed on the validation split. The per-fold threshold is the empirical $r$-quantile of those values:

\[t_{\text{fold}} = Q_r\{p_{\max}(x) : x\in \text{fold}\}.\]

The learned threshold is the average across folds:

\[t^{\ast} = \frac{1}{n_{\text{folds}}} \sum_{\text{fold}} t_{\text{fold}}.\]

The selected average is stored in threshold_. All thresholds are in thresholds_.

Predictions then follow the same rule as ThresholdFallbackClassifier using threshold_.

Parameters:

estimator (object) – The base estimator supporting probabilistic predictions.
fallback_rate (float, default=0.1) – The rate of rejected test samples.
cv (int, cross-validation generator or an iterable, default=None) –
Determines the cross-validation splitting strategy. Possible inputs for cv are:
- None, to use the efficient Leave-One-Out cross-validation
- integer, to specify the number of folds.
- CV splitter,
- An iterable yielding (train, test) splits as arrays of indices.
For integer/None inputs, if y is binary or multiclass, StratifiedKFold is used.
verbose (int, default=0) – Verbosity level.
fallback_label (any, default=-1) – The label of a rejected example. Should be compatible w/ the class labels from training data.
fallback_mode ({"return", "store"}, default="store") –
While predicting w/ the predict method, whether to return:
- ("return") a numpy ndarray of both predictions and fallbacks;
- ("store") an FBNDArray of predictions storing also fallback mask;
- ("ignore") a numpy ndarray of only estimator’s predictions.
Calling decision_function or predict_proba is equivalent to estimator’s corresponding calls except that with "store", FBNDArray is returned.

Examples

>>> import numpy as np
>>> from sklearn.linear_model import LogisticRegression
>>> from skfb.estimators import RateFallbackClassifierCV
>>> estimator = LogisticRegression(random_state=0)
>>> X_accept = [[0, 0], [6, 6], [0, 1], [5, 6], [1, 1], [5, 5], [1, 0], [6, 5]]
>>> X_ambiguous = [[3.25, 3], [3., 3.25]]
>>> X = np.array(X_accept + X_ambiguous)
>>> y = np.array([0, 1, 0, 1, 0, 1, 0, 1, 0, 1])
>>> rejector = RateFallbackClassifierCV(
...     estimator,
...     fallback_rate=0.2,
...     cv=2,
...     fallback_label=-1,
...     fallback_mode="return")
>>> rejector.fit(X, y).score(X, y)  # Both ambiguous samples were rejected.
1.0
>>> rejector.set_params(fallback_mode="store").score(X, y)
0.9

See also

skfb.estimators.ThresholdFallbackClassifier: Fallback classification based on fixed threshold.

Methods

`decision_function`(X)	Calls `decision_function` on the estimator.
`fit`(X, y, **fit_params)	Trains base estimator and sets meta-estimator attributes.
`get_metadata_routing`()	Get metadata routing of this object.
`get_params`([deep])	Get parameters for this estimator.
`predict`(X)	Makes predictions using the base estimator and the fallback rules.
`predict_log_proba`(X)	Returns log of `predict_proba` on the estimator.
`predict_proba`(X)	Calls `predict_proba` on the estimator.
`score`(X, y)	Evaluates an accuracy score.
`set_params`(**params)	Set the parameters of this estimator.
`validate_fallback_label`(fallback_label, classes)	Checks if fallback label is compatible w/ classes from training data..

decision_function(X)

Calls decision_function on the estimator.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_score – Result of the decision function for X based on the estimator.
Return type:: ndarray of shape (n_samples,) or (n_samples, n_classes) or (n_samples, n_classes * (n_classes-1) / 2)

fit(X, y, **fit_params)

Trains base estimator and sets meta-estimator attributes.

If X and y are valid, obtains unique classes and saves in self.classes_. New self.fallback_label_ is set to be of the same type as self.classes_.

Parameters:

X ({array-like, sparse matrix}, shape (n_samples, n_features)) – The training input samples.
y (array-like, shape (n_samples,) or (n_samples, n_outputs)) – The target values (class labels in classification, real numbers in regression).

Returns:

self – Returns self.

Return type:

object

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:: routing – A MetadataRequest encapsulating routing information.
Return type:: MetadataRequest

get_params(deep=True)

Get parameters for this estimator.

Parameters:: deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns:: params – Parameter names mapped to their values.
Return type:: dict

predict(X)

Makes predictions using the base estimator and the fallback rules.

Parameters:

X (indexable, length n_samples)) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.

Returns:

y_pred – Depending on self.fallback_mode:

("return") a numpy ndarray of both predictions and fallbacks, or;
("store") an FBNDArray of predictions storing also fallback mask, or;
("ignore") a numpy ndarray of only estimator’s predictions.

Return type:

ndarray or FBNDArray of shape (n_samples,)

predict_log_proba(X)

Returns log of predict_proba on the estimator.

If fallback_mode != “ignore”, returns FBNDArray w/ fallback mask.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_pred – Predicted class log-probabilities for X based on the estimator. The order of the classes corresponds to that in the fitted attribute classes_. If self.fallback_mode == "ignore", returns an ndarray.
Return type:: FBNDArray or ndarray of shape (n_samples, n_classes)

predict_proba(X)

Calls predict_proba on the estimator.

If fallback_mode != “ignore”, returns FBNDArray w/ fallback mask.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_pred – Predicted class probabilities for X based on the estimator. The order of the classes corresponds to that in the fitted attribute classes_. If self.fallback_mode == "ignore", returns an ndarray.
Return type:: FBNDArray or ndarray of shape (n_samples, n_classes)

score(X, y)

Evaluates an accuracy score.

Parameters:

X (indexable, length n_samples) – Input samples to evaluate. Must fulfill the input assumptions of the underlying estimator.
y (array-like of shape (n_samples,)) – True labels for X (excluding fallback label).

Returns:

score – Depending on self.fallback_mode, calculate:

("return") accuracy on accepted inputs;
("store") predict_reject_accuracy_score();
("ignore") accuracy on all inputs.

Return type:

float

See also

skfb.metrics.predict_reject_accuracy_score: Combined, prediction-rejection accuracy score.
skfb.metrics.predict_reject_recall_score: Combined, prediction-rejection balanced accuracy score.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:: **params (dict) – Estimator parameters.
Returns:: self – Estimator instance.
Return type:: estimator instance

static validate_fallback_label(fallback_label, classes): Checks if fallback label is compatible w/ classes from training data..

class skfb.estimators.RuleClassifier(*, validate=False, kwargs=None)[source]

ABC that defines rule-based classification.

Reimplement _predict by defining custom classification rules (e.g., label a transaction fraudulent if a receiver is in a blacklist; otherwise, it’s genuine). You can introduce new rule arguments either via reimplemented __init__ or by passing them as kwargs during model instatiation.

Parameters:

validate (bool, default=False) – Whether to validate reimplemented prediction method. Validation means checking if prediction doesn’t result in any exception.
kwargs (dict, default=None) – Any (tunable) argument required to make predictions. Each argument is returned and can be set individually by get_params and set_params, respectively.

Methods

`fit`(X[, y, sample_weight])	Fits the estimator and sets fit attributes.
`get_metadata_routing`()	Get metadata routing of this object.
`get_params`([deep])	Gets parameters for the estimator.
`partial_fit`(X, y[, classes, sample_weight])	Fits the estimator partially.
`predict`(X)	Predicts hard labels.
`score`(X, y[, sample_weight])	Return the mean accuracy on the given test data and labels.
`set_fit_request`(*[, sample_weight])	Request metadata passed to the `fit` method.
`set_params`(**params)	Sets the parameters of the estimator.
`set_partial_fit_request`(*[, classes, ...])	Request metadata passed to the `partial_fit` method.
`set_score_request`(*[, sample_weight])	Request metadata passed to the `score` method.
`validate_predict`(X)	Validates inference methods.

fit(X, y=None, sample_weight=None)[source]

Fits the estimator and sets fit attributes.

Parameters:

X ({array-like, sparse matrix}, shape (n_samples, n_features)) – The training input samples.
y (array-like, shape (n_samples,) or (n_samples, n_outputs)) – The target values.

Returns:

self – Returns self.

Return type:

object

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:: routing – A MetadataRequest encapsulating routing information.
Return type:: MetadataRequest

get_params(deep=True)[source]: Gets parameters for the estimator.

partial_fit(X, y, classes=None, sample_weight=None)[source]: Fits the estimator partially.

predict(X)[source]

Predicts hard labels.

Parameters:: X (indexable, length n_samples) – Input samples to classify.
Returns:: y_pred – Predicted hard labels.
Return type:: ndarray of shape (n_samples,)

score(X, y, sample_weight=None)

Return the mean accuracy on the given test data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters:

X (array-like of shape (n_samples, n_features)) – Test samples.
y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True labels for X.
sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.

Returns:

score – Mean accuracy of self.predict(X) w.r.t. y.

Return type:

float

set_fit_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → RuleClassifier

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to fit.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in fit.
Returns:: self – The updated object.
Return type:: object

set_params(**params)[source]: Sets the parameters of the estimator.

set_partial_fit_request(*, classes: bool | None | str = '$UNCHANGED$', sample_weight: bool | None | str = '$UNCHANGED$') → RuleClassifier

Request metadata passed to the partial_fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to partial_fit if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to partial_fit.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

classes (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for classes parameter in partial_fit.
sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in partial_fit.

Returns:

self – The updated object.

Return type:

object

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → RuleClassifier

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

validate_predict(X)[source]

Validates inference methods.

Raises:: An exception raised by one of the methods. –

class skfb.estimators.FallbackRuleClassifier(*, validate=False, fallback_label=-1, kwargs=None)[source]

Rule-based fallback classification.

Reimplement _predict by defining custom classification rules, including fallbacks (e.g., label a transaction fraudulent if a receiver is in a blacklist; otherwise, it’s either genuine for regular transactions or anomalous for irregular ones). You can introduce new rule arguments either via reimplemented __init__ or by passing them as kwargs during model instatiation.

Parameters:

validate (bool, default=False) – Whether to validate reimplemented prediction method. Validation means checking if prediction doesn’t result in any exception.
fallback_label (any, default=-1) – Label returned by fallback rules.
kwargs (dict, default=None) – Any (tunable) argument required to make predictions. Each argument is returned and can be set individually by get_params and set_params, respectively.

Methods

`fit`(X[, y, sample_weight])	Fits the estimator and sets fit attributes.
`get_metadata_routing`()	Get metadata routing of this object.
`get_params`([deep])	Gets parameters for the estimator.
`partial_fit`(X, y[, classes, sample_weight])	Fits the estimator partially.
`predict`(X)	Predicts hard labels.
`score`(X, y)	Evaluates an accuracy score on accepted samples.
`set_fit_request`(*[, sample_weight])	Request metadata passed to the `fit` method.
`set_params`(**params)	Sets the parameters of the estimator.
`set_partial_fit_request`(*[, classes, ...])	Request metadata passed to the `partial_fit` method.
`set_score_request`(*[, sample_weight])	Request metadata passed to the `score` method.
`validate_fallback_label`(fallback_label, classes)	Checks if fallback label is compatible w/ classes from training data..
`validate_predict`(X)	Validates inference methods.

fit(X, y=None, sample_weight=None)

Fits the estimator and sets fit attributes.

Parameters:

X ({array-like, sparse matrix}, shape (n_samples, n_features)) – The training input samples.
y (array-like, shape (n_samples,) or (n_samples, n_outputs)) – The target values.

Returns:

self – Returns self.

Return type:

object

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:: routing – A MetadataRequest encapsulating routing information.
Return type:: MetadataRequest

get_params(deep=True): Gets parameters for the estimator.

partial_fit(X, y, classes=None, sample_weight=None): Fits the estimator partially.

predict(X)

Predicts hard labels.

Parameters:: X (indexable, length n_samples) – Input samples to classify.
Returns:: y_pred – Predicted hard labels.
Return type:: ndarray of shape (n_samples,)

score(X, y)[source]

Evaluates an accuracy score on accepted samples.

Parameters:

X (indexable, length n_samples) – Input samples to evaluate.
y (array-like of shape (n_samples,)) – True labels for X (excluding fallback label).

Returns:

score – Accuracy on samples not labeled as fallbacks.

Return type:

float

See also

skfb.metrics.prediction_quality

set_fit_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → FallbackRuleClassifier

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to fit.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in fit.
Returns:: self – The updated object.
Return type:: object

set_params(**params): Sets the parameters of the estimator.

set_partial_fit_request(*, classes: bool | None | str = '$UNCHANGED$', sample_weight: bool | None | str = '$UNCHANGED$') → FallbackRuleClassifier

Request metadata passed to the partial_fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to partial_fit if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to partial_fit.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

classes (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for classes parameter in partial_fit.
sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in partial_fit.

Returns:

self – The updated object.

Return type:

object

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → FallbackRuleClassifier

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

static validate_fallback_label(fallback_label, classes): Checks if fallback label is compatible w/ classes from training data..

validate_predict(X)

Validates inference methods.

Raises:: An exception raised by one of the methods. –

skfb.estimators.predict_or_fallback(estimator, X, classes, threshold=0.5, ambiguity_threshold=0.0, fallback_label=-1, fallback_mode='return')[source]

For every sample, either predicts a class or rejects a prediction.

Parameters:

estimator (object) – Fitted base estimator supporting probabilistic predictions.
X ({array-like, sparse matrix}, shape (n_samples, n_features)) – Input samples.
classes (ndarray, shape = (n_classes,)) – NDArray of class labels.
threshold (float, default=0.5) – Predictions w/ the lower thresholds are rejected.
ambiguity_threshold (float, default=0.0) – Predictions w/ the close top 2 scores are rejected.
fallback_label (any, default=-1) – Rejected samples are labeled w/ this label.
fallback_mode ({"store", "return", "ignore"}, default="return") –
Whether to have:
- ("return") a numpy ndarray of both predictions and fallbacks;
- ("store") an FBNDArray of predictions storing also fallback mask;
- ("ignore") a numpy ndarray of only estimator’s predictions.

Returns:

Predictions w/o fallbacks, combined predictions, or predictions w/ fallback mask; depends on fallback_mode.

Return type:

FBNDArray or ndarray, shape (n_samples,)

skfb.estimators.multi_threshold_predict_or_fallback(estimator, X, thresholds, classes=None, ambiguity_threshold=0.0, fallback_label=-1, fallback_mode='return', return_probas=False)[source]

For every sample, either predicts a class or rejects a prediction.

Parameters:

estimator (object) – Fitted base estimator supporting probabilistic predictions.
X ({array-like, sparse matrix}, shape (n_samples, n_features)) – Input samples.
thresholds (array-like, shape (n_classes,)) – Certainty thresholds for each class.
classes (ndarray, shape = (n_classes,), default=None) – NDArray of class labels. Defaults to estimator classes.
ambiguity_threshold (float, default=0.0) – Predictions w/ the close top 2 scores are rejected.
fallback_label (any, default=-1) – Rejected samples are labeled w/ this label.
fallback_mode ({"store", "return", "ignore"}, default="return") –
Whether to have:
- ("return") a numpy ndarray of both predictions and fallbacks;
- ("store") an FBNDArray of predictions storing also fallback mask;
- ("ignore") a numpy ndarray of only estimator’s predictions.
return_probas (bool, default=False) – Whether to return also probabilities.

Returns:

Predictions w/o fallbacks, combined predictions, or predictions w/ fallback mask; depends on fallback_mode and return_probas.

Return type:

FBNDArray or ndarray, or tuple of FBNDArray of shape = n_samples and ndarray of shape (n_samples, n_classes)

Examples

>>> import numpy as np
>>> from sklearn.linear_model import LogisticRegression
>>> from skfb.estimators import multi_threshold_predict_or_fallback
>>> X = np.array(
...     [
...         [-3, -3], [-3, -2], [-2, -3],
...         [2, 3], [2, 4], [3, 4], [3.1, 3], [3, 2.9],
...         [4, 3], [3, 2], [4, 2], [2.9, 3], [3, 3.1],
...     ])
>>> y = np.array(["a"] * 3 + ["b"] * 5 + ["c"] * 5)
>>> estimator = LogisticRegression(C=10_000, random_state=0).fit(X, y)
>>> thresholds = {"a": 0.99, "b": 0.8, "c": 0.75}
>>> multi_threshold_predict_or_fallback(
...     estimator, X, estimator.classes_, thresholds, fallback_label="d")
array(['a', 'a', 'a', 'b', 'b', 'b', 'd', 'd', 'c', 'c', 'c', 'd', 'd'],
      dtype='<U1')

Anomaly-Based Fallback Classifiers

These fallback estimators reject based on outlier/novelty classification.

class skfb.estimators.AnomalyFallbackClassifier(estimator, outlier_detector, remove_outliers=False, fallback_label=-1, fallback_mode='store')[source]

A fallback classifier based on provided anomaly detector.

Augments estimator behavior with a reject option based on outlier detection. If outlier_detector predicts -1 for a given input, then returns, masks, or ignores rejections depending on fallback_mode. Otherwise, accepts the input and makes predictions.

Parameters:

estimator (object) – The base estimator making decisions w/o fallbacks.
outlier_detector (object) – The outlier detector returning 1 for inliers and -1 for outliers.
remove_outliers (bool, default=False) – Whether to remove outliers from training data before fitting.
fallback_label (any, default=-1) – The label of a rejected example. Should be compatible w/ the class labels from training data.
fallback_mode ({"return", "store", "ignore"}, default="store") –
While predicting w/ the predict method, whether to return:
- ("return") a numpy ndarray of both predictions and fallbacks;
- ("store") an FBNDArray of predictions storing also fallback mask;
- ("ignore") a numpy ndarray of only estimator’s predictions.
Calling decision_function or predict_proba is equivalent to estimator’s corresponding calls except that with "store", FBNDArray is returned.

estimator_

Trained base estimator.

Type:: object

outlier_detector_

Trained outlier detector.

Type:: object

Examples

>>> import numpy as np
>>> from skfb.estimators import AnomalyFallbackClassifier
>>> from sklearn.ensemble import IsolationForest
>>> from sklearn.linear_model import LogisticRegression
>>> estimator = LogisticRegression(random_state=0)
>>> outlier_detector = IsolationForest(n_estimators=10, max_samples=1.0,
...                                    contamination=0.2, random_state=0)
>>> rejector = AnomalyFallbackClassifier(estimator, outlier_detector)
>>> X = np.array([
...     [0, 0], [10, 10], [1, 1], [9, 9], [1, 0], [9, 10], [0, 1], [10, 9],
...     [5.5, 5], [5., 5.5]
... ])
>>> y = np.array([0, 1, 0, 1, 0, 1, 0, 1, 0, 1])
>>> rejector.fit(X, y).predict(X).get_dense_fallback_mask()
array([False, False, False, False, False, False, False, False,  True,
        True])
>>> rejector.set_params(fallback_mode="return").predict(X)
array([ 0,  1,  0,  1,  0,  1,  0,  1, -1, -1])
>>> rejector.score(X, y)
1.0

Methods

`decision_function`(X)	Calls `decision_function` on the estimator and sets fallback mask.
`fit`(X, y, **fit_params)	Trains base estimator and outlier detector then sets fit attributes.
`get_metadata_routing`()	Get metadata routing of this object.
`get_params`([deep])	Get parameters for this estimator.
`predict`(X)	Makes predictions using the base estimator and the fallback rules.
`predict_log_proba`(X)	Returns log of `predict_proba` on the estimator.
`predict_proba`(X)	Calls `predict_proba` on the estimator.
`score`(X, y)	Evaluates an accuracy score.
`set_params`(**params)	Set the parameters of this estimator.
`validate_fallback_label`(fallback_label, classes)	Checks if fallback label is compatible w/ classes from training data..

decision_function(X)[source]

Calls decision_function on the estimator and sets fallback mask.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_pred – Predicted class scores for X based on the estimator. If self.fallback_mode == "store", scores store fallback mask.
Return type:: FBNDArray of shape (n_samples,) or (n_samples, n_classes)

fit(X, y, **fit_params)[source]

Trains base estimator and outlier detector then sets fit attributes.

Parameters:

X ({array-like, sparse matrix}, shape (n_samples, n_features)) – The training input samples.
y (array-like, shape (n_samples,) or (n_samples, n_outputs)) – The target values.

Returns:

self – Returns self.

Return type:

object

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:: routing – A MetadataRequest encapsulating routing information.
Return type:: MetadataRequest

get_params(deep=True)

Get parameters for this estimator.

Parameters:: deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns:: params – Parameter names mapped to their values.
Return type:: dict

predict(X)

Makes predictions using the base estimator and the fallback rules.

Parameters:

X (indexable, length n_samples)) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.

Returns:

y_pred – Depending on self.fallback_mode:

("return") a numpy ndarray of both predictions and fallbacks, or;
("store") an FBNDArray of predictions storing also fallback mask, or;
("ignore") a numpy ndarray of only estimator’s predictions.

Return type:

ndarray or FBNDArray of shape (n_samples,)

predict_log_proba(X)

Returns log of predict_proba on the estimator.

If fallback_mode != “ignore”, returns FBNDArray w/ fallback mask.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_pred – Predicted class log-probabilities for X based on the estimator. The order of the classes corresponds to that in the fitted attribute classes_. If self.fallback_mode == "ignore", returns an ndarray.
Return type:: FBNDArray or ndarray of shape (n_samples, n_classes)

predict_proba(X)[source]

Calls predict_proba on the estimator.

Parameters:: X (indexable, length n_samples) – Input samples to classify. Must fulfill the input assumptions of the underlying estimator.
Returns:: y_pred – Predicted class probabilities for X based on the estimator. The order of the classes corresponds to that in the fitted attribute classes_. If self.fallback_mode == "store", probabilities store fallback mask.
Return type:: FBNDArray of shape (n_samples,) or (n_samples, n_classes)

score(X, y)

Evaluates an accuracy score.

Parameters:

X (indexable, length n_samples) – Input samples to evaluate. Must fulfill the input assumptions of the underlying estimator.
y (array-like of shape (n_samples,)) – True labels for X (excluding fallback label).

Returns:

score – Depending on self.fallback_mode, calculate:

("return") accuracy on accepted inputs;
("store") predict_reject_accuracy_score();
("ignore") accuracy on all inputs.

Return type:

float

See also

skfb.metrics.predict_reject_accuracy_score: Combined, prediction-rejection accuracy score.
skfb.metrics.predict_reject_recall_score: Combined, prediction-rejection balanced accuracy score.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:: **params (dict) – Estimator parameters.
Returns:: self – Estimator instance.
Return type:: estimator instance

static validate_fallback_label(fallback_label, classes): Checks if fallback label is compatible w/ classes from training data..