API Reference
RGA
Rank Graduation Accuracy (RGA).
Main functions
- rga_score
Compute one RGA value.
- rga_curve
Compute an RGA curve and normalized AURGA.
- aurga_score
Compute only the normalized area under the RGA curve.
- compare_rga
Compare several models/probability arrays using RGA curves.
- plot_rga
Plot one RGA curve or a comparison of several RGA curves.
Notes
Notes
The scalar RGA score is computed consistently across binary and multiclass classification.
AURGA depends on the curve-construction method.
For binary and multiclass classification, the default curve method is ‘removal’. This progressively removes the most confident samples inside each true class and recomputes RGA on the remaining data.
The binary ‘partial’ curve is still available with curve_method=’partial’. It is useful as a decomposition-style curve, but the removal curve is preferred for normalized AURGA.
- safeai.rga.aurga_score(y_true, y_score, *, x=None, class_order=None, positive_class=1, n_segments=10, curve_method='auto', normalize_to_perfect=True, verbose=False)[source]
Compute only the area under the RGA curve.
By default, this returns normalized AURGA.
- safeai.rga.compare_rga(models, y_true, *, x=None, n_segments=10, positive_class=1, curve_method='auto', normalize_to_perfect=True, save_path=None, show=False, verbose=True)[source]
Compare multiple models or probability arrays using RGA curves.
- Parameters:
models (dict) – Dictionary where values can be: - score array - probability matrix - fitted model with predict_proba - tuple of (probability_matrix, class_order)
y_true (array-like) – True labels.
x (array-like or DataFrame, optional) – Required if model objects are passed.
n_segments (int, default=10) – Number of curve segments.
positive_class (int or str, default=1) – Positive class for binary classification.
curve_method ({'auto', 'partial', 'removal'}, default='auto') – Method used to construct RGA curves.
normalize_to_perfect (bool, default=True) – If True, return normalized AURGA as the main ‘aurga’ value.
save_path (str or None, default=None) – If provided, save the comparison plot to this path.
show (bool, default=False) – If True, display the comparison plot with plt.show().
verbose (bool, default=True) – Whether to print summary.
- Returns:
Mapping from model name to RGA curve result.
- Return type:
dict
- safeai.rga.plot_rga(result, *, model_name='Model', fig_size=(12, 5), save_path=None, show=False)[source]
Plot one RGA curve or several RGA curves.
- Parameters:
result (dict) – Either: - result from rga_curve(…) - results from compare_rga(…)
model_name (str, default='Model') – Used only when plotting one curve.
fig_size (tuple, default=(12, 5)) – Figure size.
save_path (str or None, default=None) – If provided, save the plot.
show (bool, default=False) – If True, display the plot with plt.show().
- Returns:
If save_path is provided and show is False, returns save_path. Otherwise, returns (fig, ax).
- Return type:
str or tuple
- safeai.rga.rga_curve(y_true, y_score, *, x=None, class_order=None, positive_class=1, n_segments=10, curve_method='auto', normalize_to_perfect=True, verbose=False)[source]
Compute an RGA curve and normalized AURGA.
- Parameters:
y_true (array-like, shape (n_samples,)) – True labels.
y_score (array-like or fitted model) – One of: - binary score/probability vector, shape (n_samples,) - probability matrix, shape (n_samples, n_classes) - fitted sklearn estimator or Pipeline with predict_proba
x (array-like or DataFrame, optional) – Input features. Required if y_score is a fitted model.
class_order (array-like, optional) – Order of probability columns.
positive_class (int or str, default=1) – Positive class for binary classification.
n_segments (int, default=10) – Number of curve segments.
curve_method ({'auto', 'partial', 'removal'}, default='auto') –
Method used to construct the RGA curve.
- ’auto’:
binary -> ‘partial’ multiclass -> ‘removal’
- ’partial’:
Partial contribution decomposition. Currently supported only for binary classification.
- ’removal’:
Progressively remove high-confidence samples and recompute RGA.
normalize_to_perfect (bool, default=True) – If True, return normalized AURGA as the main ‘aurga’.
verbose (bool, default=False) – Whether to print additional information.
- Returns:
Dictionary containing: - task - curve_method - rga - x - curve - aurga - aurga_raw - aurga_perfect, when available - perfect_curve, when available - per_class_rga, only for multiclass - class_weights, only for multiclass - classes, only for multiclass
- Return type:
dict
- safeai.rga.rga_score(y_true, y_score, *, x=None, class_order=None, positive_class=1, verbose=False)[source]
Compute Rank Graduation Accuracy.
- Parameters:
y_true (array-like, shape (n_samples,)) – True labels.
y_score (array-like or fitted model) – One of: - binary score/probability vector, shape (n_samples,) - probability matrix, shape (n_samples, n_classes) - fitted sklearn estimator or Pipeline with predict_proba
x (array-like or DataFrame, optional) – Input features. Required if y_score is a fitted model.
class_order (array-like, optional) – Order of probability columns. For sklearn models this is usually
model.classes_.positive_class (int or str, default=1) – Positive class for binary classification.
verbose (bool, default=False) – Whether to print additional information.
- Returns:
RGA score.
- Return type:
float
RGR
Rank Graduation Robustness (RGR).
Main functions
- rgr_score
Compute one RGR value from original and perturbed predictions.
- rgr_curve
Compute one RGR robustness curve for one model.
- aurgr_score
Compute only the area under an RGR curve.
- compare_rgr
Compare several models using one of the supported RGR workflows.
- plot_rgr
Plot one RGR curve or a comparison of several RGR curves.
Notes
RGR measures stability of model predictions under perturbations. Values closer to 1 indicate stronger robustness, while values closer to 0 indicate stronger prediction degradation.
The recommended user-facing function is compare_rgr(…, method=…). It can run Gaussian-noise robustness, adversarial robustness, Wasserstein image-level robustness, or spatial image-level robustness from the same entry point.
- safeai.rgr.aurgr_score(model, x_data, strengths, **kwargs)[source]
Compute only the area under an RGR curve.
- safeai.rgr.compare_rgr(models, strengths, class_order, *, method: Literal['noise', 'adversarial', 'wasserstein_images', 'spatial_images'] = 'noise', y_true=None, y_true_dict=None, images=None, attack_model=None, preprocess_fn=None, attack_name: Literal['fgsm', 'pgd', 'square', 'hsj', 'simba'] = 'fgsm', base_attack_params=None, rga_dict=None, class_weights=None, fig_size=(12, 6), verbose=True, random_seed=None, save_path=None, show=False, max_iter=50, eps_step=0.01, num_translations=3, num_rotations=3)[source]
Compare several models using Rank Graduation Robustness (RGR) curves.
This is the main user-facing comparison function for RGR. It provides one unified interface for all supported robustness workflows:
Gaussian feature noise, using method=’noise’
Feature-space adversarial attacks, using method=’adversarial’
Image-level Wasserstein attacks, using method=’wasserstein_images’
Image-level spatial transformations, using method=’spatial_images’
The function returns one result dictionary per model and can optionally plot a comparison of the resulting RGR curves.
- Parameters:
models (dict) –
Dictionary containing model configurations.
For method=’noise’ or method=’adversarial’, each entry must have the form:
model_name -> ( model, x_data, prob_original, model_class_order, model_type, device )
where:
model is a trained sklearn estimator or PyTorch module.
x_data is the feature matrix used for perturbation.
prob_original is the original probability matrix, or None.
model_class_order is the class order of the model probability output.
model_type is either ‘sklearn’ or ‘pytorch’.
device is the torch device for PyTorch models, or None for sklearn.
For method=’wasserstein_images’ or method=’spatial_images’, each entry must have the form:
model_name -> ( model, prob_original, model_class_order, model_type, device )
In image-level workflows, perturbed images are first generated using attack_model, then converted to model-ready features through preprocess_fn.
strengths (array-like) –
Perturbation strengths used to build the RGR curves.
Their meaning depends on method:
- method=’noise’:
Gaussian noise standard deviations.
- method=’adversarial’:
Attack strengths for the selected ART attack.
- method=’wasserstein_images’:
Wasserstein attack eps values.
- method=’spatial_images’:
Spatial transformation strengths.
class_order (array-like) – Shared target class order used to align probability columns across all models.
method ({'noise', 'adversarial', 'wasserstein_images', 'spatial_images'}, default='noise') – Robustness workflow used for all models in the comparison.
y_true (array-like, optional) –
Shared true labels.
Required for method=’wasserstein_images’ and method=’spatial_images’.
For method=’adversarial’, either y_true or y_true_dict must be provided.
y_true_dict (dict, optional) – Per-model true labels for adversarial evaluation. This is useful when different models are evaluated on different input arrays.
images (array-like or torch.Tensor, optional) – Original image tensor or image array used for image-level robustness evaluation. Required for method=’wasserstein_images’ and method=’spatial_images’.
attack_model (torch.nn.Module, optional) – PyTorch model used by ART to generate image-level adversarial examples or spatial transformations. Required for image-level methods.
preprocess_fn (callable, optional) – Function that maps perturbed images to model-ready feature matrices. Required for image-level methods.
attack_name ({'fgsm', 'pgd', 'square', 'hsj', 'simba'}, default='fgsm') – ART adversarial attack used when method=’adversarial’.
base_attack_params (dict, optional) – Additional fixed parameters passed to the ART attack constructor. The current strength value is inserted automatically using the appropriate parameter name for the selected attack.
rga_dict (dict, optional) – Mapping from model name to full RGA score. If provided, each RGR curve is rescaled by the corresponding RGA value.
class_weights (array-like, optional) – Weights used to aggregate per-class RGR values. If None, uniform class weights are used.
fig_size (tuple, default=(12, 6)) – Figure size used for the comparison plot.
verbose (bool, default=True) – Whether to print progress and summary information.
random_seed (int, optional) – Random seed used for Gaussian noise generation when method=’noise’.
save_path (str, optional) – Path where the comparison plot should be saved. If None and show=False, no plot is saved.
show (bool, default=False) – Whether to display the comparison plot with plt.show().
max_iter (int, default=50) – Maximum number of iterations used by the Wasserstein image attack.
eps_step (float, default=0.01) – Step size used by the Wasserstein image attack.
num_translations (int, default=3) – Number of translations tested by the spatial transformation attack.
num_rotations (int, default=3) – Number of rotations tested by the spatial transformation attack.
- Returns:
Mapping from model name to RGR result dictionary.
Each result contains:
- ’rgr_scores’np.ndarray or list
Raw RGR scores at each perturbation strength.
- ’rgr_rescaled’np.ndarray or list
RGR scores after optional rescaling by rga_dict.
- ’aurgr’float
Area under the RGR curve.
- ’per_class_rgr’np.ndarray or list
Per-class RGR values at each perturbation strength.
- ’class_order’np.ndarray or list
Class order used for probability alignment.
- ’method’str
Robustness workflow used in the comparison.
Depending on method, each result also includes one of:
’noise_levels’
’attack_strengths’
For adversarial methods, results also include:
’attack_name’
- Return type:
dict
- safeai.rgr.plot_rgr(result, *, model_name='Model', x_key=None, x_label=None, y_key='rgr_rescaled', title=None, fig_size=(12, 6), save_path=None, show=False)[source]
Plot one RGR curve or a comparison of several RGR curves.
- Parameters:
result (dict) – Either one result returned by rgr_curve(…) or the full dictionary returned by compare_rgr(…).
model_name (str, default='Model') – Label used when result is a single curve.
x_key (str, optional) – Name of the x-axis field in each result. If None, it is inferred from the result keys, for example ‘noise_levels’ or ‘attack_strengths’.
x_label (str, optional) – Human-readable x-axis label. If None, a default label is selected.
y_key (str, default='rgr_rescaled') – Name of the y-axis field to plot.
title (str, optional) – Plot title. If None, a default title is selected.
fig_size (tuple, default=(12, 6)) – Matplotlib figure size.
save_path (str or None, default=None) – If provided, save the plot to this path.
show (bool, default=False) – If True, display the plot with plt.show().
- Returns:
If save_path is provided and show is False, returns save_path. Otherwise, returns (fig, ax).
- Return type:
str or tuple
- safeai.rgr.rgr_curve(model, x_data, strengths, *, method: Literal['noise', 'adversarial'] = 'noise', prob_original=None, model_class_order=None, class_order=None, y_true=None, attack_name: Literal['fgsm', 'pgd', 'square', 'hsj', 'simba'] = 'fgsm', base_attack_params=None, class_weights=None, model_type='sklearn', device=None, rga_full=None, model_name='Model', random_seed=None, plot=False, fig_size=(10, 6), save_path=None, show=False, verbose=True)[source]
Compute one Rank Graduation Robustness (RGR) curve and its AURGR value.
This function evaluates the robustness of one trained model under increasing feature-space perturbation strength. It supports two single-model workflows:
Gaussian noise perturbation, using method=’noise’
Adversarial perturbation, using method=’adversarial’
Image-level robustness workflows, such as Wasserstein and spatial transformations, are handled by compare_rgr(…), because they require a shared attack model and preprocessing pipeline across several models.
- Parameters:
model (object) –
Trained model to evaluate.
For model_type=’sklearn’, the model must provide predict_proba(…).
For model_type=’pytorch’, the model must return logits when called on an input tensor.
x_data (array-like or torch.Tensor) – Input feature matrix used for robustness evaluation. For sklearn models, this is usually a NumPy array or pandas-compatible matrix. For PyTorch models, this may be either a tensor or an array that can be converted to a float tensor.
strengths (array-like) –
Perturbation strengths used to build the RGR curve.
For method=’noise’, values are interpreted as Gaussian noise standard deviations.
For method=’adversarial’, values are interpreted according to the selected ART attack. For FGSM, PGD, and Square Attack, these are eps values. For SimBA, these are epsilon values. For HopSkipJump, these are interpreted as max_iter values.
method ({'noise', 'adversarial'}, default='noise') – Feature-space robustness workflow used to construct the curve.
prob_original (array-like, optional) –
Original predicted probabilities for x_data.
If None, probabilities are computed from model and x_data. If provided, the probabilities should follow model_class_order and will be aligned to class_order.
model_class_order (array-like, optional) –
Class order produced by the model probability output.
For sklearn models, this is usually
model.classes_. If omitted, the function tries to infer it from the model or from prob_original.class_order (array-like, optional) –
Target class order used to align probability columns across models.
If omitted, model_class_order is used.
y_true (array-like, optional) – True class labels. Required when method=’adversarial’, because ART attacks need labels to generate adversarial examples.
attack_name ({'fgsm', 'pgd', 'square', 'hsj', 'simba'}, default='fgsm') – ART adversarial attack used when method=’adversarial’.
base_attack_params (dict, optional) – Additional fixed parameters passed to the ART attack constructor. The current strength value is inserted automatically using the appropriate parameter name for the selected attack.
class_weights (array-like, optional) – Weights used to aggregate per-class RGR values. If None, uniform class weights are used.
model_type ({'sklearn', 'pytorch'}, default='sklearn') – Type of model being evaluated.
device (torch.device or str, optional) – Device used for PyTorch inference. Ignored for sklearn models.
rga_full (float, optional) – Full RGA score used to rescale the RGR curve. If provided and finite, rgr_rescaled = rgr_scores * rga_full. If None, no rescaling is applied.
model_name (str, default='Model') – Name stored in the result dictionary and used in plot labels.
random_seed (int, optional) – Random seed used for Gaussian noise generation when method=’noise’.
plot (bool, default=False) – Whether to create a plot for the computed RGR curve.
fig_size (tuple, default=(10, 6)) – Figure size used when plotting.
save_path (str, optional) – Path where the plot should be saved. If provided, the plot is written to disk.
show (bool, default=False) – Whether to display the plot with plt.show().
verbose (bool, default=True) – Whether to print progress and summary information.
- Returns:
Dictionary with the computed RGR curve and metadata.
Common keys include:
- ’rgr_scores’np.ndarray
Raw RGR scores at each perturbation strength.
- ’rgr_rescaled’np.ndarray
RGR scores after optional rescaling by rga_full.
- ’aurgr’float
Area under the RGR curve.
- ’per_class_rgr’np.ndarray
Per-class RGR values at each perturbation strength.
- ’class_order’np.ndarray
Class order used for probability alignment.
- ’method’str
Robustness workflow used to build the curve.
- ’model_name’str
Name of the evaluated model.
For method=’noise’, the result also contains:
’noise_levels’ : np.ndarray
For method=’adversarial’, the result also contains:
’attack_name’ : str
’attack_strengths’ : np.ndarray
- Return type:
dict
- safeai.rgr.rgr_score(pred_original, pred_perturbed, *, class_order=None, class_weights=None, verbose=False)[source]
Compute Rank Graduation Robustness between two prediction arrays.
- Parameters:
pred_original (array-like) – Original predictions. Can be a 1D score vector or a 2D probability matrix.
pred_perturbed (array-like) – Predictions after perturbation. Must have the same shape as pred_original.
class_order (array-like, optional) – Class order. If provided with 1D binary probabilities, the vectors are converted to two-column probability matrices.
class_weights (array-like, optional) – Weights for multiclass aggregation. If None, uses uniform weights.
verbose (bool, default=False) – Whether to print per-class values for multiclass inputs.
- Returns:
RGR score.
- Return type:
float
RGE
Rank Graduation Explainability (RGE).
Main functions
- rge_score
Compute one RGE value from full and reduced/occluded predictions.
- rge_curve
Compute one RGE curve and AURGE value for one model.
- aurge_score
Compute only the area under an RGE curve.
- compare_rge
Compare several models using one of the supported RGE workflows.
- plot_rge
Plot one RGE curve or a comparison of several RGE curves.
Notes
RGE measures how model predictions change when information is removed or occluded. In this implementation, higher values mean stronger preservation of the original predictions under feature/image removal, and lower values mean stronger degradation.
- safeai.rge.aurge_score(model, data, removal_fractions=None, **kwargs)[source]
Compute only the area under an RGE curve.
- safeai.rge.compare_rge(models, class_order, *, method: Literal['image', 'text', 'tabular'] = 'tabular', removal_fractions=None, images_dataset=None, occlusion_method='random', patch_size=32, batch_size=64, class_weights=None, rga_dict=None, device=None, fig_size=(12, 6), verbose=True, random_seed=None, patch_rankings=None, patch_meta=None, save_path=None, show=False, mask_value=0.0, use_shared_feature_cache=True, masking_method='greedy', baseline='zero', n_steps=None, feature_rankings=None)[source]
Compare several models using Rank Graduation Explainability (RGE) curves.
This is the main user-facing comparison function for RGE. It provides one unified interface for all supported RGE workflows:
Image occlusion, using method=’image’
Text or generic feature removal, using method=’text’
Tabular feature removal, using method=’tabular’
The function returns one result dictionary per model and can optionally plot a comparison of the resulting RGE curves.
- Parameters:
models (dict) –
Dictionary containing model configurations.
For method=’image’, each entry must have the form:
model_name -> ( model, preprocess_fn, model_class_order, model_type )
where:
model is a trained sklearn estimator or PyTorch module.
preprocess_fn maps image tensors to model-ready feature matrices.
model_class_order is the class order of the model probability output.
model_type is either ‘sklearn’ or ‘pytorch’.
For method=’text’, each entry must have the form:
model_name -> ( model, x, prob_full, model_class_order, model_type, device )
where:
x is the feature matrix to mask.
prob_full is the original probability matrix, or None.
device is the torch device for PyTorch models, or None for sklearn.
For method=’tabular’, each entry must have the form:
model_name -> ( model, x, feature_names, prob_full, model_class_order, model_type, device )
where feature_names contains the names of the columns in x.
class_order (array-like) – Shared target class order used to align probability columns across all models.
method ({'image', 'text', 'tabular'}, default='tabular') – RGE workflow used for all models in the comparison.
removal_fractions (array-like, optional) –
Fractions of removed information.
Required for method=’image’ and method=’text’.
For method=’tabular’, this argument is not used. The tabular curve is controlled by n_steps.
images_dataset (torch.utils.data.Dataset, optional) – Image dataset required when method=’image’. The dataset should return image tensors, or tuples/lists where the first element is the image tensor.
occlusion_method ({'random', 'gradcam_most'} or dict, default='random') –
Image occlusion method used when method=’image’.
If a string is provided, the same occlusion method is used for all models.
If a dictionary is provided, it should map model names to occlusion methods.
patch_size (int, default=32) – Patch size used for image occlusion.
batch_size (int, default=64) – Batch size used when loading images and when running PyTorch prediction.
class_weights (array-like, optional) – Weights used to aggregate per-class RGE values. If None, uniform class weights are used.
rga_dict (dict, optional) – Mapping from model name to full RGA score. If provided, each RGE curve is rescaled by the corresponding RGA value.
device (torch.device or str, optional) – Device used for PyTorch inference.
fig_size (tuple, default=(12, 6)) – Figure size used for the comparison plot.
verbose (bool, default=True) – Whether to print progress and summary information.
random_seed (int, optional) – Random seed used for random image occlusion or random feature masking.
patch_rankings (list or array-like, optional) – Patch rankings used when occlusion_method=’gradcam_most’.
patch_meta (dict, optional) – Patch metadata used when occlusion_method=’gradcam_most’.
save_path (str, optional) – Path where the comparison plot should be saved. If None and show=False, no plot is saved.
show (bool, default=False) – Whether to display the comparison plot with plt.show().
mask_value (float, default=0.0) – Constant value used for image patch masking.
use_shared_feature_cache (bool, default=True) – Whether to cache image features shared across models when method=’image’. This can speed up comparison when all models use the same preprocessing function and occlusion method.
masking_method ({'random', 'most_important', 'greedy'}, default='greedy') –
Feature masking method used for method=’text’ and method=’tabular’.
For method=’text’, only ‘random’ and ‘most_important’ are supported.
For method=’tabular’, ‘random’, ‘most_important’, and ‘greedy’ are supported.
baseline ({'zero', 'mean'}, default='zero') – Baseline value used when masking text or tabular features.
n_steps (int, optional) – Number of feature-removal steps for method=’tabular’. If None, all features are removed one by one.
feature_rankings (array-like or dict, optional) –
Feature rankings used when masking_method=’most_important’.
If an array is provided, the same ranking is used for all models.
If a dictionary is provided, it should map model names to feature-ranking arrays.
- Returns:
Mapping from model name to RGE result dictionary.
Each result contains:
- ’rge_scores’np.ndarray
Raw RGE scores at each removal level.
- ’rge_rescaled’np.ndarray
RGE scores after optional rescaling by rga_dict.
- ’aurge’float
Area under the RGE curve.
- ’per_class_rge’np.ndarray or None
Per-class RGE values at each removal level.
- ’class_order’np.ndarray
Class order used for probability alignment.
- ’method’str
RGE workflow used in the comparison.
Depending on method, each result may also include:
’removal_fractions’
’x_axis’
’occlusion_method’
’masking_method’
’baseline’
’removed_features’
- Return type:
dict
- safeai.rge.plot_rge(result, *, model_name='Model', x_key=None, x_label=None, x_scale=1.0, y_key='rge_rescaled', title=None, fig_size=(12, 6), save_path=None, show=False)[source]
Plot one RGE curve or a comparison of several RGE curves.
- Parameters:
result (dict) – Either one result returned by rge_curve(…) or the full dictionary returned by compare_rge(…).
model_name (str, default='Model') – Label used when result is a single curve.
x_key (str, optional) – Name of the x-axis field in each result. If None, it is inferred from the result keys.
x_label (str, optional) – Human-readable x-axis label. If None, a default label is selected.
x_scale (float, default=1.0) – Multiplier applied to x values before plotting. Use 100 for percentage axes when the stored values are fractions.
y_key (str, default='rge_rescaled') – Name of the y-axis field to plot.
title (str, optional) – Plot title. If None, a default title is selected.
fig_size (tuple, default=(12, 6)) – Matplotlib figure size.
save_path (str or None, default=None) – If provided, save the plot to this path.
show (bool, default=False) – If True, display the plot with plt.show().
- Returns:
If save_path is provided and show is False, returns save_path. Otherwise, returns (fig, ax).
- Return type:
str or tuple
- safeai.rge.rge_curve(model, data, removal_fractions=None, *, method: Literal['image', 'text', 'tabular'] = 'tabular', preprocess_fn=None, feature_names=None, model_class_order=None, class_order=None, model_type='sklearn', device=None, batch_size=64, class_weights=None, model_name='Model', rga_full=None, occlusion_method: Literal['random', 'gradcam_most'] = 'random', masking_method: Literal['random', 'most_important', 'greedy'] = 'greedy', baseline: Literal['zero', 'mean'] = 'zero', feature_ranking=None, patch_size=32, patch_rankings=None, patch_meta=None, n_steps=None, random_seed=None, mask_value=0.0, prob_full=None, plot=False, fig_size=(10, 6), save_path=None, show=False, verbose=True)[source]
Compute one Rank Graduation Explainability (RGE) curve and AURGE value.
- Parameters:
model (object) – Trained sklearn estimator with predict_proba or PyTorch module that returns logits.
data (object) –
Input data used for the selected workflow.
For method=’image’, pass a PyTorch Dataset or compatible dataset returning image tensors.
For method=’text’ or method=’tabular’, pass a feature matrix.
removal_fractions (array-like, optional) – Fractions of removed information. For image and text workflows this is required. For tabular workflow, if None, the curve is built using n_steps or all available features.
method ({'image', 'text', 'tabular'}, default='tabular') – RGE workflow used to construct the curve.
preprocess_fn (callable, optional) – Required for method=’image’. Maps image tensors to model-ready feature matrices.
feature_names (array-like, optional) – Required for method=’tabular’. Names of the feature columns.
model_class_order (array-like) – Class order produced by the model probability output.
class_order (array-like) – Target class order used to align probability columns.
model_type ({'sklearn', 'pytorch'}, default='sklearn') – Type of model being evaluated.
device (torch.device or str, optional) – Device used for PyTorch inference.
batch_size (int, default=64) – Batch size used when loading images or running PyTorch prediction.
class_weights (array-like, optional) – Weights used to aggregate per-class RGE values. If None, uniform class weights are used.
model_name (str, default='Model') – Name stored in the result dictionary and used in plot labels.
rga_full (float, optional) – If provided and finite, the RGE curve is rescaled by this RGA value.
occlusion_method ({'random', 'gradcam_most'}, default='random') – Image occlusion workflow used when method=’image’.
masking_method ({'random', 'most_important', 'greedy'}, default='greedy') – Feature masking workflow used when method=’text’ or method=’tabular’. Text supports ‘random’ and ‘most_important’. Tabular supports all three.
baseline ({'zero', 'mean'}, default='zero') – Baseline value used when masking text/tabular features.
feature_ranking (array-like, optional) – Feature ranking required when masking_method=’most_important’.
patch_size (int, default=32) – Patch size used for image occlusion.
patch_rankings (optional) – Required when occlusion_method=’gradcam_most’.
patch_meta (optional) – Required when occlusion_method=’gradcam_most’.
n_steps (int, optional) – Number of feature-removal steps for method=’tabular’. If None, all features are removed one by one.
random_seed (int, optional) – Random seed used for random masking or occlusion.
mask_value (float, default=0.0) – Constant value used for image patch masking.
prob_full (array-like, optional) – Cached full/original probability matrix. If None, it is computed.
plot (bool, default=False) – Whether to create a plot for the computed RGE curve.
fig_size (tuple, default=(10, 6)) – Figure size used when plotting.
save_path (str, optional) – Path where the plot should be saved.
show (bool, default=False) – Whether to display the plot with plt.show().
verbose (bool, default=True) – Whether to print progress and summary information.
- Returns:
Dictionary containing the RGE curve, AURGE, removed fractions, optional rescaled curve, per-class RGE values, and method metadata.
- Return type:
dict
- safeai.rge.rge_score(pred_full, pred_reduced, *, class_order=None, class_weights=None, verbose=False)[source]
Compute Rank Graduation Explainability between two prediction arrays.
- Parameters:
pred_full (array-like) – Predictions from the full/original model input. Can be a 1D score vector or a 2D probability matrix.
pred_reduced (array-like) – Predictions after feature removal, masking, or occlusion. Must have the same shape as pred_full.
class_order (array-like, optional) – Class order. If provided with 1D binary probabilities, the vectors are converted to two-column probability matrices.
class_weights (array-like, optional) – Weights for multiclass aggregation. If None, uses uniform weights.
verbose (bool, default=False) – Whether to print per-class values for multiclass inputs.
- Returns:
RGE score. Higher values indicate stronger prediction preservation after removal/occlusion.
- Return type:
float
Cramer utilities
Lorenz/concordance Cramer-von Mises utilities.
The functions in this module implement the Lorenz-curve and concordance-curve building blocks used by SAFE-AI metrics. They are based on the identity between weighted concordance-curve divergence and the rank-based Cramer-von Mises formulation discussed in:
https://www.worldscientific.com/doi/abs/10.1142/S0218202526420030
- safeai.cramer.concordance_curve(y, yhat)[source]
Compute the concordance curve between true and predicted values.
- Parameters:
y (array-like) – True values
yhat (array-like) – Predicted values
- Returns:
Concordance curve
- Return type:
np.ndarray
- safeai.cramer.cvm1_concordance_weighted(y, yhat)[source]
Weighted Cramer von Mises distance between Lorenz and Concordance curves.
- Parameters:
y (array-like) – True values
yhat (array-like) – Predicted values
- Returns:
Weighted CvM distance
- Return type:
float
- safeai.cramer.gini_via_lorenz(y)[source]
Calculate Gini coefficient.
- Parameters:
y (array-like) – Input values
- Returns:
Gini coefficient
- Return type:
float
- safeai.cramer.lorenz_curve(y)[source]
Compute the Lorenz curve for a given array.
- Parameters:
y (array-like) – Input values
- Returns:
Normalized cumulative sum (Lorenz curve)
- Return type:
np.ndarray