safeai.rgr.rgr_curve

safeai.rgr.rgr_curve(model, x_data, strengths, *, method: Literal['noise', 'adversarial'] = 'noise', prob_original=None, model_class_order=None, class_order=None, y_true=None, attack_name: Literal['fgsm', 'pgd', 'square', 'hsj', 'simba'] = 'fgsm', base_attack_params=None, class_weights=None, model_type='sklearn', device=None, rga_full=None, model_name='Model', random_seed=None, plot=False, fig_size=(10, 6), save_path=None, show=False, verbose=True)[source]

Compute one Rank Graduation Robustness (RGR) curve and its AURGR value.

This function evaluates the robustness of one trained model under increasing feature-space perturbation strength. It supports two single-model workflows:

  • Gaussian noise perturbation, using method=’noise’

  • Adversarial perturbation, using method=’adversarial’

Image-level robustness workflows, such as Wasserstein and spatial transformations, are handled by compare_rgr(…), because they require a shared attack model and preprocessing pipeline across several models.

Parameters:

  • model (object) –

    Trained model to evaluate.

    For model_type=’sklearn’, the model must provide predict_proba(…).

    For model_type=’pytorch’, the model must return logits when called on an input tensor.

  • x_data (array-like or torch.Tensor) – Input feature matrix used for robustness evaluation. For sklearn models, this is usually a NumPy array or pandas-compatible matrix. For PyTorch models, this may be either a tensor or an array that can be converted to a float tensor.

  • strengths (array-like) –

    Perturbation strengths used to build the RGR curve.

    For method=’noise’, values are interpreted as Gaussian noise standard deviations.

    For method=’adversarial’, values are interpreted according to the selected ART attack. For FGSM, PGD, and Square Attack, these are eps values. For SimBA, these are epsilon values. For HopSkipJump, these are interpreted as max_iter values.

  • method ({'noise', 'adversarial'}, default='noise') – Feature-space robustness workflow used to construct the curve.

  • prob_original (array-like, optional) –

    Original predicted probabilities for x_data.

    If None, probabilities are computed from model and x_data. If provided, the probabilities should follow model_class_order and will be aligned to class_order.

  • model_class_order (array-like, optional) –

    Class order produced by the model probability output.

    For sklearn models, this is usually model.classes_. If omitted, the function tries to infer it from the model or from prob_original.

  • class_order (array-like, optional) –

    Target class order used to align probability columns across models.

    If omitted, model_class_order is used.

  • y_true (array-like, optional) – True class labels. Required when method=’adversarial’, because ART attacks need labels to generate adversarial examples.

  • attack_name ({'fgsm', 'pgd', 'square', 'hsj', 'simba'}, default='fgsm') – ART adversarial attack used when method=’adversarial’.

  • base_attack_params (dict, optional) – Additional fixed parameters passed to the ART attack constructor. The current strength value is inserted automatically using the appropriate parameter name for the selected attack.

  • class_weights (array-like, optional) – Weights used to aggregate per-class RGR values. If None, uniform class weights are used.

  • model_type ({'sklearn', 'pytorch'}, default='sklearn') – Type of model being evaluated.

  • device (torch.device or str, optional) – Device used for PyTorch inference. Ignored for sklearn models.

  • rga_full (float, optional) – Full RGA score used to rescale the RGR curve. If provided and finite, rgr_rescaled = rgr_scores * rga_full. If None, no rescaling is applied.

  • model_name (str, default='Model') – Name stored in the result dictionary and used in plot labels.

  • random_seed (int, optional) – Random seed used for Gaussian noise generation when method=’noise’.

  • plot (bool, default=False) – Whether to create a plot for the computed RGR curve.

  • fig_size (tuple, default=(10, 6)) – Figure size used when plotting.

  • save_path (str, optional) – Path where the plot should be saved. If provided, the plot is written to disk.

  • show (bool, default=False) – Whether to display the plot with plt.show().

  • verbose (bool, default=True) – Whether to print progress and summary information.

Returns:

Dictionary with the computed RGR curve and metadata.

Common keys include:

  • ’rgr_scores’np.ndarray

    Raw RGR scores at each perturbation strength.

  • ’rgr_rescaled’np.ndarray

    RGR scores after optional rescaling by rga_full.

  • ’aurgr’float

    Area under the RGR curve.

  • ’per_class_rgr’np.ndarray

    Per-class RGR values at each perturbation strength.

  • ’class_order’np.ndarray

    Class order used for probability alignment.

  • ’method’str

    Robustness workflow used to build the curve.

  • ’model_name’str

    Name of the evaluated model.

For method=’noise’, the result also contains:

  • ’noise_levels’ : np.ndarray

For method=’adversarial’, the result also contains:

  • ’attack_name’ : str

  • ’attack_strengths’ : np.ndarray

Return type:

dict