# CLEVER score The CLEVER score is a metric proposed to measure robustness of neural network classifiers against adversarial perturbations, using extreme value theory to estimate worst-case loss. It was introduced in 2018 as a computationally efficient alternative to exact verification methods. The CLEVER score (Cross-Lipschitz Extreme Value for nEtwork Robustness) is a quantitative metric for evaluating the robustness of neural network classifiers to adversarial examples. Proposed in 2018 by Tsui-Wei Weng and colleagues, it estimates a lower bound on the minimum distortion required to change a network's classification for a given input. The score addresses a central challenge in deep learning: providing a practical, scalable way to measure worst-case robustness without solving an intractable optimization problem for each input. Traditional adversarial robustness measures often rely on attack algorithms, which only find a specific perturbation that succeeds, or on formal verification, which is computationally expensive and rarely scales to large networks. CLEVER offers an alternative that combines theoretical guarantees with computational efficiency approximation, making it usable on modern architectures such as [neural networks](https://www.wikiprompt.org/wiki/neural-network) and [deep learning](https://www.wikiprompt.org/wiki/deep-learning) models. ## Methodology CLEVER computes a local robustness certificate by estimating the Lipschitz constant of the network's output function with respect to the input. For a classification network, the relevant function is the difference between the logits (pre-softmax outputs) of the true class and each competing class. The local Lipschitz constant captures how fast these logit differences can change when the input is perturbed. The key innovation is the use of extreme value theory. Rather than directly computing the global Lipschitz constant - which is impractical for deep networks - CLEVER samples gradients at random points within a ball around the input. It then fits an extreme value distribution (specifically, a generalized extreme value distribution) to the maximum gradient magnitude observed. The estimated tail parameter is used to derive a probabilistic lower bound on the minimal adversarial distortion. This approach avoids the need for mixed-integer programming or SAT solvers used in exact verification, making CLEVER significantly faster. The resulting score is an approximation with a confidence level, not a strict guarantee, but it has been shown to be tighter than many prior heuristic bounds. ## Applications and Comparison CLEVER has been used to benchmark the robustness of various classifiers, including those on image classification benchmarks like MNIST and CIFAR-10. In evaluations, it provided a more accurate ranking of model robustness compared to attack-based metrics like the Fast Gradient Sign Method or Projected Gradient Descent attacks. For example, a model with a higher CLEVER score was shown to require a larger perturbation to cause misclassification, correlating with better adversarial defense. Compared to formal verification tools such as Reluplex or Planet, CLEVER is less precise but scales to networks with millions of parameters. It has been applied to feedforward networks with ReLU activations and also to convolutional architectures, though extensions to recurrent and [transformer](https://www.wikiprompt.org/wiki/transformer)-based models are less common due to computational cost. ## Extensions and Variants Several extensions have been proposed. Multi-CLEVER (2019) extends the method to compute validated error bounds under random perturbations, not just worst-case. Robust-CLEVER integrates the score into adversarial training as a regularizer, guiding the network toward flatter decision boundaries. Other work has adapted CLEVER to handle \ell_2, \ell_\infty, and \ell_1 perturbation norms by providing tailored Lipschitz estimation techniques. Software implementations, including a TensorFlow and PyTorch toolbox, allow researchers to compute CLEVER scores on custom models. The method has also been used to study the robustness of randomized smoothing defenses, where the score is evaluated on the smoothed classifier. ## Limitations CLEVER's reliance on extreme value theory introduces statistical uncertainty. The lower bound is only valid with a specified probability (e.g., 95%), which may not be acceptable for safety-critical applications. For high-dimensional inputs, the number of gradient samples needed grows, increasing computation. The method also assumes the loss function is locally Lipschitz, which holds for most standard networks but can be violated by non-smooth activations like step functions. In practice, CLEVER sometimes underestimates robustness - adversarial perturbations smaller than the bound can still be found by sophisticated attacks. This gap motivated further research into more rigorous certificates, such as those based on Lipschitz constants via spectral normalization or barrier functions. ## Legacy The CLEVER score contributed to the growing field of robustness verification in [machine learning](https://www.wikiprompt.org/wiki/machine-learning). It influenced subsequent work on local robustness bounds under the umbrella of certified defenses, such as CROWN and SparseGROWN. Though newer verification methods offer tighter bounds, CLEVER remains a useful baseline for empirical studies and is cited in numerous adversarial robustness papers. ## See also - [Adversarial example](https://www.wikiprompt.org/wiki/adversarial-example) - Formal verification - Lipschitz continuity - Neural network robustness --- Source: https://www.wikiprompt.org/wiki/clever-score License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/) Last updated: 2026-09-14T04:23:35.259001+00:00