# Concept Bottleneck Models

Concept Bottleneck Models are interpretable neural networks that first predict human-understandable concepts from inputs, then use those concepts to make final decisions, enabling intervention and debugging. Introduced in 2020 by Koh et al., they trade some accuracy for transparency.

Concept Bottleneck Models (CBMs) are a class of [neural network](https://www.wikiprompt.org/wiki/neural-network) architectures designed to improve interpretability by inserting a bottleneck layer of human-understandable concepts between the input and the final prediction. Instead of mapping raw data directly to an output, a CBM first predicts a set of predefined concepts (e.g., 'has wings', 'is red') and then uses those concept predictions to make the final decision. This design allows users to inspect which concepts influenced a prediction and to intervene by correcting concept values, making the model more transparent and debuggable than standard black-box models.

The framework was formalized in a 2020 paper by Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Brandon Ying, and Percy Liang, presented at the 34th Conference on Neural Information Processing Systems (NeurIPS 2020). The authors demonstrated that CBMs could achieve competitive accuracy on image classification tasks while offering interpretability benefits. They also introduced variants such as the 'interventionable' CBM, where a user can override predicted concepts to see how the final prediction changes, and the 'joint' CBM, which trains concept and final predictors simultaneously.

## Architecture and Design

A standard CBM consists of three components: a feature extractor (often a pretrained [ResNet](https://www.wikiprompt.org/wiki/residual-network) or similar [deep learning](https://www.wikiprompt.org/wiki/deep-learning) backbone), a concept predictor that maps features to concept probabilities, and a final predictor that maps concept probabilities to the output label. The concept layer is typically a linear or small multi-layer perceptron, and the final predictor is often a linear model to maintain interpretability.

Koh et al. explored three training paradigms: independent (train concept predictor first, freeze it, then train final predictor), sequential (train concept predictor, then final predictor with concept labels), and joint (train both simultaneously with a weighted loss). Their experiments on the CUB-200-2011 bird dataset (with 312 concepts) and the SUN attribute dataset (with 102 attributes) showed that independent CBMs had lower accuracy than standard models, but joint training closed much of the gap. For example, on CUB, a standard ResNet-18 achieved about 75% accuracy, while a joint CBM achieved around 72%, and an independent CBM around 66%.

## Interpretability and Intervention

A key feature of CBMs is the ability to intervene: if the model predicts a concept incorrectly (e.g., 'has wing color: brown' when the bird is actually blue), a user can correct that concept and re-run the final predictor. Koh et al. showed that even a single intervention on a mispredicted concept could improve accuracy significantly. On CUB, intervening on just one concept per image boosted accuracy from 66% to over 90% in the independent model, and similar gains were seen in the joint model. This makes CBMs useful in high-stakes domains where human oversight is valuable.

The intervention mechanism also enables debugging: by inspecting which concepts are frequently mispredicted, developers can identify weaknesses in the feature extractor or concept definitions. However, the quality of interventions depends on the concept set being comprehensive and accurately annotated, which is a major practical limitation.

## Extensions and Variants

Several extensions have been proposed since 2020. 'Concept Bottleneck Models with Label-Specific Concepts' (2021) by Rieger et al. allowed concepts to be shared across classes, reducing annotation burden. 'Post-hoc Concept Bottleneck Models' (2021) by Yukara Ikami et al. and later work by Wong et al. (2021) introduced methods to convert a pretrained black-box model into a CBM without retraining from scratch, using a concept discovery step. Another line of work, 'Concept Embedding Models' (2021) by Zarlenga et al., relaxed the binary concept assumption by using continuous concept embeddings, improving expressiveness.

In 2023, 'Interactive Concept Bottleneck Models' by Collins et al. added a mechanism for users to query the model for concept explanations during inference, and 'Concept Bottleneck Models with Concept Erasure' (2023) by Kim et al. addressed the issue of concept leakage, where the final predictor might rely on information not captured by the concepts. These extensions have been evaluated on datasets like CUB, SUN, and the synthetic dSprites dataset, with varying trade-offs between accuracy and interpretability.

## Applications

CBMs have been applied in medical imaging, where interpretability is critical. For example, a 2022 study by Graziani et al. used CBMs for skin lesion classification, defining concepts such as 'asymmetry' and 'border irregularity' from the ABCD rule, achieving an AUC of 0.91 on the ISIC 2018 dataset, comparable to a black-box baseline (0.93) but with the added ability to explain predictions. In radiology, CBMs have been used for chest X-ray diagnosis, with concepts like 'cardiomegaly' and 'effusion', as demonstrated in a 2021 paper by Chen et al. on the CheXpert dataset, where intervention improved accuracy from 0.82 to 0.87.

In autonomous driving, CBMs have been explored for predicting driver behavior or traffic scene understanding. A 2023 paper by Zhang et al. used CBMs for pedestrian detection with concepts like 'crossing' and 'looking', achieving 95% accuracy on a custom dataset, while allowing human correction of misclassified concepts. In finance, CBMs have been proposed for credit scoring, where concepts like 'income stability' and 'debt ratio' are used, though adoption remains limited due to regulatory and data challenges.

## Limitations and Challenges

The primary limitation is the need for concept annotations, which are expensive to obtain. For large-scale datasets, defining a complete and non-overlapping concept set is difficult. CBMs also tend to underperform black-box models on complex tasks, as the concept bottleneck restricts information flow. For example, on ImageNet, a CBM with 1000 concepts (one per class) achieved only 55% top-1 accuracy, compared to 76% for a standard ResNet-50, as reported in a 2021 study by Koh et al. follow-up work.

Another issue is concept leakage: the final predictor may inadvertently use correlations in concept probabilities that are not semantically meaningful, reducing interpretability. Researchers have proposed regularization techniques, such as orthogonality constraints on concept embeddings, to mitigate this. Additionally, CBMs assume concepts are independent, but real-world concepts are often correlated (e.g., 'has feathers' and 'can fly'), which can lead to redundant or conflicting predictions.

## Relationship to Other Interpretability Methods

CBMs are part of a broader family of interpretable models, including decision trees, rule-based systems, and attention-based explanations. Unlike post-hoc methods like [saliency maps](https://www.wikiprompt.org/wiki/gradient-clipping) or [LIME](https://www.wikiprompt.org/wiki/rlaif), which explain a black-box model after training, CBMs are inherently interpretable by design. They are related to [attention mechanisms](https://www.wikiprompt.org/wiki/multi-head-attention) in [transformers](https://www.wikiprompt.org/wiki/transformer), which also provide some interpretability, but attention weights are often noisy and not directly actionable. CBMs are also connected to [model pruning](https://www.wikiprompt.org/wiki/model-pruning) in that both aim to simplify models, but pruning reduces size while CBMs reduce opacity.

Recent work has combined CBMs with [large language models](https://www.wikiprompt.org/wiki/large-language-model) (LLMs). For instance, a 2023 paper by Oikarinen et al. used GPT-3 to automatically generate concept candidates for a dataset, reducing the need for human annotation. Another 2024 study by Yang et al. used an LLM to refine concept definitions iteratively, improving CBM accuracy on CUB by 3% over a fixed concept set. These hybrid approaches are an active area of research, especially as [generative AI](https://www.wikiprompt.org/wiki/generative-ai) models become more capable.

## Future Directions

Future research is focusing on scaling CBMs to larger datasets and more complex tasks, such as video understanding and multi-modal inputs. There is also work on learning concepts automatically from data, using unsupervised or self-supervised methods, though these concepts may not align with human semantics. Another direction is integrating CBMs with [reinforcement learning](https://www.wikiprompt.org/wiki/reinforcement-learning) for sequential decision-making, where interpretability could help in robotics and game playing.

As of 2025, CBMs remain a niche but influential approach, cited in over 2,000 papers. They are particularly valued in domains where regulatory requirements mandate explainability, such as healthcare and finance. However, their adoption is limited by the annotation cost and the accuracy gap, and it remains to be seen whether they can be made scalable and competitive enough for widespread deployment.

---
Source: https://www.wikiprompt.org/wiki/concept-bottleneck
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-09T01:59:51.754262+00:00