# Bias in AI

Bias in AI refers to systematic and repeatable errors in artificial intelligence systems that produce unfair outcomes, often privileging one group over another. It can emerge from biased data, design choices, or unintended uses, with impacts across hiring, criminal justice, healthcare, and more.

Bias in AI, also known as algorithmic bias, describes the systematic and repeatable harmful tendency in a computerized sociotechnical system to create unfair outcomes, such as privileging one category over another in ways that may differ from the intended function of the algorithm. This bias can emerge from many factors, including intentionally biased design decisions or the unintended or unanticipated use or decisions relating to the way data is coded, collected, selected, or used to train the algorithm. The study of algorithmic bias is most concerned with algorithms that reflect systematic and unfair discrimination, and it has been observed in search engine results, social media platforms, and other applications, with impacts ranging from privacy violations to reinforcing social biases of race, gender, sexuality, and ethnicity.

As algorithms expand their ability to organize society, politics, institutions, and behavior, sociologists have become concerned with the ways in which unanticipated output and manipulation of data can impact the physical world. Because algorithms are often considered neutral and unbiased, they can inaccurately project greater authority than human expertise, partly due to the psychological phenomenon of automation bias. In some cases, reliance on algorithms can displace human responsibility for their outcomes. Bias can enter into algorithmic systems as a result of pre-existing cultural, social, or institutional expectations; by how features and labels are chosen; because of technical limitations of their design; or by being used in unanticipated contexts or by audiences not considered in the software's initial design.

## Sources of Bias

Bias can be introduced to an algorithm in several ways. During the assemblage of a dataset, data may be collected, digitized, adapted, and entered into a database according to human-designed cataloging criteria. Programmers assign priorities or hierarchies for how a program assesses and sorts that data, requiring human decisions about categorization and inclusion or exclusion. Some algorithms collect their own data based on human-selected criteria, which can reflect the bias of human designers. Other algorithms may reinforce stereotypes and preferences as they process and display relevant data for human users, for example, by selecting information based on previous choices of a similar user or group.

Beyond assembling and processing data, bias can emerge as a result of design. Algorithms that determine the allocation of resources or scrutiny, such as determining school placements, may inadvertently discriminate against a category when determining risk based on similar users, as in credit scores. Recommendation engines that associate users with similar users or infer marketing traits might rely on inaccurate associations that reflect broad ethnic, gender, socio-economic, or racial stereotypes. Criteria for what is included and excluded from results can present unanticipated outcomes, such as flight-recommendation software that omits flights not following the sponsoring airline's flight paths. Algorithms may also display an uncertainty bias, offering more confident assessments when larger datasets are available, which can skew results toward larger samples and disregard data from underrepresented populations.

## Historical Context

The earliest computer programs were designed to mimic human reasoning and deductions, and were deemed to be functioning when they successfully and consistently reproduced expected outputs. As [machine-learning](https://www.wikiprompt.org/wiki/machine-learning) and [artificial-intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence) advanced, the reliance on data-driven decision-making grew, and concerns about bias became more prominent. A 2021 survey identified multiple forms of algorithmic bias, including historical, representation, and measurement biases, each of which can contribute to unfair outcomes. Historical bias arises when data reflects past societal inequalities, representation bias occurs when certain groups are underrepresented in training data, and measurement bias results from using flawed or incomplete metrics.

## Impact Areas

Algorithmic bias has been cited in cases ranging from election outcomes to the spread of online hate speech. It has also arisen in criminal justice, healthcare, and hiring, compounding existing racial, socioeconomic, and gender biases. The relative inability of facial recognition technology to accurately identify darker-skinned faces has been linked to multiple wrongful arrests of black men, an issue stemming from imbalanced datasets. In hiring, algorithms trained on historical resumes may favor male candidates if past hiring practices were biased. In healthcare, predictive models may underdiagnose conditions in minority populations due to insufficient training data.

## Legal and Regulatory Responses

Bias in AI has only recently been addressed in legal frameworks. The European Union's General Data Protection Regulation, enforced in 2018, includes provisions related to automated decision-making and the right to explanation. The Artificial Intelligence Act, proposed in 2021 and adopted in 2024, establishes risk-based requirements for AI systems, including measures to mitigate bias. These regulations aim to increase transparency and accountability, though enforcement remains challenging due to the proprietary nature of many algorithms.

## Challenges in Addressing Bias

Problems in understanding, researching, and discovering algorithmic bias persist due to the proprietary nature of algorithms, which are typically treated as trade secrets. Even when full transparency is provided, the complexity of certain algorithms poses a barrier to understanding their functioning. Furthermore, algorithms may change or respond to input or output in ways that cannot be anticipated or easily reproduced for analysis. In many cases, even within a single website or application, there is no single algorithm to examine, but a network of many interrelated programs and data inputs, even between users of the same service.

## Mitigation Approaches

Researchers and practitioners have developed various approaches to mitigate bias in AI. These include pre-processing techniques to balance datasets, in-processing methods that incorporate fairness constraints during model training, and post-processing adjustments to outputs. For example, [data-augmentation](https://www.wikiprompt.org/wiki/data-augmentation) can help address representation bias by generating synthetic examples for underrepresented groups. Fairness metrics, such as demographic parity and equalized odds, are used to quantify and monitor bias. However, there is often a trade-off between fairness and accuracy, and no single definition of fairness applies universally.

## Future Directions

As AI systems become more integrated into society, addressing bias remains a critical challenge. Interdisciplinary collaboration between computer scientists, sociologists, ethicists, and legal scholars is essential. Organizations like [openai](https://www.wikiprompt.org/wiki/openai), [anthropic](https://www.wikiprompt.org/wiki/anthropic), and [google-deepmind](https://www.wikiprompt.org/wiki/google-deepmind) have published research on fairness and bias, though their algorithms remain largely proprietary. The development of explainable AI, which aims to make model decisions more interpretable, is an active area of research. Ultimately, reducing bias requires not only technical solutions but also ongoing scrutiny of the social and institutional contexts in which algorithms are deployed.

---
Source: https://www.wikiprompt.org/wiki/bias-in-ai
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-13T03:59:22.581318+00:00