In data analysis, anomaly detection is the identification of rare items, events, or observations that deviate significantly from the majority of data and do not conform to a well-defined notion of normal behavior. Such examples may arouse suspicion of being generated by a different mechanism or appear inconsistent with the remainder of the dataset. The concept is also referred to as outlier detection and sometimes as novelty detection, depending on the context and application domain.
Anomaly detection finds application in numerous domains, including cybersecurity, medicine, machine vision, statistics, neuroscience, law enforcement, and financial fraud detection. Historically, anomalies were first sought for clear rejection or omission from data to aid statistical analysis, such as computing the mean or standard deviation. They were also removed to improve predictions from models like linear regression, and more recently their removal aids the performance of machine learning algorithms. However, in many applications, anomalies themselves are of primary interest and are the observations most desired in the entire dataset, needing identification and separation from noise or irrelevant outliers.
Definition and terminology
Anomaly detection is the identification of observations, events, or patterns that depart substantially from expected normal behavior in a particular setting. There is no single operational definition that applies universally across domains, because the notion of normality depends on the data, the representation of observations, the context in which they occur, and the assumptions made by the detection method.
An anomaly is not necessarily a data error. Depending on the application, it may represent a measurement or recording error, a change in the process that generated the data, an unusual but valid observation, or an event of interest such as fraud, equipment failure, or a security intrusion. Anomalies are also not defined solely by rarity or distance from other observations; different methods characterize normality using probability distributions, distances or local densities, cluster membership, boundaries around normal data, prediction errors, reconstruction errors, or other model-specific criteria.
Many anomaly-detection methods return a numerical anomaly score rather than an immediate categorical label. The score represents the degree to which an observation departs from the fitted model of normality and is converted into a binary decision by applying a decision threshold. Consequently, whether an observation is labeled anomalous depends on the model of normality, the available context, and the decision rule used.
Types of anomalies
A widely used classification distinguishes three types of anomalies. A point anomaly is an individual observation considered anomalous relative to the rest of the data - for example, an unusually large transaction compared with other transactions in a dataset. A contextual anomaly, also called a conditional anomaly, is anomalous only within a particular context - an observation may be normal under one set of circumstances but anomalous under another, such as a temperature that is normal in summer but unusual in winter. Context can be temporal, spatial, demographic, or otherwise defined by the application. A collective anomaly is a related collection of observations that is anomalous as a whole, even when individual observations are not anomalous by themselves - for instance, an unusual subsequence in a time series or an unexpected pattern of activity in a computer network.
The appropriate category depends partly on the structure of the data. Point-anomaly methods may suffice for independent observations, whereas time series, spatial data, sequences, and graphs often require contextual or collective relationships to be modeled explicitly.
Related terms
The terms anomaly, outlier, and novelty are sometimes used interchangeably, but their usage varies among statistics, Machine learning, signal processing, and individual application domains. An outlier commonly denotes an observation that appears inconsistent with other observations in a dataset; outlier-detection methods often allow such observations to be present in the data used to fit the model. In novelty detection, the training data are generally assumed to represent normal behavior, and previously unseen test observations are assessed against the resulting model of normality. Novelty detection is consequently often formulated as a form of one-class classification. These distinctions are not applied consistently throughout the literature, and the broader term anomaly detection frequently includes all of these settings.
Techniques and categories
Three broad categories of anomaly detection techniques exist. Supervised anomaly detection techniques require a dataset labeled as "normal" and "abnormal" and involve training a classifier. However, this approach is rarely used because labeled data are generally unavailable and the classes are inherently unbalanced. Semi-supervised anomaly detection techniques assume that some portion of the data is labeled - typically normal data - and construct a model representing normal behavior, then test the likelihood that a test instance was generated by the model. Unsupervised anomaly detection techniques assume the data are unlabeled and are by far the most commonly used due to their wider and more relevant application.
Statistical methods
Statistical approaches model the normal behavior of data using probability distributions. Methods include parametric techniques such as Gaussian mixture models and non-parametric techniques based on histograms or kernel density estimation. Observations with low probability under the fitted model are flagged as anomalies. These methods often assume independence among observations:anomalies, making them less suitable for complex structured data without adaptation.
Distance-based and density-based methods
Distance-based methods characterize an observation as anomalous if it is far from its nearest neighbors. Density-based methods, such as local outlier factor, compare the local density around an observation with that of its neighbors; observations in regions of significantly lower density are flagged. These methods are particularly useful for identifying point anomalies in datasets with varying densities and do not require a priori assumptions about data distribution.
Clustering-based methods
Clustering-based anomaly detection groups data into clusters and identifies anomalies as points that do not belong to any cluster, are far from cluster centroids, or belong to very small clusters. Algorithms like k-means and DBSCAN have been adapted for this purpose. The approach is intuitive and can be efficient, but performance depends on the chosen clustering algorithm and its parameters, and the definition of what constitutes an anomalous cluster varies by application.
Machine learning and deep learning approaches
Modern anomaly detection increasingly relies on machine learning and deep learning techniques. Autoencoders, a type of neural network, learn to reconstruct normal data; high reconstruction error indicates an anomaly. One-class support vector machines learn a boundary around normal data in a high-dimensional feature space. Methods based on transformer architectures and large language models have also been explored for detecting anomalies in sequential data, such as logs or time series, leveraging their ability to model long-range dependencies and contextual patterns.
Applications
Cybersecurity
Anomaly detection is a foundational component of intrusion detection systems. The concept evolved significantly over time, starting as a manual process where system administrators monitored for unusual activities, such as a vacationing user's account being accessed or unexpected printer activity. By the late 1970s and early 1980s, analysis of audit logs and system logs was primarily retrospective for incident investigation, as data volume made real-time monitoring impractical. The affordability of digital storage enabled online analysis of audit logs with specialized programs, though these programs were typically run during off-peak hours due to computational intensity. The 1990s brought real-time intrusion detection systems capable of analyzing audit data as it was generated, shifting to proactive detection. Modern systems apply anomaly detection to network traffic, user behavior, and endpoint activities to identify zero-day attacks, insider threats, and other malicious patterns.
Financial fraud detection
In finance, anomaly detection identifies fraudulent transactions, such as credit card fraud, insurance claims abuse, and money laundering. Unusual patterns in transaction amounts, frequency, or geographic location can trigger alerts for further investigation. The field benefits from both supervised methods using historical fraud labels and unsupervised methods for discovering novel fraud schemes.
Medicine and healthcare
Medical applications include detecting abnormal physiological signals, identifying unexpected responses to treatments, and flagging anomalies in medical images such as MRI or CT scans. These systems help clinicians focus attention on cases that deviate from established norms, potentially improving diagnosis of rare conditions or early detection of equipment malfunctions.
Manufacturing and predictive maintenance
In industrial settings, anomaly detection monitors sensor readings from equipment to and predict failures before they occur. Unusual vibration patterns, temperature spikes, or acoustic signatures can indicate impending breakdowns, enabling maintenance teams to intervene proactively. This reduces downtime and maintenance costs across sectors such as manufacturing, aviation, and energy.
Challenges and limitations
Anomaly detection faces several persistent challenges. The definition of normality often changes over time, requiring models to adapt to concept drift. The rarity of anomalies makes labeled data scarce, limiting supervised approaches and complicating evaluation. Imbalanced datasets can lead to high false positive rates, which are costly in domains like cybersecurity where analysts must triage numerous alerts. Additionally, anomalies are context-dependent, and methods that perform well on independent point observations may fail on structured data like graphs or sequences without explicit contextual modeling.
The choice of decision threshold is critical; lowering it increases detection rate but also raises false alarms, while raising it may miss subtle anomalies. There is no universally accepted metric for comparing anomaly detection methods across domains, as the relative costs of false positives and false negatives vary significantly by application.
Relationship to other fields
Anomaly detection intersects with several areas of artificial intelligence and statistics. In generative AI, models like transformers can be used to estimate the likelihood of sequences, providing a basis for detecting unusual inputs. Research at institutions such as MIT CSAIL and Stanford AI Lab has contributed to theoretical foundations and practical algorithms. Companies like Amazon Web Services and Google Cloud offer anomaly detection services as part of their cloud platforms, integrating with machine learning pipelines. The field also draws on work in time series analysis, signal processing, and robust statistics, and it informs practices in data quality management and system monitoring.