The Kalman filter, also known as linear quadratic estimation, is an algorithm in statistics and control theory that estimates unknown variables from a series of measurements observed over time. It accounts for statistical noise and other inaccuracies to produce estimates that are typically more accurate than those based on a single measurement. The filter operates recursively by estimating a joint probability distribution over the variables for each time-step, and it is named after the Hungarian-born engineer Rudolf E. Kálmán, who introduced the method in 1960. Kalman filtering has found widespread use in guidance, navigation, and control of vehicles, such as aircraft, spacecraft, and ships, as well as in time series analysis, signal processing, econometrics, and robotics. It is also applied to model the control of movement by the central nervous system, where it compensates for delays between motor commands and sensory feedback. The algorithm is designed for real-time operation, using only the present measurement, the previously calculated state, and its uncertainty matrix, requiring no additional past information. The filter is optimal under certain conditions: if the process and measurement covariances are known and the errors have a zero-mean Gaussian distribution, it is the best possible linear estimator in the minimum mean-square-error sense. Extensions such as the extended Kalman filter and the unscented Kalman filter handle nonlinear systems, and the filter has been effectively used in multi-sensor fusion and distributed sensor networks. The filter is named after Rudolf E. Kálmán, a Hungarian émigré, though Thorvald Nicolai Thiele and Peter Swerling developed similar algorithms earlier. Richard S. Bucy contributed to the theory, leading to the sometimes-used Kalman–Bucy filter name. Kálmán based his derivation on state-space variables applied to the Wiener filtering problem. The first implementation is attributed to Stanley F. Schmidt, who realized the filter could fit nonlinear problems during a visit by Kálmán to NASA's Ames Research Center. This led to the filter being incorporated into the Apollo program's mission navigation computer, using only 2k of magnetic core RAM, 36k of wire rope memory, and a clock speed below 100 kHz. Digital filter is also a special case of a more general nonlinear filter developed by Ruslan Stratonovich, and it is sometimes referred to as the Stratonovich–Kalman–Bucy filter. The method was first described in papers by Swerling (1958), Kalman (1960), and Kalman and Bucy (1961). Kalman filters have been essential in the navigation systems of U.S. Navy nuclear ballistic missile submarines and cruise missiles, as well as in the guidance, navigation, and control of reusable launch vehicles and spacecraft docking at the International Space Station.
Algorithm and Mathematical Framework
The Kalman filter is grounded in a hidden Markov model with a continuous state space, where both latent and observed variables follow normal distributions. The filter estimates the state of a dynamic system through a two-phase process: prediction and update. In the prediction phase, the filter uses the system's dynamic model and known control inputs to project the current state and its uncertainty (covariance) forward to the next time-step. In the update phase, the filter incorporates a new measurement, which is corrupted by noise, to refine the estimate. The refinement is a weighted average, where weights assign higher trust to sources with lower uncertainty. The weights are computed from a covariance matrix, leading to a new state estimate that lies between the predicted state and the measured value. The filter is recursive, allowing for real-time processing with limited memory and computation. The optimality of the filter is that, when the process and measurement noises are zero-mean and Gaussian with known covariances, the Kalman filter is the minimum mean-square error estimator. Even if noise is not Gaussian, it remains the best linear estimator in the minimum mean-square error sense provided the means and covariances are known. A common misconception is that the filter requires Gaussian noises, but it can be applied more broadly, albeit not as an optimal estimator.
History and Development of the Kalman Filter
The filtering method is named for Rudolf E. Kálmán, who was inspired to apply state variables to the Wiener filtering problem. The theory was developed in part by Peter Swerling and Kalman, with Kalman and Bucy extending it in 1961. Stanley F. Schmidt, working on the Apollo guidance and navigation system, is credited with the first implementation. He divided the filter into two parts: one for time periods between sensor outputs and another for incorporating measurements. This implementation ran on the Apollo navigation computer, which had a 2K-core memory and a 36k wire rope control memory, with a central processing unit built from integrated circuits running at under 100 kHz. The ability to run a Kalman filter on such limited hardware was a remarkable engineering achievement. The filter has since been essential in the guidance and navigation of U.S. Navy nuclear ballistic cruise missiles, including the Tomahawk missile and the Air-Launched Cruise Missile; it is also used in reusable launchers and spacecraft that dock at the International Space Station.
Sensor Fusion and Multi-Modal Integration
A key strength of the Kalman filter is its ability to fuse data from multiple sensors, also known as sensor fusion or data fusion. It produces a state estimate based on noisy sensor data, approximations in system equations, and other external factors. The filter deals effectively with uncertainty from sensor noise and random external disturbances, providing a robust method for tracking systems. For example, in a vehicle navigation system, the filter can leverage GPS, inertial measurement unit readings, and models to adjust estimates. In distributed sensor networks, consensus algorithms can be built, enabling systems to coordinate decentralized tracking. The state estimate is a weighted mean of the predicted state and the measurement statuses, with weights computed from the covariance, which represents the uncertainty. This approach ensures that more reliable data have a larger effect on the final estimate.
Extensions and Generalizations
Although the basic Kalman filter is restricted to linear systems with Gaussian noise, many extensions have been developed to handle nonlinearities. The extended Kalman filter (EKF) linearizes the system dynamics and measurement models via Taylor series around the current estimate, enabling in nonlinear contexts. The unscented Kalman filter (UKF) uses a deterministic sample technique to propagate the state distribution through a nonlinear system, achieving improved accuracy for certain cases. These methods are applied in fields such as Robotics, autonomous navigation, and time series analysis. Additionally, Kalman filters are used in econometrics to estimate state space models and in signal processing for filtering and prediction. In reality, the filter's model of a system is always an approximation, and the filter’s ability to correct for error via the update step is still important.
External Links and References
The Kalman filter is a foundational tool in the engineering and social sciences, with a wide range of applications including autonomous driving, autopilot systems, and navigation software. Its development was inspired by the Wiener filter, and its theoretical basis is deeply tied to control theory and signal processing. In artificial intelligence and machine learning, concepts and nonlinear extensions integrate with modern state estimation and data assimilation for training dynamic systems. The filter is also utilized in the center nervous system modeling for motor control, where it is proposed as a mechanism for the brain to handle delays in sensory feedback.