Federated learning is a machine learning technique in which a shared model is trained across many decentralized devices or servers that each hold their own local data, without that raw data ever being centralized. Instead of uploading data to a central server, each participant computes an update to the model locally and sends only that update, which is aggregated with updates from other participants to improve a shared global model.
History and motivation
Google introduced the term and a practical system for federated learning in a 2016 blog post and accompanying paper, motivated by Gboard, its mobile keyboard app, which wanted to improve next-word prediction using what users typed without uploading that often-sensitive text to Google's servers. The technique addressed a growing tension between the deep-learning era's appetite for large training data and rising concerns about data privacy and regulation, formalized in laws such as the EU's GDPR. It offered a way to benefit from data generated on edge devices themselves, phones, hospitals' local servers, or bank branches, without the legal, logistical, or ethical burden of centralizing that data.
How it works
In the standard federated averaging algorithm, a central server sends the current global neural network model to a sample of participating devices. Each device trains the model briefly using its own local data via ordinary gradient descent, then sends the resulting model update, not the data, back to the server. The server averages the updates from all participating devices to produce an improved global model, and the cycle repeats over many rounds. Because raw data never leaves the device, federated learning offers a structural privacy advantage over centralized training, though it is not privacy-proof on its own; techniques such as differential privacy and secure aggregation are often layered on top to prevent the model updates themselves from leaking information about individual users, a concern closely related to broader AI safety and data-governance work.
Applications and limitations
Federated learning has been deployed in Google's Gboard keyboard prediction and Android's song-recognition feature, and has been explored in healthcare, where hospitals want to collaboratively train diagnostic models without sharing patient records across institutions, and in finance, for fraud detection across banks. Its limitations are significant: devices vary widely in compute, connectivity, and how much data they hold, which can bias the aggregated model toward the most active or best-connected participants; communication cost is nontrivial, since model updates must be transmitted every round; and coordinating training across many unreliable, low-power devices is more complex than training on a single well-controlled cluster. As of the mid-2020s, federated learning remains a specialized technique used where data cannot or should not be centralized, rather than a default alternative to conventional centralized training of large models such as large language models, which are still trained on centralized, curated datasets drawn largely from sources like Common Crawl.