{
  "id": "federated-learning",
  "title": "Federated Learning",
  "lang": "en",
  "source_lang": "en",
  "summary": "Federated learning trains models across independently held datasets by exchanging computed updates rather than routinely pooling the underlying records.",
  "category": "computer-science",
  "subcategory": "artificial-intelligence",
  "type": "concept",
  "aliases": [
    "Federated machine learning"
  ],
  "generated_at": "2026-11-03",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/federated-learning/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/federated-learning/",
      "json": "https://aiwiki.page/zh/federated-learning.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Federated learning is an approach to machine learning in which multiple participants collaborate to train a model while retaining their underlying datasets within their own devices or administrative environments. Instead of routinely sending records to a central repository, participants compute information locally—such as parameter updates, gradients, or aggregate statistics—and exchange it through a coordinated training protocol. The participants may be mobile devices, hospitals, financial institutions, industrial facilities, or other organizations.\n\nThe approach connects distributed computing with data privacy, statistical learning, and institutional cooperation. Its importance lies in enabling some forms of joint analysis when collecting all training data in one place would be impractical, costly, or restricted. Keeping records local, however, does not by itself make the training process private or secure: transmitted updates and the resulting model can reveal information, and participants may supply unreliable or malicious contributions.\n\nFederated learning therefore encompasses more than a distributed optimization algorithm. Its design depends on how datasets differ, who controls the participants and coordinator, which parties are trusted, how communication is organized, and what outcomes the collaboration is intended to improve. These considerations distinguish large-scale learning across consumer devices from collaboration among a small number of institutions."
    },
    {
      "heading": "Origins and intellectual foundations",
      "text": "Federated learning developed from several older research traditions rather than from an entirely new mathematical foundation. Distributed optimization had long studied how multiple processors could jointly minimize an objective. Parallel machine learning used clusters to divide computation over large datasets, while privacy-preserving data mining examined collaboration without unrestricted disclosure of records. Research on decentralized learning and communication-efficient optimization supplied additional methods for exchanging less information between computational nodes.\n\nAn influential formulation appeared in the paper Communication-Efficient Learning of Deep Networks from Decentralized Data, first circulated in 2016 and published at the 2017 International Conference on Artificial Intelligence and Statistics. Its authors—H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas—described the Federated Averaging algorithm, usually abbreviated FedAvg. The work emphasized training on data distributed over devices, where communication was expensive and data were neither balanced nor identically distributed. On the experimental tasks studied, it reported reductions in required communication rounds of roughly 10–100 times relative to synchronized stochastic gradient descent.\n\nThe mobile-device setting gave the field several of its characteristic constraints. A phone may be available only intermittently, its network connection may be metered, and its owner’s usage patterns determine what examples it contains. Unlike workers inside a managed computing cluster, these devices cannot generally be treated as interchangeable, continuously available processors. Local computation became attractive because several training steps could be performed between network exchanges.\n\nThe term subsequently expanded to include collaboration among organizations. Hospitals and companies may operate reliable servers but face restrictions on transferring sensitive records or commercially valuable information. This setting shares the separation of data ownership found in mobile applications, yet its operating assumptions differ substantially. Participants can often be identified, contractual relationships can be established, and each institution may remain involved throughout training.\n\nFederated learning is consequently not defined by a single privacy technique, network topology, or model family. Artificial neural networks are prominent, but federated protocols also train linear models, tree-based systems, and other statistical models. Nor must a federation lack a central coordinator: many systems use a server to distribute models and combine contributions. The defining separation is between collaborative model training and routine central collection of the underlying examples, not necessarily between collaboration and central control."
    },
    {
      "heading": "Training objectives and the federated learning cycle",
      "text": "A common mathematical formulation assigns each participant a local objective and combines these objectives into a global one. For \\(K\\) participants, the optimization problem is\n\n\\[\\min_w F(w)=\\sum_{k=1}^{K}p_k F_k(w), \\qquad p_k\\geq 0,\\quad \\sum_{k=1}^{K}p_k=1,\\]\n\nwhere \\(w\\) denotes the model parameters and \\(F_k\\) measures performance on participant \\(k\\)’s data. For a local dataset \\(D_k\\) containing \\(n_k\\) examples, a typical objective is\n\n\\[F_k(w)=\\frac{1}{n_k}\\sum_{z\\in D_k}\\ell(w;z),\\]\n\nwith \\(\\ell\\) a loss function. This is a distributed form of empirical risk minimization, although privacy constraints or secure computation may prevent any single party from directly evaluating all its components.\n\nThe weights \\(p_k\\) encode a substantive choice. Setting \\(p_k=n_k/\\sum_j n_j\\) gives each training example equal weight, so institutions or devices with more examples exert greater influence. Equal participant weights instead give each client the same formal importance, regardless of dataset size. Neither choice automatically matches a desired population-level outcome: the participating clients may themselves be an unrepresentative sample, and records can differ in quality or relevance.\n\nA typical round begins when a coordinator selects an available subset of participants and sends them a model state. Each participant performs local training, often using backpropagation and a stochastic optimizer, then returns an updated model or its difference from the received model. The coordinator combines accepted contributions to produce the next global state. Training repeats until a resource limit or a performance-based stopping condition is reached.\n\nIn FedAvg, participant \\(k\\) begins round \\(t\\) at the shared parameters \\(w_t\\), performs several local optimization steps, and produces \\(w_{t+1}^{(k)}\\). A conventional aggregation rule is\n\n\\[w_{t+1} = \\sum_{k\\in S_t} \\frac{n_k}{\\sum_{j\\in S_t}n_j} w_{t+1}^{(k)},\\]\n\nwhere \\(S_t\\) is the set of participants whose contributions are included. This expression describes sample-weighted averaging among that round’s contributors; it does not guarantee that intermittent participation yields an unbiased estimate of the full federation’s objective.\n\nSeveral apparently minor implementation choices change the effective algorithm. Clients may perform a fixed number of steps or a fixed number of passes through their datasets. They may retain optimizer state between rounds or reset it. The server may apply its own optimizer to aggregated changes rather than directly adopt their average. Thus, “federated training” specifies a collaboration pattern, while convergence and model behavior depend on the detailed local and server-side procedures."
    },
    {
      "heading": "Participation models and data partitioning",
      "text": "A widely used distinction separates cross-device from cross-silo federated learning. Cross-device systems involve potentially very large populations of devices, with only a small subset participating in any round. Individual devices may have little data, limited computing resources, and unpredictable availability. Training typically assumes that many selected devices will fail to complete a round. Persistent per-device state can also be difficult to maintain reliably across a vast population.\n\nCross-silo systems involve a comparatively small number of organizations or other stable administrative units. Each participant may operate substantial computing infrastructure and contribute a large dataset. Identity, participation agreements, and operational responsibilities are usually more explicit. A hospital collaboration involving ten institutions is therefore not simply a smaller version of training across millions of phones: its communication patterns, incentives, privacy risks, and feasible security mechanisms can be different.\n\nA separate classification concerns how records and features are divided. In horizontal federated learning, participants possess different examples described by broadly compatible feature spaces. Several institutions might each hold patient records with the same variables, or devices might each contain text examples represented with a common vocabulary. A shared training procedure is then possible once label meanings, preprocessing, and model interfaces have been aligned.\n\nIn vertical federated learning, participants possess different attributes associated with overlapping entities. For example, two organizations may hold distinct information about some of the same customers. Training requires a way to determine relevant record correspondences and compute across feature partitions. Private set intersection can support selected forms of entity matching, while secure protocols can protect intermediate calculations. Matching itself nevertheless has a disclosure profile: revealing that a person appears in both datasets may be sensitive, even if additional attributes remain hidden.\n\nHorizontal and vertical partitioning do not exhaust the possibilities. Datasets may overlap in neither examples nor features, motivating methods related to transfer learning and representation alignment. Within a horizontal federation, label definitions may still disagree, sensors may use different units, or annotation practices may vary. Nominally identical columns do not guarantee semantic compatibility.\n\nThese classifications are independent of network topology. A cross-silo federation may use a central server, several regional aggregators, or a peer-to-peer exchange structure. Hierarchical aggregation can combine devices first at local or regional nodes and then at a higher-level coordinator. Each arrangement changes which parties observe individual contributions, where failures can occur, and which communication links become bottlenecks. Architecture must therefore be described separately from both data partitioning and participant scale."
    },
    {
      "heading": "Statistical heterogeneity and optimization methods",
      "text": "The statistical difficulty most associated with federated learning is that participants’ datasets need not be independent and identically distributed. This condition, often abbreviated as non-IID data, covers several different phenomena. Participants may have different proportions of the same labels, different input distributions, different relationships between inputs and labels, or different numbers of observations. A text-prediction model, for example, encounters vocabulary and writing styles that vary systematically across users rather than being randomly scattered among devices.\n\nLocal training amplifies some consequences of this heterogeneity. Starting from the same model, clients can move toward different local optima during several consecutive optimization steps. Averaging their resulting parameters is generally not equivalent to taking the same number of steps on the pooled dataset. This divergence is often called client drift. Increasing local computation can reduce communication needs while simultaneously worsening drift, creating a trade-off between network efficiency and optimization behavior.\n\nFedProx, introduced in 2020, modifies local training by adding a proximal penalty that discourages excessive departure from the current shared model. A common local objective is\n\n\\[F_k(w)+\\frac{\\mu}{2}\\lVert w-w_t\\rVert_2^2,\\]\n\nwhere \\(\\mu\\) controls the strength of the constraint. The method was designed to address both statistical differences and variation in how much computation clients complete. It does not make all heterogeneous learning problems equivalent, but it provides a mechanism for limiting local movement.\n\nSCAFFOLD, also introduced in 2020, uses control variates to correct discrepancies between local and global update directions. Other approaches normalize contributions when clients perform different amounts of work, maintain momentum at the server, or use adaptive server optimizers related to the Adam optimizer. These methods address distinct sources of instability; an algorithm effective under one experimental partition need not dominate under another.\n\nOptimization theory reflects this dependence on assumptions. Results may require smooth objectives, bounded gradient variance, controlled differences between local objectives, particular sampling schemes, or sufficiently small learning rates. Guarantees for convex problems do not directly establish the behavior of large nonconvex networks. Moreover, reducing the training objective does not by itself establish better generalization to unseen people, institutions, or future data.\n\nSome heterogeneity indicates that a single global model is an unsuitable target rather than that the optimizer is inadequate. If two institutions face genuinely different prediction tasks, forcing parameter agreement can reduce performance for both. Regularization, shared representations, and personalized models offer alternative formulations. A central research question is therefore not merely how to approximate centralized training more closely, but when centralized-style averaging represents the right statistical objective at all."
    },
    {
      "heading": "Communication, scheduling, and implementation",
      "text": "Federated systems exchange model-related information repeatedly, making communication a major resource cost. If a model has \\(d\\) parameters represented using \\(b\\) bits each, sending one uncompressed model requires approximately \\(db\\) bits, excluding protocol overhead. A model with ten million 32-bit parameters occupies about 40 megabytes in decimal units. Sending that model and a similarly sized update to thousands of participants across many rounds can outweigh the computational cost of local training.\n\nSeveral techniques reduce this burden. Update quantization represents values with fewer bits, while sparsification transmits only selected coordinates. Some compression methods retain an accumulated residual so that information omitted in one round can influence later transmissions. Such methods interact with optimization: biased compression can alter update directions, and independent sparse coordinate selections can be awkward to combine through privacy-preserving aggregation protocols. Smaller models and parameter-efficient adaptation reduce communication more directly but can restrict model capacity or trainable structure.\n\nCross-device scheduling must also respect device conditions. Production designs may make training eligible only when a smartphone is charging, idle, and connected to an appropriate network. This reduces interference with ordinary use but changes who is represented. Users with reliable connectivity and regular charging patterns may participate more often than others. The resulting availability bias can persist even when the coordinator randomly samples from currently eligible devices.\n\nSynchronous training waits for enough selected participants to complete a round. Deadlines prevent a few slow clients from delaying the entire system, but excluding slow participants can systematically remove particular hardware classes or user groups. Asynchronous methods accept updates at different times, reducing waiting but introducing staleness: a contribution may have been computed from parameters that the server has already replaced. Weighting, buffering, or limiting stale contributions becomes part of the optimization design.\n\nReliability requires more than collecting parameter arrays. Servers must track model versions, enforce compatible update formats, handle interrupted execution, and distinguish missing contributions from malformed ones. If secure aggregation is used, participant dropout affects the cryptographic protocol as well as the learning algorithm. Hierarchical deployments introduce additional coordination between aggregation levels, while organization-based systems must accommodate maintenance windows and local access controls.\n\nOpen-source software projects provide reusable components for these tasks. TensorFlow Federated supports federated computation and experimentation; Flower provides infrastructure for implementing federated workflows; and FATE emphasizes federated collaboration, including organization-based scenarios. Frameworks differ in their assumptions and supported protocols. A successful simulation on a single machine establishes neither realistic network performance nor resilience against adversarial participants, because simulated clients often share a trusted runtime and simplified failure behavior."
    },
    {
      "heading": "Privacy guarantees and information leakage",
      "text": "Data locality is a restriction on where records are processed, not a complete privacy guarantee. Model updates are functions of local examples and can carry information about them. In some settings, an observer with access to gradients or parameter changes can infer labels, detect properties of a participant’s dataset, or reconstruct aspects of training examples. The severity depends on factors such as batch size, model architecture, auxiliary knowledge, and whether individual or aggregated updates are visible.\n\nA useful analysis begins with a threat model. An honest-but-curious coordinator follows the protocol but attempts to infer additional information from what it sees. A malicious coordinator may deviate from the protocol, for example by sending specially chosen models to different participants. Other threats include colluding clients, compromised devices, and observers of the final model. Protecting against one of these actors does not automatically protect against the others.\n\nSecure aggregation allows a server to recover an aggregate, usually a sum, without directly learning each participant’s individual contribution. A practical protocol published by Keith Bonawitz and colleagues in 2017 became an influential design for federated settings. Masking relationships between participants can make individual vectors unreadable while allowing masks to cancel in the sum, with additional mechanisms handling some participant failures. The protection depends on protocol-specific assumptions about collusion, dropout, and the minimum number of contributors.\n\nDifferential privacy provides a different kind of protection by limiting how much an output distribution can change when a defined unit of data is added or removed. A randomized mechanism \\(M\\) satisfies \\((\\varepsilon,\\delta)\\)-differential privacy if, for neighboring datasets \\(D,D'\\) and every relevant output event \\(A\\),\n\n\\[\\Pr[M(D)\\in A] \\leq e^\\varepsilon\\Pr[M(D')\\in A]+\\delta.\\]\n\nThe neighboring-dataset definition is crucial. Example-level protection concerns one record; user-level or client-level protection can concern all records associated with one participant. In a cross-silo setting, treating an entire institution as the protected unit is a particularly demanding requirement.\n\nFederated differential privacy commonly clips contribution norms, adds calibrated noise, and tracks privacy loss over repeated rounds. The resulting privacy–utility trade-off depends on cohort size, clipping, sampling, noise, and training duration. Secure aggregation can conceal individual updates while differential privacy limits disclosure through released aggregates or models, but their composition requires careful accounting of every observable output.\n\nOther approaches use secure multiparty computation, homomorphic encryption, or trusted execution environments. Each changes computational costs and trust assumptions. Ordinary transport-layer security protects data in transit but does not prevent the receiving server from inspecting updates. Consequently, a description such as “encrypted federated learning” is incomplete unless it identifies what is encrypted, who holds keys, and what information remains observable."
    },
    {
      "heading": "Malicious participants, robustness, and accountability",
      "text": "A federation can fail even when no private information is disclosed. Participants may submit corrupted updates because of software defects, incompatible preprocessing, compromised devices, or deliberate manipulation. Since raw examples are not normally inspected centrally, the coordinator has fewer direct opportunities to verify whether a contribution accurately reflects legitimate data and training. A well-formed parameter vector is not evidence that the prescribed computation was performed.\n\nData poisoning alters the local training examples, labels, or preprocessing so that ordinary training produces a harmful contribution. Model poisoning acts more directly by replacing or modifying the transmitted update. An attacker may try to reduce overall accuracy, target a particular class, or implant a backdoor that causes a specific response when a trigger is present while leaving most ordinary predictions unchanged.\n\nThe distinction between an unusual but legitimate client and a malicious client is especially difficult under statistical heterogeneity. An institution serving a rare population can produce updates far from the majority without acting improperly. A defense that rejects all distant updates may therefore suppress precisely the information the federation was created to include. Conversely, an attacker can deliberately keep updates near expected statistical patterns to evade simple anomaly detection.\n\nResearch on Byzantine fault tolerance has inspired robust aggregation methods, including coordinate-wise medians, trimmed means, and distance-based selection rules. These methods can reduce the influence of some corrupted contributions under particular assumptions about attacker numbers, honest-update distributions, or update geometry. Their guarantees are conditional: defenses that work for tightly clustered honest updates can perform poorly when legitimate client distributions differ substantially.\n\nPrivacy protections create further constraints. If secure aggregation reveals only a sum, the server cannot apply an ordinary per-client distance test to hidden individual vectors. More elaborate protocols can verify selected contribution properties or compute certain robust statistics without exposing every update, but they add complexity and may disclose additional information. There is no general rule that stronger confidentiality automatically yields stronger integrity.\n\nOrganization-based federations can use authenticated membership, contractual duties, audited software, and signed protocol messages to establish accountability. Device-scale systems need scalable methods for limiting fabricated identities and coordinating abuse prevention. Norm bounds, validation checks, and restricted model changes can limit some attacks, but they do not prove that a participant trained honestly.\n\nEvaluation of defenses must specify the attacker’s knowledge and control: how many participants are compromised, whether the server is malicious, whether attacks adapt to defenses, and whether adversaries can coordinate across rounds. Reporting only average accuracy after one fixed attack leaves important questions unresolved, especially when a backdoor affects a narrow subgroup or a rare input pattern."
    },
    {
      "heading": "Personalization, evaluation, and benchmarks",
      "text": "A shared global model is only one possible product of federated training. Personalized federated learning seeks models adapted to individual participants or groups while retaining benefits from collaboration. Simple approaches first train a global model and then apply local fine-tuning. Other methods learn shared feature extractors with client-specific prediction layers, cluster clients with similar distributions, or jointly optimize related but distinct parameter sets.\n\nPersonalization changes what counts as success. A global model may be intended for a previously unseen device with no training history, whereas a personalized model may assume access to local examples at deployment. An institution with thousands of labeled cases can adapt extensively; a new user may have almost none. Comparisons are meaningful only when the amount and timing of adaptation data are specified, including whether labels become available before evaluation.\n\nThe allocation of importance across clients is another distinct issue. High aggregate accuracy can coexist with poor performance for small institutions or infrequent users. Useful reports therefore include both example-weighted and client-weighted results, distributions of per-client performance, and measures for relevant groups. These statistics describe different objectives rather than interchangeable estimates of one quantity. Improving the average does not necessarily improve the least well-served participant.\n\nEvaluation must also preserve the boundary between training and testing. Holding out examples within already-seen clients measures a different capability from holding out entire clients. The former tests prediction for known participants; the latter tests transfer to new participants. Time-based evaluation can be necessary when behavior changes, because a random split may hide distribution shift that occurs after deployment. Preprocessing, vocabulary construction, and hyperparameter selection can introduce data leakage even when the final training loop is federated.\n\nBenchmarks attempt to make these comparisons reproducible. LEAF, introduced in 2018, organized datasets and evaluation around federated characteristics such as naturally occurring user partitions. FedScale, introduced in 2022, combined datasets and an evaluation platform intended to study large and heterogeneous deployments. Such resources help compare algorithms, but no benchmark captures all combinations of device availability, institutional policy, privacy mechanisms, and adversarial behavior.\n\nArtificial partitions are particularly easy to misinterpret. Dividing a centralized dataset among synthetic clients with a chosen label-skew parameter provides controlled experiments, but it does not reproduce every form of real-world heterogeneity. Client availability can correlate with labels, data quality can differ by institution, and local tasks can evolve over time.\n\nResource reporting is equally important. Communication rounds alone omit differences in model size, participants per round, bytes transmitted, and local computation. A credible comparison accounts for wall-clock time, communication volume, client resource use, achieved performance, and any privacy budget. Otherwise, a method may appear more efficient simply because substantial work or disclosure has moved outside the reported metric."
    },
    {
      "heading": "Applications, institutional governance, and emerging directions",
      "text": "Mobile applications provided early demonstrations of practical federated training. Google reported work on improving keyboard query suggestions through federated learning in 2018, and subsequent systems used related methods for language and input models. Such applications exploit examples generated during ordinary device use while avoiding routine central collection of the complete local training corpus. They also illustrate the distinction between federated training and inference: a model can be trained collaboratively and then run locally, or federated training can produce a model deployed elsewhere.\n\nHealthcare collaborations are a prominent cross-silo application because relevant datasets are dispersed among institutions and governed by different access arrangements. In a 2021 Nature Medicine study, the EXAM collaboration trained a model using data from 20 institutions to predict supplemental oxygen requirements in patients with COVID-19 symptoms. This demonstrated multi-institutional model development without pooling the underlying institutional datasets in the ordinary centralized manner. It did not establish that federated learning generally guarantees clinical benefit or eliminates the need for external validation.\n\nHealthcare projects also expose problems unrelated to parameter exchange. Institutions may encode diagnoses differently, acquire images using different equipment, or serve populations with different clinical characteristics. A model can learn site-specific shortcuts rather than transferable relationships. Local data quality assessment, compatible outcome definitions, and evaluation at institutions excluded from training remain necessary components of the scientific study, whether records are pooled or federated.\n\nFinancial services, industrial monitoring, and recommendation systems provide other possible settings. Organizations may seek shared fraud indicators or equipment-failure patterns without transferring complete transaction histories or sensor archives. The value of collaboration depends on whether information generalizes across participants and whether the joint system outperforms strong local alternatives. Commercial incentives can be complicated when participants bear unequal costs or contribute data of unequal usefulness.\n\nGovernance determines who may join, which model versions are authorized, how purposes are limited, and who may receive or deploy the result. Agreements may address intellectual property, retention of updates, incident response, auditing, and responsibility for errors. Federated architecture does not automatically satisfy data-protection law: model updates and outputs can remain sensitive, and legal obligations depend on the data, processing activities, and jurisdictions involved.\n\nLarger foundation models introduce additional constraints because distributing complete parameter sets and maintaining local training state can be expensive. Parameter-efficient methods such as low-rank adaptation restrict training to a smaller set of weights, while knowledge distillation can exchange predictive behavior instead of matching every parameter. These alternatives can reduce some costs but introduce their own assumptions about shared inputs, model compatibility, and information disclosure.\n\nDeletion is another unresolved operational issue. Removing a participant’s raw records from its own storage does not necessarily remove their influence from an already trained global model. Machine unlearning studies ways to reduce or remove such influence, but federated settings add incomplete records of past participation, distributed checkpoints, personalized model copies, and privacy mechanisms that deliberately obscure individual contributions. Exact retraining, approximate removal, and verifiable deletion represent different technical and governance requirements."
    }
  ],
  "links": [
    "machine-learning",
    "distributed-computing",
    "data-privacy",
    "training-data",
    "artificial-neural-network",
    "loss-function",
    "empirical-risk-minimization",
    "backpropagation",
    "transfer-learning",
    "peer-to-peer",
    "adam-optimizer",
    "learning-rate",
    "generalization",
    "regularization",
    "smartphone",
    "open-source-software",
    "differential-privacy",
    "transport-layer-security",
    "fine-tuning",
    "distribution-shift",
    "data-leakage",
    "neural-network-inference",
    "foundation-model",
    "low-rank-adaptation",
    "knowledge-distillation"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}