{
  "id": "load-balancing",
  "title": "Load balancing",
  "lang": "en",
  "source_lang": "en",
  "summary": "Load balancing distributes computational work or network traffic among resources to improve performance, utilization, scalability, and availability.",
  "category": "computer-science",
  "subcategory": "networks-security",
  "type": "concept",
  "aliases": [
    "Computational load balancing",
    "traffic load balancing"
  ],
  "generated_at": "2026-11-03",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/load-balancing/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/load-balancing/",
      "json": "https://aiwiki.page/zh/load-balancing.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Load balancing is the distribution of computational work or network traffic among multiple resources, such as processors, servers, storage devices, or network paths. Its purpose is to prevent individual resources from becoming overloaded while others remain underused, thereby improving throughput, response time, resource utilization, or availability. It is a central technique in distributed computing, parallel computing, and large-scale online services. A load balancer may be a dedicated appliance, a software component, or a decentralized mechanism implemented by the participating resources themselves."
    },
    {
      "heading": "Objectives and basic principles",
      "text": "A load-balancing system receives or identifies work, determines which resources are eligible to process it, and assigns that work according to a policy. The work may consist of individual requests, network connections, queued jobs, data partitions, or units of a parallel computation.\n\n“Balanced” does not necessarily mean that every resource receives the same number of tasks. Tasks can differ greatly in cost, and resources can differ in processing power, memory, or network bandwidth. An effective assignment distributes work relative to available capacity and the system’s objectives.\n\nCommon objectives include:\n\n- Throughput: increasing the amount of work completed per unit of time.\n- Latency: reducing the time required to complete an individual request, especially unusually slow requests.\n- Utilization: using available capacity without creating excessive queues.\n- Scalability: allowing resources to be added or removed as demand changes.\n- Availability: directing new work away from failed or unavailable resources.\n- Fairness: preventing particular users, workloads, or resources from being systematically disadvantaged.\n\nThese objectives can conflict. Consolidating work onto fewer machines may reduce energy consumption but leave less spare capacity for sudden demand. Spreading work widely may improve responsiveness while increasing communication costs.\n\nFor a batch of independent tasks, a common objective is to minimize the makespan, the time at which the last resource finishes its assigned work. In a simplified model with identical processors,\n\n\\[C_{\\max}=\\max_{1\\leq i\\leq m}\\sum_{j\\in A_i}p_j,\\]\n\nwhere \\(m\\) is the number of processors, \\(A_i\\) is the set of tasks assigned to processor \\(i\\), and \\(p_j\\) is the processing time of task \\(j\\). Real systems may also need to account for dependencies, transfer costs, and unequal resource speeds. Optimal assignment is computationally difficult for many formulations, so practical systems commonly use a heuristic or an approximate algorithm."
    },
    {
      "heading": "Historical development and scope",
      "text": "Load balancing developed across several related areas rather than as a single invention. Multiprocessor computers needed to distribute computational tasks; operating systems needed to schedule runnable work; and distributed systems needed to allocate jobs among networked machines.\n\nThe growth of commercial Internet services in the 1990s made server load balancing increasingly important. Popular services could distribute requests across a group of servers rather than rely on one increasingly powerful machine. Dedicated network appliances and software proxies became common ways to perform this distribution.\n\nLater developments in cloud computing, microservices, and serverless computing extended load balancing to rapidly changing groups of service instances. The underlying problem remains similar, but the available resources, their locations, and the unit of work can change on short timescales."
    },
    {
      "heading": "Static and dynamic approaches",
      "text": "Static load balancing\n\nStatic policies assign work without continuously measuring current resource load. Assignments may be predetermined or calculated from relatively stable information, such as configured server weights or task identifiers.\n\nExamples include cyclic assignment and fixed partitioning. Static methods have low coordination overhead and can be predictable, but they may perform poorly when task costs vary or machines slow down unexpectedly.\n\nA policy can be static even if it makes a new decision for every request: the defining feature is that it does not adapt that decision to current load.\n\nDynamic load balancing\n\nDynamic policies use observations such as queue length, active connections, processing utilization, or recent response times. They can adapt to changing demand and unequal task costs, but collecting and distributing measurements introduces overhead.\n\nMeasurements are also imperfect. Reported load may already be outdated when an assignment occurs. If many dispatchers simultaneously select the apparently least-loaded server, they can collectively overload it. Randomization, smoothing, and limits on assignment rates can reduce such effects.\n\nDynamic systems may be:\n\n- Centralized, with one logical dispatcher making decisions.\n- Distributed, with multiple dispatchers or workers coordinating assignments.\n- Hierarchical, with decisions made at several levels, such as region, cluster, and individual server.\n\nA centralized dispatcher need not be a single physical machine; its implementation may itself be replicated."
    },
    {
      "heading": "Common distribution policies",
      "text": "Policy | Principle | Main limitations\nRound robin | Assign successive work items to successive resources in a cycle. | Equal request counts do not imply equal computational work.\nWeighted round robin | Allocate more work to resources with higher configured weights. | Fixed weights may not reflect temporary congestion or changing task costs.\nLeast connections | Select the resource with the fewest active connections, sometimes adjusted by weight. | Connections may differ greatly in activity and processing cost.\nLeast outstanding requests | Select the resource handling the fewest unfinished requests. | Outstanding requests can still differ substantially in cost.\nRandom selection | Choose a resource randomly, possibly using weights. | May produce temporary imbalance, especially in small pools.\nPower of two choices | Sample two resources and choose the less-loaded one. | Requires a useful load signal; performance depends on workload and implementation.\nResponse-time-based selection | Favor resources with lower measured response times, often combined with concurrency information. | Measurements can be noisy and may reflect network delay rather than server capacity.\nHash-based assignment | Map a client, connection, or object identifier to a resource using a hash function. | Popular identifiers can concentrate disproportionate work on one resource.\n\nThe power-of-two-choices approach illustrates how limited information can be effective: sampling a small number of candidates can avoid much of the overhead of finding the globally least-loaded resource.\n\nHash-based assignment is useful when repeated work benefits from reaching the same destination. Consistent hashing limits how many assignments change when resources join or leave. It reduces remapping but does not by itself eliminate uneven load or exceptionally popular data items."
    },
    {
      "heading": "Network and application load balancing",
      "text": "In networked services, a load balancer commonly presents an endpoint to clients and distributes traffic among backend servers. It may operate at different protocol layers.\n\nTransport-layer balancing\n\nA transport-layer load balancer commonly selects a backend using information such as source and destination addresses, ports, and protocol. This is often called Layer 4 load balancing, referring to the transport layer of the OSI reference model.\n\nFor TCP, a backend is typically selected for a connection, and subsequent packets must follow a compatible forwarding arrangement. Implementations may use packet forwarding, address translation, connection proxying, or direct server return, in which response traffic bypasses the load balancer.\n\nThis approach can have low processing overhead, but generally does not make decisions using application content.\n\nApplication-layer balancing\n\nAn application-layer balancer interprets an application protocol, commonly HTTP. It can route requests according to hostnames, URL paths, headers, or other application information. This is often called Layer 7 load balancing.\n\nSuch a balancer commonly functions as a reverse proxy, accepting client requests and communicating with backend servers on their behalf. It may terminate TLS, maintain backend connection pools, and direct different requests from a client connection to different servers.\n\nRequest-level distribution is particularly important when a protocol carries many requests over one connection. Counting connections alone may provide a poor estimate of actual load.\n\nDNS and geographic distribution\n\nThe Domain Name System can return different service addresses to distribute clients among endpoints or regions. DNS-based distribution operates before the client establishes a service connection. Resolver caching and client behavior mean that address changes do not immediately redirect all clients, and DNS cannot move an already established connection.\n\nGeographic traffic management may consider estimated network latency, regional capacity, cost, or data-location requirements. Anycast provides another mechanism: multiple locations advertise the same network address, and routing selects a reachable destination according to network policy. It does not inherently select the location with the most spare application capacity."
    },
    {
      "heading": "Load balancing in parallel computation",
      "text": "In parallel programs, load balancing distributes computational tasks among processors or workers. A program can have ample total processing capacity yet perform poorly if some workers finish early while others retain most of the remaining work.\n\nStatic partitioning can be effective when task costs are predictable. For irregular workloads, dynamic task queues allow workers to obtain new work as they finish previous tasks.\n\nIn work stealing, an idle worker obtains tasks from another worker’s queue. This can reduce reliance on a central dispatcher and adapt to unpredictable task durations. Its effectiveness depends on task granularity, queue-management costs, and the amount of transferable parallel work.\n\nLocality is a major constraint. Moving a task may require transferring data or may prevent reuse of data already present in a CPU cache. On machines with nonuniform memory access, work may execute more slowly when assigned far from its data. A perfectly even distribution of tasks can therefore be slower than a somewhat uneven distribution that preserves locality.\n\nSystems such as MapReduce and Apache Spark also face stragglers: tasks that finish much later than others. Work redistribution or speculative duplicate execution can sometimes reduce completion time, but duplication consumes extra resources and requires suitable handling of outputs and side effects."
    },
    {
      "heading": "Health, state, and availability",
      "text": "Health checking\n\nA load balancer needs to distinguish eligible destinations from those that cannot accept work. Active health checks send probes; passive checks infer problems from observed failures or timeouts.\n\nA successful basic probe does not necessarily mean that an application can serve useful requests. A process may respond while a critical dependency, such as a database, is unavailable. Conversely, overly strict probes can remove functioning servers during transient congestion and overload the remaining pool.\n\nHealth checks commonly use thresholds to avoid rapidly alternating between healthy and unhealthy states. During planned removal, connection draining allows existing work to finish while preventing new assignments.\n\nSession persistence\n\nSome applications store session state on the server handling a user’s requests. Session affinity, also called sticky sessions, attempts to send related requests to the same backend. Implementations may use a source address, a cookie, or another identifier.\n\nAffinity can conflict with even distribution, especially when a few sessions are unusually active. It also does not preserve server-local state if that server fails. Storing session state externally can allow requests to move more freely, but adds storage and coordination requirements.\n\nReliability of the balancing layer\n\nThe balancer can itself become a bottleneck or point of failure. Systems may use redundant instances, multiple network paths, or client-side selection to avoid depending on a single device.\n\nFailover does not always preserve established connections or in-progress requests. Retries may conceal some failures, but repeating an operation with side effects can produce duplicate results unless the application provides suitable guarantees."
    },
    {
      "heading": "Limitations and operational trade-offs",
      "text": "Load balancing redistributes available capacity; it does not create capacity. Autoscaling changes the number or size of resources and is therefore complementary rather than equivalent.\n\nSeveral limitations recur across implementations:\n\n- Shared bottlenecks: adding application servers may not help if all requests depend on one saturated database, storage system, or network link.\n- Overload feedback: long queues increase latency, timeouts trigger retries, and retries can further increase load.\n- Hot spots: a popular data partition or expensive request type can remain overloaded despite apparently even request counts.\n- Decision overhead: detailed monitoring and frequent reassignment consume resources and may negate the benefit of balancing.\n- Coordination effects: multiple independent dispatchers can make conflicting decisions using stale information.\n- Security boundaries: a public load balancer can support traffic filtering and rate limiting, but distribution alone does not prevent a denial-of-service attack.\n\nPerformance evaluation therefore considers more than average utilization. Relevant measurements include throughput, error rates, queue lengths, resource saturation, and latency percentiles. The same policy can perform well for short, uniform requests and poorly for long-lived connections or highly variable computational tasks."
    }
  ],
  "links": [
    "distributed-computing",
    "parallel-computing",
    "heuristic",
    "algorithm",
    "cloud-computing",
    "serverless-computing",
    "osi-model",
    "hypertext-transfer-protocol",
    "transport-layer-security",
    "domain-name-system",
    "cpu-cache",
    "mapreduce",
    "apache-spark",
    "database",
    "http-cookie",
    "denial-of-service-attack"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}