Equal-cost multipath routing (ECMP) is a network routing technique in which a router forwards traffic toward a destination through multiple next hops that have the same best routing cost. Instead of selecting a single next hop, the router retains a set of equally preferred alternatives and distributes traffic among them. ECMP provides a form of load balancing within the network and allows multiple links to carry traffic simultaneously. “Equal cost” refers to the routing system’s selection criteria, not necessarily equal physical distance, bandwidth, latency, or congestion.
Routing and forwarding
ECMP involves two related but distinct operations:
- Route selection: the routing control plane identifies multiple eligible next hops for a destination.
- Traffic distribution: the forwarding plane chooses an eligible next hop for each packet, usually in a way that keeps packets belonging to the same flow together.
A routing protocol can discover equal-cost alternatives without prescribing exactly how packets must be distributed among them. The number of alternatives installed, the packet fields used for selection, and the selection algorithm depend on the router’s software, hardware, and configuration.
In a packet-switched network, a router normally forwards a packet toward a destination address using its forwarding table. An ECMP forwarding entry refers to a group of next hops rather than just one. The router chooses from that group locally; it does not ordinarily select or encode a complete end-to-end path.
Consequently, ECMP at several successive routers can produce many possible end-to-end paths. The forwarding decisions at different routers need not be coordinated.
Meaning of equal cost
For a shortest-path routing system with additive link costs, let be the minimum cost from router to destination , and let be the cost of a link from to neighboring router . The equal-cost next-hop set is
If this set contains multiple eligible neighbors, they can form an ECMP group.
For example, suppose router A can reach a destination through router B at total cost 20 and through router C at total cost 20. Both next hops may be installed. A third route with cost 25 is not part of that equal-cost group.
This formulation relates ECMP to the shortest-path problem. Link-state protocols such as Open Shortest Path First (OSPF) use shortest-path calculations related to Dijkstra’s algorithm and can retain several next hops associated with equal-cost routes. Intermediate System to Intermediate System (IS-IS) also supports equal-cost alternatives.
With positive link costs and consistent, converged shortest-path information, each selected next hop moves closer to the destination in routing-cost terms. During topology changes, however, inconsistent forwarding updates can still cause transient loops or packet loss.
Policy-based routing protocols
Border Gateway Protocol (BGP) does not rank routes using a single additive distance metric. Its route selection considers attributes and routing policy. BGP multipath implementations can nevertheless install several sufficiently equivalent routes.
Which BGP paths qualify depends on implementation and configuration. Multipath selection generally requires agreement on specified attributes, and some implementations offer options that relax particular equality requirements. Thus, BGP multipath should not be understood as accepting every advertised route with an apparently similar path length.
Traffic-distribution methods
Per-flow hashing
The most common ECMP distribution method uses a hash function over selected packet-header fields. For ordinary IP transport traffic, these often include the five-tuple:
- Source IP address.
- Destination IP address.
- Transport protocol.
- Source transport port.
- Destination transport port.
The resulting value selects a next hop, directly or through an intermediate bucket table. A simplified selection rule is
where is the flow key, is a hash function, and is the number of next hops.
Packets with the same selected fields normally receive the same next hop while the ECMP group and forwarding configuration remain unchanged. This preserves packet order more effectively than independently selecting a path for every packet.
The hash need not be cryptographic. Its primary purpose is to spread the expected traffic population across the available choices.
Per-packet distribution
A router can distribute individual packets using round-robin or another packet-by-packet rule. This may distribute traffic more evenly, including traffic from a single large flow. However, differences in path delay can cause packets to arrive out of order.
Reordering can complicate receiver processing and affect Transmission Control Protocol (TCP) loss detection and congestion-control behavior. For this reason, per-packet distribution is not interchangeable with conventional flow-hashed ECMP.
Flowlet-based selection
A flowlet is a burst of packets within a flow, separated from another burst by a sufficiently long idle gap. Flowlet-based schemes may select a different path at a burst boundary.
If the gap exceeds the relevant difference in path delay, switching paths can avoid reordering between bursts. Flowlet switching is used in some adaptive load-balancing designs, but it is not a required feature of ECMP.
Capacity and statistical balance
ECMP allows multiple paths to carry traffic concurrently, but it does not guarantee equal byte rates on those paths.
With uniform, independent flow hashing across next hops, the expected number of flows assigned to each is approximately
where is the total number of flows. Actual assignments vary, and equal flow counts do not imply equal traffic volumes.
A few large flows can dominate utilization while many small flows contribute little traffic. This is often described using the distinction between elephant flows and mice flows. Consequently:
- A network with many similarly sized flows may obtain good aggregate balance.
- A small number of large flows may leave some paths congested and others underused.
- A single flow normally uses only one selected path at each ECMP decision point.
Adding paths therefore increases potential aggregate capacity, but does not ordinarily combine their bandwidth for one flow.
Equal routing costs also do not establish equal link capacity. Two next hops may have identical configured metrics while leading to links of different speeds. Weighted distribution can account for unequal capacities, but weighted multipath is a distinct extension rather than a defining property of equal-cost forwarding.
Failures and changes to the next-hop group
When a link or next hop fails, the router must detect the failure and update the eligible forwarding group. Subsequent packets can then use surviving next hops. ECMP supplies alternatives, but does not by itself determine failure-detection speed, routing convergence time, or whether packets are lost during the transition.
Changing group membership also affects existing flow assignments. In the simple modulo rule, changing can remap many flows, including flows that were not using the failed next hop. Those changes can introduce packet reordering or disrupt systems that depend on path affinity.
More stable selection methods include:
- Hash-threshold selection: partitions a hash space among next hops.
- Rendezvous hashing: computes a score for each candidate and selects the highest-ranked one.
- Resilient hashing: commonly uses indirection through buckets so that membership changes can preserve many existing assignments.
These methods differ in computational cost, memory requirements, balance, and the amount of traffic remapped when membership changes. Consistent hashing provides a related general approach to limiting reassignment when the set of destinations changes.
Applications
ECMP is widely used in data-center networks, especially leaf–spine and Clos-style topologies that provide many equal-cost routes between endpoints. It enables parallel links and switches to contribute to aggregate network capacity without requiring a separate routing decision for every application connection.
It is also used in enterprise networks, service-provider networks, and cloud infrastructure where several equivalent routes are available.
Overlay networks introduce additional considerations. When packets are encapsulated in tunnels, an underlay router may see only the outer headers. If many inner flows share identical visible outer fields, they may all select the same path. Tunnel implementations can expose flow diversity through varying outer-header fields, and some forwarding devices can hash selected inner headers directly.
Limitations and operational considerations
Hash polarization
Repeated or correlated hash choices at successive routers can concentrate traffic onto a subset of the available end-to-end paths. This is called hash polarization. Different hash seeds, suitable field selection, or other implementation techniques can reduce such correlations.
Limited visibility
Ordinary hash-based ECMP distributes traffic without measuring current queue occupancy or congestion. A next hop remains eligible even if it is heavily loaded, provided the routing system still regards it as valid and equal cost. Congestion-aware or adaptive routing adds information and decision mechanisms beyond basic ECMP.
Path asymmetry and stateful processing
Forward and reverse traffic are not guaranteed to traverse the same devices. This matters when traffic crosses stateful firewalls, network address translation devices, or other systems that require packets from a connection to share state.
Symmetric hashing can help align decisions under suitable conditions, but does not guarantee identical bidirectional paths across an arbitrarily routed network.
Hardware limits
Forwarding hardware limits the number of next hops per group, the total number of groups, and the packet fields available for hashing. A control plane may therefore discover more equal-cost routes than a device can install.
Shared failure risks
Different next hops do not necessarily provide physically independent paths. Routes may converge downstream or share a cable route, power source, or other infrastructure. ECMP diversity alone is not proof of resilience against a particular failure.
Relationship to other mechanisms
Link aggregation combines multiple physical links into one logical link, usually between the same pair of devices. It often uses flow hashing, but operates at a different abstraction level from routing-layer ECMP. Both can be used together.
Unequal-cost multipath permits traffic distribution among paths with different routing costs or explicitly different weights. Its eligibility and allocation rules differ from ordinary ECMP.
Traffic engineering deliberately steers traffic according to capacity, policy, or performance objectives. ECMP can be one component of a traffic-engineering design, but basic ECMP does not optimize network-wide traffic placement.
Multipath TCP is an end-to-end transport mechanism that can use several subflows for one connection. Unlike ordinary ECMP hashing, it can allow a single application connection to benefit from multiple network paths, subject to the paths and subflows available.