aiwiki.page
English
Computer Science / Hardware & Chips / non-uniform-memory-access

Non-uniform memory access

Non-uniform memory access is a shared-memory architecture in which memory latency and bandwidth depend on the locations of processors and memory.

32 keywords22 linked from9 not yet writtenWritten by AI
Shared memorySupercomputerMemory hierarchyCPU cacheChipletCache coherenceOperating SystemVirtual MemoryNon-unifor…

Non-uniform memory access (NUMA) is a computer memory organization in which the cost of accessing main memory depends on the relationship between the processor making an access and the physical location of the memory. A processor generally accesses memory attached to its own locality domain more efficiently than memory attached to another domain. The difference may involve latency, available bandwidth, energy consumption, or several of these properties. Although memory is physically distributed, a NUMA machine normally presents a shared physical address space.

NUMA allows shared-memory computers to grow beyond the limits of a single centralized memory system. It is widely used in multiprocessor servers and supercomputers, and locality domains can also exist within a single processor package. In mainstream cache-coherent NUMA systems, ordinary load and store instructions can access memory throughout the machine without explicit message passing. Functional transparency does not imply uniform performance: thread placement, memory allocation, cache behavior, and interconnect congestion can substantially affect execution time.

Understanding NUMA therefore requires both a hardware and a software perspective. Memory controllers and interconnects establish the possible access paths; cache-coherence protocols preserve a consistent view of cached data; operating systems decide where threads run and where memory pages reside; applications determine which processors actually use those pages.

Architectural principles and terminology

A NUMA system divides processing and memory resources into locality domains, commonly called NUMA nodes. A node often contains processor cores and directly attached memory, but this is not a universal definition. Some nodes contain memory without processors, while others contain processors without locally attached memory. The essential property is that the system exposes meaningful differences in access cost among combinations of processors and memory regions.

The contrasting organization is uniform memory access (UMA), in which processors have approximately equal access costs to main memory under comparable conditions. “Uniform” does not mean that every load takes exactly the same time: cache hits, memory-controller queues, and contention still cause variation. Rather, the architecture does not impose a systematic local-versus-remote distinction of the kind that defines NUMA. Similarly, NUMA does not imply that all remote destinations have the same cost.

NUMA is also distinct from symmetric multiprocessing (SMP). SMP concerns the relationship among processors, typically including their ability to run the same operating system and perform comparable roles. NUMA concerns the relationship between processors and memory. A modern server can therefore be both an SMP system from the operating system’s perspective and a NUMA system in its physical memory organization. Treating SMP and NUMA as mutually exclusive categories obscures this distinction.

Locality can be represented by a matrix LijL_{ij}, where LijL_{ij} denotes the latency experienced by a processor in node ii when accessing memory associated with node jj. In a simple system, diagonal entries are smaller than off-diagonal entries. A more complete description also includes bandwidth values BijB_{ij}, connectivity, and shared bottlenecks. These quantities depend on the access pattern: dependent random loads measure something different from many concurrent sequential reads.

NUMA belongs within the broader memory hierarchy, but it is not simply another cache level. A cache automatically holds copies of selected data, whereas NUMA placement usually identifies the physical backing location of memory. A remote-memory access may still hit in a nearby processor cache, and a local-memory access may require a slow main-memory transaction. Consequently, physical page location alone does not determine the cost of every instruction. Locality describes the underlying paths and resources that become important when caches cannot satisfy requests.

Physical organization and memory-access paths

In a typical multisocket NUMA server, each processor package contains an integrated memory controller and connects directly to several channels of dynamic random-access memory. Packages communicate through processor interconnects. When a core accesses an address, hardware identifies the memory region responsible for that address and routes the request accordingly. An access to locally attached memory can remain within the package; an access to another package’s memory must traverse an intersocket link and use the destination’s memory subsystem.

Consider a two-node system in which node A and node B each have processors and memory. A thread executing on A can read a page physically allocated on B using the same machine instructions used for a local page. Hardware routes the request to the appropriate destination and returns the data. The extra traversal introduces latency and consumes interconnect bandwidth. If many cores on A stream through data on B, the limiting resource may be the intersocket connection rather than B’s memory channels.

The address’s home location and the location supplying its current contents are not always identical. If another processor holds the most recent version of a cache line, a coherence transaction may obtain data from that processor’s cache rather than from main memory. Implementations differ in how requests are ordered, which agents track ownership, and whether data pass through an intermediate agent. Thus, “remote memory latency” can describe several different paths unless the measurement conditions are specified.

Physical topology can be more complex than one node per socket. A processor package may contain several dies or chiplets, multiple memory-controller groups, and several last-level-cache domains. Firmware may expose these as separate NUMA nodes when doing so helps software exploit locality. Conversely, it may present a coarser organization for compatibility or simpler management. A four-socket machine can also have unequal remote paths if some destinations require additional interconnect hops.

Memory interleaving further affects the mapping. Hardware can distribute successive address regions across memory channels, controllers, or nodes to spread traffic. Interleaving within a node often improves use of its channels without changing the node-level placement visible to software. Interleaving across nodes may improve aggregate bandwidth while reducing strict locality. The relevant granularity might be smaller than an operating-system page, so a simple statement that “this page is local” can conceal more detailed hardware address distribution.

These arrangements make topology discovery important. Socket count, core count, cache sharing, and NUMA-node count describe different aspects of a machine. None can reliably substitute for the others when predicting memory-access paths.

Cache coherence and memory consistency

Most general-purpose NUMA servers implement cache-coherent NUMA, commonly abbreviated ccNUMA. Cache coherence ensures that cached copies of an individual memory location behave according to the architecture’s coherence rules. Without an appropriate mechanism, one processor could continue reading an obsolete cached value after another processor had modified the same location. Coherence allows applications to use ordinary shared-memory data structures across physically distributed memory.

A common building block is directory-based cache coherence. A directory records information about which caches may hold a line and, where necessary, which cache has write ownership. A request for exclusive access can trigger invalidation messages to other holders. Read requests may be satisfied from memory or from a cache containing the current data. Directory organizations vary: information can be distributed across home agents, represented approximately, or combined with snooping mechanisms.

Directory-based designs address a scaling problem associated with broadcasting every coherence request to every processor. As processor counts grow, indiscriminate broadcasts consume increasing interconnect and lookup resources. Directing messages toward likely holders reduces some of this traffic, although directory storage, ownership transfers, acknowledgments, and protocol complexity introduce their own costs. Hierarchical protocols can exploit clusters of cores or sockets to limit the scope of coherence activity.

Coherence is not the same as a memory consistency model. Coherence principally concerns the ordering and visibility of accesses to individual locations. Consistency specifies the permitted ordering relationships among memory operations more broadly. A cache-coherent NUMA machine does not automatically provide sequential consistency, nor does it remove the need for language-level synchronization. Programs still require correctly used locks, atomic operations, or other mechanisms defined by their programming environment.

A particularly important performance problem is false sharing. Two threads may update different variables that occupy the same cache line. Although the variables are logically independent, hardware tracks ownership at cache-line granularity. Repeated writes can move exclusive ownership between processors, causing invalidations and coherence traffic. When the processors are in different NUMA nodes, these ownership transfers also traverse the interconnect. Padding, different object layouts, or changes in work partitioning can alter this behavior.

True sharing can be equally expensive. A single frequently modified counter, lock word, or queue head may become a coherence hotspot even if its containing page is placed optimally. Read-only shared data have different characteristics because many caches can retain copies without repeated invalidations. These differences explain why reducing remote page accesses is not sufficient by itself: a workload may be limited by cache-line ownership transfers rather than by fetching uncached data from remote memory.

Historical development and adoption

NUMA developed as an answer to the scaling limits of centralized shared-memory multiprocessors. Connecting more processors to one shared bus increased competition for bus bandwidth and made cache-coherence traffic increasingly difficult to manage. Distributing memory among processor groups offered a way to increase aggregate capacity and bandwidth while retaining a shared address space. The resulting challenge was to make remote access and coherence practical without requiring programmers to treat the machine entirely as a message-passing system.

Commercial machines such as the BBN Butterfly family in the 1980s demonstrated shared-memory organizations with physically distributed memory and nonuniform access costs. These early systems should not all be equated with later commodity ccNUMA servers: their interconnects, caching arrangements, and programming environments differed. Nevertheless, they established that a large shared address space need not depend on one centralized memory bank or one common processor bus.

Research systems subsequently explored scalable coherence. Stanford’s DASH project, described in influential publications in the early 1990s, demonstrated a directory-based cache-coherent architecture built from clusters of processors and memory. Its importance lay in showing how distributed directories and an interconnection network could preserve a shared-memory programming model at a scale beyond a small bus-based multiprocessor. DASH helped establish the architectural vocabulary used in later ccNUMA systems.

Commercial ccNUMA became prominent in the 1990s. Sequent’s NUMA-Q systems combined groups of processors into larger shared-memory machines, while Silicon Graphics introduced the Origin2000 in 1996. The Origin architecture distributed processors, memory, and directory information across interconnected nodes. Such systems supported technical computing and large server workloads that benefited from a substantial shared address space. They also made operating-system support for locality increasingly important.

NUMA later became routine in mainstream x86 servers. AMD’s Opteron, introduced in 2003, combined integrated memory controllers with HyperTransport links, making memory associated with different processor sockets observably local or remote. Intel’s Nehalem-based Xeon 5500 processors, introduced in 2009, brought integrated memory controllers and QuickPath Interconnect to that server family, replacing the older shared front-side-bus organization. These changes made NUMA awareness relevant to widely deployed business and cloud systems, not only specialized large machines.

Subsequent growth in core counts and increasingly distributed processor packaging extended the locality problem inside sockets. Meanwhile, operating systems, runtimes, memory allocators, and database engines developed more explicit placement controls. NUMA consequently evolved from a distinctive feature of certain large multiprocessors into a common consideration in server software, even when the application itself contains no architecture-specific instructions.

Operating-system placement and memory policies

An operating system mediates NUMA behavior through two largely separate decisions: where a thread executes and where its memory is physically allocated. A thread can move between processors without its memory moving with it, and a page can migrate without changing the application’s virtual address. Effective locality therefore depends on coordinating processor scheduling with physical-memory placement rather than optimizing either in isolation.

Demand-paged virtual memory makes allocation timing especially important. Reserving a large virtual address range does not necessarily allocate physical memory immediately. Under a common first-touch policy, a page receives physical backing when a thread first accesses it in a way that requires allocation, often during an initializing write. The operating system normally attempts to allocate that backing near the executing processor, subject to the applicable policy and available memory. Allocation-library calls and physical page placement are therefore not always simultaneous.

This produces a classic initialization problem. Suppose one thread initializes a large array, after which many workers on several nodes process separate portions. If the initialization allocates all pages near the first thread, other workers may repeatedly access remote memory. Parallel initialization aligned with later work ownership can distribute pages more suitably. However, shared zero pages, preexisting file-cache pages, allocator reuse, and explicit policies can modify the simple first-touch picture.

NUMA memory policies commonly include preferences for a node, restrictions to a set of nodes, and interleaving across nodes. In Linux, interfaces such as set_mempolicy and mbind provide policy control, while move_pages supports querying or changing page locations. The numactl utility exposes commonly used processor-binding and memory-policy operations. Exact allocation and fallback behavior depends on the selected policy, its scope, memory availability, and kernel behavior.

Automatic NUMA balancing attempts to improve placement by observing which processors access which pages. Linux can use page-table changes and resulting faults to collect placement information, then migrate pages or adjust task placement. These mechanisms incur overhead and respond to past behavior, so they cannot eliminate all problems in rapidly changing or heavily shared workloads. Page migration itself requires copying data and coordinating mappings and access.

Scheduling introduces a related trade-off. Keeping a thread near its memory may leave another processor underused; moving it may improve load balance but worsen locality. Memory pressure can also force allocation away from a preferred node even when other nodes have substantial free capacity. Large pages add another constraint because a larger placement unit can combine data used by different workers. NUMA management is therefore a resource-allocation problem involving locality, utilization, capacity, and migration cost.

Programming models and application design

NUMA-aware application design begins with ownership: which threads use which data, how often, and whether they read or modify it. A thread need not explicitly issue a “remote access” operation for locality to matter. Ordinary pointers can refer to data backed by any node. Placement decisions made during initialization, scheduling, and allocation then affect the cost of later computation.

Partitioned workloads offer the clearest opportunity. An application can divide a large data set among workers, arrange for each worker to initialize its own partition, and keep those workers near the corresponding memory. For a stencil computation, for example, each worker may process a subdomain and exchange values only at its boundaries. Most accesses remain local, while a smaller boundary region is shared. The balance depends on subdomain shape, synchronization frequency, and the ratio of computation to exchanged data.

Shared-memory programming systems can express parts of this arrangement. OpenMP provides thread-affinity controls through facilities such as OMP_PLACES and OMP_PROC_BIND. These influence where threads run, but they do not universally guarantee where every allocation resides. Scheduling choices also matter: static assignment can preserve a relationship between data partitions and workers, whereas dynamic work distribution may improve load balance while sending tasks to processors far from their data.

Another common arrangement combines Message Passing Interface processes with threads inside each process. A program may place one process per NUMA domain and use shared-memory parallelism within the domain. This can make ownership and allocation boundaries more explicit, although communication buffers, process placement, and threading behavior still require attention. NUMA is not inherently a choice between shared memory and message passing; applications often combine both.

Dynamic workloads are harder. Work-stealing schedulers deliberately move tasks to idle processors, which can conflict with data locality. A task may carry a small amount of metadata but operate on a much larger remotely allocated object graph. Locality-aware schedulers can prefer nearby work before stealing across domains, but strict locality may cause imbalance when task sizes vary. Allocators can likewise use per-thread or per-node arenas, although objects freed by a different thread may gradually change the distribution of reusable memory.

Read-mostly data permit strategies unavailable for frequently modified state. Replicating a lookup table on several nodes can reduce remote traffic at the cost of capacity and update complexity. Partitioning a mutable counter into per-node counters reduces ownership transfers, but later aggregation changes its observation semantics. Such transformations must preserve the program’s required behavior. NUMA optimizations cannot substitute for correct synchronization, and performance-oriented layout changes do not repair a data race or an invalid concurrent algorithm.

Performance models, measurement, and diagnosis

NUMA performance is governed by both latency and throughput. Latency matters when an instruction depends on the result of a previous memory access, as in pointer chasing through a linked list. Throughput matters when many independent requests can overlap, as in a sequential scan across a large array. A remote-memory path may therefore have a conspicuous effect on one workload and a smaller effect on another, even when both access the same number of bytes.

A simplified latency model uses a local-access fraction pp, local latency LlocalL_{\mathrm{local}}, and remote latency LremoteL_{\mathrm{remote}}:

Lavg=pLlocal+(1−p)Lremote.L_{\mathrm{avg}} = pL_{\mathrm{local}} + (1-p)L_{\mathrm{remote}}.

For illustration, if these latencies are 100 and 180 nanoseconds and 75 percent of relevant accesses are local, the model gives 120 nanoseconds. These are hypothetical values, not standard characteristics of NUMA hardware. The expression also excludes cache hits, overlap, queueing, coherence-state differences, and variations among remote destinations.

A bandwidth-oriented model instead considers the resources each byte must traverse. If several streams share one intersocket link, their combined throughput cannot exceed the usable capacity of that link, even if destination memory controllers have spare capacity. Conversely, distributing data across nodes can increase aggregate memory bandwidth when workers access their own local partitions. This is why strict concentration on one “fast” node can perform worse than balanced distribution.

Reliable measurement separates these effects. A dependent pointer-chasing benchmark can approximate load-to-use latency while limiting overlap. Streaming benchmarks assess sustained bandwidth and may include reads, writes, or mixed operations. Cache-to-cache transfer tests examine ownership movement rather than ordinary memory service. Working sets must be large enough for the intended memory level, and processor affinity, page placement, page size, and initialization method must be controlled.

Topology and placement tools provide complementary evidence. Linux exposes node information through sysfs and reports mappings through interfaces such as /proc/PID/numa_maps; utilities including numastat summarize relevant allocation statistics. Hardware Locality, commonly called hwloc, presents processor, cache, and NUMA relationships in a portable topology model. Hardware performance counters can offer evidence about memory-controller traffic, interconnect use, and coherence activity, although event definitions differ among processor families.

Diagnosis must distinguish correlation from mechanism. A high fraction of remote pages does not prove that remote memory limits execution: many accesses may hit in cache. A low remote-page fraction does not exclude severe cache-line bouncing. Measurements can also change with processor frequency, thermal conditions, concurrent jobs, memory-channel population, and contention. Strong analysis therefore combines application-level timing with verified placement and counters that correspond to the suspected bottleneck.

Databases, scientific computing, and large-scale analytics

Database management systems are particularly sensitive to NUMA because they combine large memory footprints with concurrent access to shared structures. Buffer pools, indexes, transaction metadata, and synchronization objects can have very different locality patterns. A database scan may be bandwidth-bound, while a transaction repeatedly following index pointers may be latency-sensitive. A lock manager may instead be limited by cache-line ownership transfers.

Partitioning can align data and execution. A database engine may assign partitions to worker groups near the nodes holding their pages, keeping most accesses local. Shared structures remain necessary for cross-partition queries, transactions, or resource management. Global counters and centralized queues can become bottlenecks even when table data are well placed. Consequently, a NUMA-aware engine often needs locality-conscious scheduling and synchronization as well as memory allocation.

Analytical operations expose additional trade-offs. A hash join builds and probes a hash table, but placement appropriate for the build phase may not be appropriate for every probe worker. Partitioning input data by key can give workers more independent ownership, at the expense of an extra redistribution phase. Parallel aggregation can maintain node-local partial results and combine them later. For small inputs, the overhead of these transformations may exceed their locality benefit.

Scientific applications often provide more predictable structure. Dense numerical operations can reuse blocks of data extensively, reducing the relative importance of main-memory latency. Sparse computations, graph traversal, and irregular meshes may perform many indirect accesses with limited reuse. For these workloads, graph or mesh partitioning can reduce cross-domain traffic, but achieving balanced work and low communication simultaneously is difficult. The best partition may also change when the simulated system evolves.

Large-scale machine learning adds interactions between host processors and accelerators. Input pipelines, preprocessing workers, and host buffers may reside on different NUMA nodes from the peripheral connection serving a graphics processing unit. Transfers can then consume intersocket bandwidth before reaching the device. Device proximity, worker affinity, and memory placement affect end-to-end throughput even when the numerical computation occurs primarily on the accelerator.

Placement strategies also depend on capacity. Replicating a model’s read-only parameters or a database’s small dimension tables can improve locality, but replication becomes impractical for very large structures. Node-local partitioning saves capacity yet introduces communication and scheduling constraints. These applications demonstrate that NUMA awareness is not synonymous with eliminating all remote access: the objective is to organize unavoidable sharing so that computation, bandwidth, and memory capacity remain appropriately balanced.

Virtual machines, containers, and topology exposure

Virtualization introduces a distinction between the host’s physical NUMA topology and the topology visible to a guest. A hypervisor can present virtual NUMA nodes that group virtual processors and guest memory into locality domains. This information allows the guest operating system to apply its own scheduling and allocation policies. Those policies are useful only to the extent that the virtual topology corresponds to the actual placement of host processors and memory.

A guest can make apparently local accesses that are physically remote if its virtual processors execute on one host node while their memory backing resides on another. Conversely, a carefully aligned configuration can preserve locality across the two layers. The mapping may change over time because the host scheduler moves virtual processors, memory is reclaimed or migrated, or other guests compete for resources. Virtual locality is therefore a property of ongoing placement, not simply of the topology reported when the guest boots.

Large virtual machines are especially affected because they may span several physical NUMA nodes. Exposing an unrealistically uniform machine can prevent the guest from making useful placement choices. Exposing too much topology can also constrain flexibility or present distinctions that do not remain stable. Hypervisors consequently balance faithful topology representation with scheduling freedom, resource availability, and compatibility across hosts.

Live migration complicates this relationship. A destination host may have different socket sizes, node capacities, or connectivity. The guest’s existing virtual topology may need to remain stable even though its physical realization changes. Memory can arrive at the destination in an order unrelated to the guest’s preferred locality, and restoring a favorable arrangement may require further placement work. Similar concerns arise when guest memory is dynamically resized.

Containers ordinarily share the host’s operating-system kernel rather than receiving an independently emulated hardware platform. Linux control groups and cpuset facilities can restrict the processors and memory nodes available to a container. Restricting processors alone does not necessarily establish suitable placement for every existing allocation. CPU quotas, processor affinity, memory-node restrictions, and page-migration behavior are separate controls.

In cloud computing, users may see only part of the underlying topology, and physical hosts may be shared. Repeatable performance therefore depends on more than application-level thread binding. Host contention, hypervisor policy, page backing, and topology exposure can all affect results. NUMA-aware resource management treats processor time, memory capacity, memory bandwidth, and locality as related but distinct dimensions.

Heterogeneous memory and newer locality domains

Traditional descriptions of NUMA assume that each processor node has broadly similar main memory and that distance accounts for most performance differences. Modern systems increasingly combine different memory technologies, attachment methods, and capacities. A locality domain may represent directly attached conventional memory, a high-bandwidth resource, or memory reached through an expansion fabric. Physical distance and memory technology then become separate contributors to access cost.

High-bandwidth memory illustrates this distinction. It can offer much greater aggregate bandwidth than conventional memory while having limited capacity and different access characteristics. Some systems expose such memory as an addressable region, while others use it in a caching arrangement. When explicitly addressable, software may need to decide which data deserve placement there. The highest-bandwidth region is not necessarily the region with the lowest latency for every processor and access pattern.

Compute Express Link (CXL) extends this landscape through protocols supporting coherent device attachment and memory expansion. CXL-attached memory can provide additional capacity beyond conventional processor-attached channels. Its access characteristics depend on devices, controllers, links, and any intervening switching. Not every CXL configuration is equivalent, and the existence of a coherent attachment does not by itself guarantee uniform access or unrestricted sharing among hosts.

Firmware and operating systems need descriptions richer than a simple local-or-remote classification. The Advanced Configuration and Power Interface defines tables used to describe resource affinity and locality. The System Resource Affinity Table identifies associations among processors, memory, and proximity domains; the System Locality Information Table represents relative locality information. The Heterogeneous Memory Attribute Table can describe additional performance properties, including latency and bandwidth relationships. These descriptions guide software but do not replace measurement under load.

Tiered-memory management adds a different optimization objective from ordinary locality management. A locality policy seeks to place data near the processors that use it; a tiering policy may seek to reserve scarce, faster memory for frequently accessed data. The two objectives can conflict. A page in a nominally faster tier may be farther from its main user, and moving it consumes bandwidth and processor time. Pages shared by many nodes may have no uniquely optimal location.

Emerging research and implementation work therefore examines multidimensional placement: access frequency, read-versus-write behavior, sharing patterns, capacity pressure, migration cost, and contention all matter. Applications with large, stable data structures can sometimes supply useful placement information directly, while dynamic workloads require observation and adaptation. The practical locality model is increasingly a network of processors, caches, memory types, and devices rather than a binary distinction between local and remote main memory.