The Roofline model is a performance-analysis framework that estimates the attainable computational throughput of a program from two hardware limits: peak computation rate and memory bandwidth. It relates these limits to a program’s arithmetic intensity, the number of arithmetic operations performed per byte transferred across a specified memory boundary. Originally developed for floating-point workloads on multicore processors, it is widely used to analyze parallel computing on CPUs, GPUs, and other accelerators. Its name comes from the roof-shaped upper boundary formed by a sloping bandwidth limit and a horizontal compute limit.
Origins and purpose
Samuel Williams, Andrew Waterman, and David Patterson introduced the Roofline model in a 2009 Communications of the ACM article, “Roofline: An Insightful Visual Performance Model for Multicore Architectures.” The original presentation used the term operational intensity, emphasizing operations relative to traffic between caches and main memory.
The model addresses a central problem in performance engineering: a processor’s advertised arithmetic throughput does not, by itself, predict application performance. A program must supply its execution units with data, and the rate of data movement may limit performance long before arithmetic capacity is exhausted.
Roofline analysis combines a compact hardware description with a workload characteristic. It is principally a tool for establishing performance bounds, classifying potential bottlenecks, and prioritizing optimizations—not a detailed simulation of processor execution.
Mathematical formulation
For a particular computational kernel, let:
- be the number of arithmetic operations performed;
- be the number of bytes transferred across the memory boundary under study;
- be the elapsed execution time;
- be the relevant peak or calibrated compute throughput;
- be the relevant memory bandwidth.
Arithmetic intensity is
and measured computational throughput is
The basic Roofline bound is
For floating-point workloads, is commonly expressed in FLOP/s, in bytes per second, and in FLOP per byte. FLOP denotes a floating-point operation, whereas FLOP/s denotes a rate.
The bound follows from two lower bounds on execution time:
Combining them gives
This formulation does not claim that computation and data transfer always overlap perfectly. Rather, perfect overlap is an optimistic condition consistent with the upper bound; incomplete overlap and additional overheads can make execution slower.
The ridge point
The compute and bandwidth limits intersect at the ridge point, also called the knee:
Below this intensity, the bandwidth term is the lower ceiling. Above it, the compute term is the lower ceiling. The ratio is often called the machine’s compute-to-bandwidth balance.
A machine with a high ridge-point intensity requires substantial data reuse before a workload can potentially reach its compute ceiling.
Reading a Roofline plot
A conventional Roofline plot uses logarithmic axes:
- The horizontal axis shows arithmetic intensity.
- The vertical axis shows computational throughput.
The bandwidth ceiling, , is a straight line with slope one on these axes. The compute ceiling, , is horizontal. Their lower envelope forms the roof.
A measured kernel appears as a point at . Its position supports two distinct observations:
- The roof above the point identifies the resource limiting the model’s upper bound.
- The vertical gap between the point and that roof indicates performance not explained by the two basic limits alone.
A point on the low-intensity side is therefore not automatically proof that the measured execution saturates memory bandwidth. It may instead be limited by memory latency, insufficient concurrency, dependencies, or instruction overhead.
For a consistently measured region,
so the ratio of throughput to intensity equals the achieved bandwidth at the selected memory boundary.
Defining work and data movement
Arithmetic operation counts
For floating-point arithmetic, a fused multiply-add is conventionally counted as two operations: one multiplication and one addition. This convention must match the one used to specify the compute ceiling.
The relevant ceiling also depends on precision and instruction type. Scalar arithmetic, vector arithmetic, and specialized matrix operations can have substantially different peak rates. A kernel using ordinary vector instructions should not be compared directly with a hardware peak attainable only through specialized matrix instructions.
For integer or mixed-operation workloads, analogous roofs can be constructed, but the operation definition must remain meaningful and consistent. Different instruction mixes may require separate throughput models.
Memory traffic
The byte count is not simply the size of the program’s input or the number of load and store expressions in its source code. It represents transfers across a specified boundary.
For a main-memory Roofline, data reused within a CPU cache may contribute many arithmetic operations without causing additional main-memory reads. Conversely, cache eviction, write allocation, writeback, and ineffective use of transferred cache lines can increase traffic.
This distinction connects Roofline analysis to the memory hierarchy. The same kernel may have different arithmetic intensities at the main-memory, last-level-cache, and first-level-cache boundaries.
“Arithmetic intensity” and “operational intensity” are often used interchangeably, but some sources distinguish them according to which traffic is counted. An analysis should therefore state the memory boundary explicitly rather than rely on terminology alone.
Worked examples
Consider a hypothetical device with:
using decimal prefixes. Its ridge point is
A kernel with intensity has a Roofline upper bound of
Increasing arithmetic capacity alone would not raise this bandwidth ceiling. A kernel with intensity , by contrast, has an upper bound of .
Vector addition
For double-precision vector addition,
each element requires one addition. An idealized traffic model counts two 8-byte reads and one 8-byte write, giving
This very low intensity makes vector addition a typical bandwidth-oriented workload. Actual traffic can be higher, for example when stores cause write-allocate reads, or lower at main memory when relevant data are already cached.
Matrix multiplication
For square dense matrix multiplication, the arithmetic work is approximately floating-point operations. Under an idealized data-movement assumption in which each input matrix is read once and the output is written once, transferred data scale as , so arithmetic intensity grows proportionally to .
That assumption requires sufficient effective storage and reuse. Real traffic depends on cache capacity, blocking, and how partial results are retained. Tiling improves reuse by organizing computation around blocks that fit into faster memory, moving the kernel toward higher intensity at the targeted boundary.
Ceilings and hierarchical extensions
Additional performance ceilings
The basic model can be supplemented with ceilings representing particular hardware or implementation constraints, including:
- Scalar rather than vector execution.
- Limited use of fused multiply-add instructions.
- Reduced active-core counts.
- Insufficient independent instructions.
- Bandwidth achievable only with suitable access patterns.
These ceilings show why a kernel may remain below the machine’s nominal roof. Changes made by a compiler, such as vectorization or loop transformation, may raise attainable throughput without changing the underlying arithmetic intensity substantially.
Hierarchical Roofline models
A hierarchical Roofline includes several memory levels, each with bandwidth and boundary-specific intensity :
Each intensity uses the bytes transferred across its corresponding boundary. A kernel can be efficient relative to main memory while still being constrained by traffic between a cache and the execution units.
Other extensions model instruction throughput, energy consumption, communication, or particular accelerator resources. In distributed computing, network data movement can be treated with related bandwidth-and-work reasoning, although communication latency and collective operations generally require additional modeling.
Measurement and optimization
A practical Roofline study normally defines a kernel or execution region, establishes appropriate hardware ceilings, measures runtime and operation counts, and estimates or measures memory traffic.
Two kinds of ceiling are common:
- Theoretical ceilings, calculated from hardware specifications.
- Empirical ceilings, obtained from benchmark measurements under documented conditions.
Empirical ceilings may better represent sustainable behavior, but depend on benchmark design, working-set size, access pattern, thread placement, and system state. A measured ceiling from one access pattern is not necessarily applicable to all others.
Performance counters and profiling tools can provide operation and traffic measurements. Their interpretation requires care: counter events may cover only particular instruction classes, count transfers at an unexpected boundary, or include activity outside the timed region.
Optimization generally targets one of three changes:
- Increase arithmetic intensity: improve data reuse, block loops, fuse kernels, avoid unnecessary intermediate arrays, or reduce transferred representation size.
- Raise achieved throughput at similar intensity: improve vectorization, scheduling, parallelism, or memory-access organization.
- Reduce total work: select a more efficient algorithm, even if the resulting FLOP/s is lower.
The last distinction is important: maximizing arithmetic throughput is not identical to minimizing runtime. Adding unnecessary arithmetic can increase reported intensity without making a program faster.
On GPUs, CUDA implementations may use shared storage and coordinated memory access to improve reuse and transfer efficiency. In deep learning, kernel fusion and attention implementations such as FlashAttention reduce intermediate data movement. Quantization can reduce storage traffic, but may also change operation counts, available compute instructions, and numerical behavior, requiring a new comparison rather than merely moving a point on an unchanged plot.
Limitations and interpretation
Roofline analysis deliberately compresses execution into a small number of quantities. Important limitations include:
- Latency and concurrency: dependent accesses or too few outstanding requests can prevent bandwidth saturation.
- Control flow and instruction mix: branches, address calculations, conversions, and non-arithmetic instructions can dominate execution.
- Synchronization and imbalance: waiting, unequal work distribution, and serial regions reduce throughput without necessarily changing intensity.
- Phase variation: an application-wide average can hide alternating compute-heavy and bandwidth-heavy phases.
- Hardware variability: clock changes, power limits, thermal conditions, and resource contention affect attainable ceilings.
- Placement effects: on NUMA systems, memory placement and thread affinity affect bandwidth and latency.
- Problem-size effects: small kernels may be dominated by launch or scheduling overhead rather than steady-state resource limits.
An observed point above a nominal roof usually indicates inconsistent assumptions or measurements—for example, cache-resident data compared with a main-memory roof, mismatched operation conventions, or a bandwidth ceiling that is too low.
The model complements, rather than replaces, other performance models. Amdahl’s law describes limits imposed by serial work when parallel resources increase; Roofline describes throughput limits arising from computation and data movement. Explaining a real application may require both, together with profiling of latency, synchronization, and communication.