OpenMP is a standardized application programming interface for parallel computing in C, C++, and Fortran. It defines compiler directives, runtime library routines, and environment variables for expressing parallel work and controlling its execution. Originally developed for shared-memory multiprocessors, it also supports task-based execution, vectorization, and offloading computation to accelerators such as graphics processing units (GPUs). OpenMP is maintained by the OpenMP Architecture Review Board, a consortium of hardware vendors, software vendors, and research organizations.
Unlike a standalone programming language, OpenMP extends existing languages. Programmers annotate selected regions of a program, while a supporting compiler and runtime system implement thread management, work distribution, synchronization, and device execution.
Programming interface
OpenMP has three complementary components:
- Directives describe parallel regions, loop worksharing, tasks, synchronization, data attributes, and device execution.
- Runtime routines query or configure execution, including thread identifiers, team sizes, timing, locks, and device information.
- Environment variables influence execution without changing the source, including the requested thread count, loop scheduling, thread placement, and nesting limits.
In C and C++, directives traditionally use #pragma omp. In Fortran, they use specially recognized comment sentinels, commonly !$omp. Some versions of the specification also provide additional directive spellings.
A C or C++ compiler can generally ignore an unrecognized OpenMP pragma, and Fortran sentinels can be treated as comments, allowing many annotated programs to retain a sequential form. This does not make every OpenMP program automatically portable to a non-OpenMP build: calls to runtime routines, conditional compilation, and code relying on parallel execution semantics may require separate handling.
OpenMP specifies program behavior rather than a particular implementation. A runtime may reuse a pool of worker threads, and different implementations may make different choices about task scheduling, thread placement, and device execution.
Execution model
Parallel regions and thread teams
The traditional execution model is fork–join. Execution begins with an initial thread. When a thread encounters a parallel construct, it forms a team of threads, each of which executes the associated structured block. At the end of the region, the team synchronizes, and the encountering thread continues beyond it.
This is a conceptual model: entering a parallel region does not necessarily create new operating-system threads on every occasion. Implementations commonly keep workers available for reuse.
A parallel region alone does not divide its code among threads. Without another construct, each participating thread executes the block. Worksharing constructs divide selected work within a team:
forin C and C++, ordoin Fortran, distributes loop iterations.sectionsdistributes separately written sections of code.singleselects one thread to execute a block.
Combined constructs such as parallel for create a team and distribute loop iterations in one directive.
OpenMP permits nested parallelism, but the number of active parallel levels and the actual team sizes depend on runtime settings, implementation limits, and available resources.
Task-based execution
An explicit task construct creates a unit of work that may execute immediately or be deferred to a thread in the current team. Tasks are useful for irregular computations whose work cannot conveniently be divided into a fixed collection of loop iterations.
Typical examples include tree traversal, recursive algorithms, and dependency-driven computations. The depend clause can describe ordering constraints between tasks, while taskwait and taskgroup provide mechanisms for waiting for related work.
Tasks are not equivalent to threads: a program may create many more tasks than there are worker threads. Their execution order should not be assumed unless the program establishes the necessary ordering.
Data sharing and synchronization
Shared and private data
OpenMP assigns data-sharing attributes to variables used in a construct.
- Shared variables refer to storage accessible by multiple participating threads.
- Private variables give each relevant task or execution context a separate instance.
firstprivatevariables provide private instances initialized from an existing value.lastprivatevariables copy a value from the logically last relevant iteration or section back to the original variable.- Reduction variables provide private accumulators whose values are combined according to a specified operation.
The applicable defaults depend on the construct and the variable. For example, associated loop iteration variables have predetermined data-sharing attributes. The default(none) clause requires explicit treatment of variables without predetermined attributes, making data-sharing decisions more visible.
Making a pointer private does not automatically make the memory to which it points private. The distinction between a variable and the storage it references is important in pointer-based programs.
Synchronization and memory consistency
OpenMP provides synchronization constructs with different purposes:
| Construct | Main purpose |
|---|---|
barrier |
Wait for the participating team threads to reach a common point |
critical |
Permit only one thread at a time to execute a protected region with the relevant name |
atomic |
Protect a supported operation on a designated storage location |
ordered |
Enforce specified ordering within supported loop regions |
taskwait / taskgroup |
Wait for specified sets of tasks |
| Lock routines | Provide explicit lock-based coordination |
Many constructs have an implicit barrier at their end. Where permitted, nowait removes that barrier, but the program must still satisfy any required data dependencies.
OpenMP defines a memory model governing when memory operations become visible across threads. Synchronization constructs establish the required ordering and visibility effects; simply placing data in shared storage does not make conflicting accesses safe.
Unsynchronized conflicting accesses can produce a data race, a particular form of race condition. Incorrect coordination can also cause deadlock. A flush operation participates in the memory-consistency model, but it is not a general substitute for mutual exclusion or a correctly designed synchronization protocol.
Loop scheduling and reductions
Scheduling
Loop scheduling determines how iterations are assigned to threads.
- Static scheduling assigns iterations according to a predetermined distribution, with low scheduling overhead.
- Dynamic scheduling lets threads obtain additional chunks as they finish previous chunks, which can improve load balancing when iteration costs vary.
- Guided scheduling generally starts with larger chunks and reduces their size as work is assigned.
- Runtime scheduling obtains the policy from runtime settings.
- Automatic scheduling delegates the decision to the implementation.
No schedule is universally best. Fine-grained dynamic scheduling can reduce imbalance but increase coordination overhead. Static scheduling can improve locality and predictability but perform poorly when work is uneven.
Ordinary worksharing loops require the program to respect dependencies between iterations. An annotation does not make an inherently dependent loop safely parallel.
Reductions
A reduction combines contributions from multiple execution contexts using an operation such as addition, multiplication, minimum, or maximum. OpenMP also supports user-defined reductions.
The following C function calculates a dot product:
double dot_product(const double *a, const double *b, int n)
{
double sum = 0.0;
#pragma omp parallel for default(none) shared(a, b, n) \
reduction(+:sum)
for (int i = 0; i < n; ++i) {
sum += a[i] * b[i];
}
return sum;
}
The iterations are distributed among threads, each participating thread accumulates a private contribution, and the reduction combines those contributions into sum. The arrays must contain the required elements and must not be modified concurrently in a conflicting way.
Floating-point reductions need not produce exactly the same bit pattern as sequential accumulation. Floating-point addition is not generally associative, so changing the grouping of additions can change rounding results. A scheduling policy alone does not guarantee reproducible floating-point results.
Vectorization and accelerator offloading
SIMD execution
The simd construct expresses opportunities for single instruction, multiple data execution, in which multiple operations are performed using vector lanes. SIMD parallelism is distinct from thread parallelism: a loop can use vectorization, multiple threads, or both.
Clauses describe properties such as reductions and alignment assumptions. The compiler must still generate code appropriate to the target architecture, and unsupported assumptions or unresolved dependencies can prevent correct or effective vectorization.
Target devices
OpenMP's device model allows selected code to execute on a target device. A target construct identifies device work; associated constructs such as teams, distribute, parallel, and simd express different levels of parallel execution.
Host and device memory cannot always be treated as a single interchangeable storage space. Mapping clauses and data-environment constructs control the association and movement of data. For devices with separate memory, transfers can be a major component of execution time.
A target region may execute on the host under specified fallback conditions. Its presence therefore does not, by itself, establish that an accelerator performed the computation. Successful device execution also depends on compiler support, runtime support, device code availability, and configuration.
OpenMP offers a common source-level interface across multiple device types, but performance portability is not automatic. Efficient execution on a CPU and on a GPU can require different choices of data layout, work decomposition, and memory management.
History and standardization
OpenMP emerged from efforts to standardize shared-memory multiprocessing directives across vendors. The first Fortran specification appeared in 1997, followed by a C and C++ specification in 1998.
Major stages in its development include:
- OpenMP 2.5, released in 2005: brought the language interfaces into a unified specification.
- OpenMP 3.0, released in 2008: introduced explicit tasking, expanding support for irregular parallelism.
- OpenMP 4.0, released in 2013: introduced major facilities for accelerator offloading and explicit SIMD programming.
- OpenMP 5.0, released in 2018: expanded support for heterogeneous execution, memory management, and interoperability.
Subsequent revisions have continued to extend these facilities. Implementations do not necessarily support every feature of a specification at the same time, and host support may be more complete than device support.
The name “OpenMP” does not mean that every implementation must be open-source software. The specification is implemented by both open-source and proprietary toolchains.
Applications and relationship to other models
OpenMP is widely used in scientific and engineering software, including numerical simulation, computational fluid dynamics, molecular modeling, and array-based numerical kernels. Its directive-based approach permits incremental parallelization: selected regions can be annotated while much of the surrounding program remains unchanged.
Compared with explicit threading interfaces such as POSIX threads, OpenMP usually requires less thread-management code. The runtime manages teams and schedules work, although programmers remain responsible for dependencies, data sharing, and synchronization.
The Message Passing Interface (MPI) primarily supports communication between processes with separate address spaces. OpenMP primarily provides shared-memory parallelism within a host and execution on associated devices. Hybrid MPI–OpenMP applications use MPI between processes or compute nodes and OpenMP within each process. OpenMP alone is not a general replacement for distributed computing across a cluster.
For accelerators, OpenMP overlaps with CUDA and OpenCL, but emphasizes directives integrated into existing application code rather than requiring all device work to be expressed through a separate kernel-oriented interface.
Performance and limitations
OpenMP reduces the amount of infrastructure needed to express parallel work, but it does not remove the underlying limits of parallel computation.
Serial work and overhead. Thread coordination, barriers, task creation, and scheduling consume time. Under the simplified assumptions of Amdahl's law, speedup on processors is bounded by
where is the parallelizable fraction of the original execution time. Real programs also incur parallel overhead and may encounter memory-system limits.
Memory bandwidth and locality. More threads do not necessarily increase the rate at which data can be supplied. Non-uniform memory access can make execution sensitive to where memory is allocated and where threads run.
Cache interaction. Threads updating distinct variables can still contend when those variables occupy the same cache line, a problem known as false sharing. This can cause performance loss without constituting a data race.
Work granularity. Very small tasks or loop chunks may cost more to schedule than they save. Conversely, a few oversized tasks may leave threads idle.
Correctness and reproducibility. Parallel execution can expose hidden dependencies and assumptions about execution order. Race freedom does not by itself imply deterministic results or an efficient algorithm.
Implementation differences. Supported clauses, optimization quality, device backends, and runtime behavior vary across compilers and versions. Source portability, functional correctness, numerical reproducibility, and performance portability are separate properties.