A memory-mapped file is a file, or part of a file, associated with a range of addresses in a process’s virtual address space. A program accesses the mapped contents using ordinary memory reads and writes rather than explicitly transferring each block through file-input/output operations. The operating system manages the relationship between these addresses, physical memory, and the underlying file. Depending on the mapping mode, changes may update the file or remain private to the process.
Memory mapping provides a bridge between file storage and memory access. It is used for large data sets, executable loading, shared libraries, and communication between processes, but does not automatically make file access faster, concurrent access safe, or updates durable.
How it works
An application normally creates a mapping through an operating-system application programming interface, specifying a file, an offset, a length, access permissions, and sharing behavior. The operating system establishes a virtual address range associated with that part of the file.
If the mapping begins at virtual address and corresponds to file offset , then an address represents the byte at file offset , within the valid extent of the mapping.
Creating a mapping does not ordinarily require reading the entire mapped region into physical memory. On systems using demand paging, file contents can be brought into memory as they are accessed:
- The process reads or writes an address in the mapped region.
- If the required address translation or resident page is unavailable, the processor raises a page fault.
- The operating system establishes the appropriate mapping, using an already resident page or obtaining the data from storage.
- The interrupted instruction resumes.
A page fault therefore does not necessarily cause a storage read. The data may already be present in the operating system’s page cache, for example because another process accessed it.
After a page becomes resident and accessible, subsequent accesses can proceed through ordinary processor memory operations without a file-related system call for every access. The processor’s caches may further accelerate access, just as they do for other memory.
Virtual size versus physical residency
A mapping reserves or occupies virtual address space; it does not necessarily consume an equal amount of physical memory. A process can map a file larger than the available physical memory if its address space and operating-system limits permit it. The system keeps some pages resident and reclaims others as needed.
This does not make physical-memory limits irrelevant. Repeated access to a working set larger than available memory can cause heavy paging and poor performance. On systems with limited address space, applications may map successive windows of a file rather than the whole file at once.
Mapping modes
Mappings have both protection settings—such as read-only or writable—and rules governing whether modifications are shared.
| Mode | Effect of writes | Typical purpose |
|---|---|---|
| Read-only | Writes through the mapping are prohibited | Reading immutable data or executable resources |
| Shared writable | Writes modify the shared file-backed contents and can be written back to the file | File updates and shared data structures |
| Private writable | Writes create private modified data rather than updating the underlying file | Temporary modifications and process-private working data |
These are conceptual categories; exact combinations and behavior depend on the operating system and API.
Shared mappings
A shared mapping allows processes mapping the same file region to access common file-backed data. This can support shared memory and interprocess communication without copying each message through a separate communication channel.
Sharing bytes does not provide a complete coordination protocol. Concurrent readers and writers may still need synchronization, including process-shared locks or appropriately supported atomic operations. Without it, a reader may observe partially updated data or encounter a race condition.
Visibility between processes must also be distinguished from persistence on storage. Another process may observe an update while it is still only in memory.
Private mappings and copy-on-write
Private writable mappings commonly use copy-on-write. Unmodified pages can initially share physical storage with other mappings or with cached file contents. When a process writes to a page, the operating system provides a private writable copy.
These private modifications are not written back to the original file through the mapping. A private mapping is also not generally a guaranteed snapshot of the entire file at the instant it was created: the treatment of subsequent external file modifications depends on the platform and its documented semantics.
Interfaces and platform behavior
POSIX systems
On POSIX systems, including Linux and many Unix systems, the principal interface is mmap. Common operations and flags include:
PROT_READ,PROT_WRITE, and related protection flags.MAP_SHAREDfor shared mappings.MAP_PRIVATEfor private copy-on-write mappings.munmapto remove a mapping.msyncto request synchronization of mapped modifications with the underlying file.
File offsets supplied to mmap generally must satisfy page-alignment requirements. To map data beginning at an arbitrary byte offset, a program can map from an earlier aligned offset and access the desired data at an offset within that region.
A successful mapping normally remains valid after the original file descriptor is closed. Closing that descriptor does not itself unmap the memory.
Windows
Microsoft Windows separates the creation of a file-mapping object from the mapping of a view into a process’s address space. Common interfaces include CreateFileMapping, MapViewOfFile, and UnmapViewOfFile.
A mapping object may support multiple views, including views in different processes. File-offset alignment requirements are tied to the system’s allocation granularity, which need not equal its page size.
Windows also supports mapping objects backed by the system paging file rather than an ordinary data file. These provide shared-memory facilities, but are distinct from mapping the contents of an existing disk file.
Related mechanisms
Anonymous mappings allocate virtual memory without an ordinary file as the backing object. They share much of the underlying virtual-memory machinery with file-backed mappings.
Memory-mapped input/output is a different concept: it assigns addresses to device registers or device memory. Accesses to those addresses can have hardware-specific side effects and caching rules. Ordinary file mappings generally represent stored data rather than device control registers.
Performance characteristics
Memory mapping can reduce some costs of conventional buffered file access. An explicit read often copies data from the operating system’s cache into an application buffer. A file mapping can expose the cached file-backed pages directly to the process, avoiding that particular copy.
It can also simplify random access. An application can address records directly instead of issuing separate seek and read operations or managing its own file-buffer cache.
However, memory mapping is not universally faster:
- Page-fault overhead: First access to a page may require kernel work even when no storage read is necessary.
- Unpredictable stalls: An ordinary-looking memory access may block while data is fetched.
- Address-translation costs: Large mappings can incur page-table memory and translation-cache overhead.
- Access-pattern effects: Sequential buffered reads may achieve excellent throughput with predictable read-ahead and fewer faults.
- Memory pressure: Performance depends on competition for physical memory and the operating system’s page-replacement behavior.
- Concurrency costs: Fault handling, page-table operations, and shared kernel structures can limit scaling in some workloads.
Calling mapping “zero-copy” describes the elimination of particular copying steps, not the absence of all data movement. Storage reads, cache-line transfers, private-page copying, and application-level transformations may still occur.
Persistence and consistency
A write through a shared mapping usually first changes a memory-resident page. The operating system can write that page back later. Successfully executing the memory store therefore does not establish that the data will survive a crash or power failure.
Three separate properties matter:
- Visibility: whether other participants can observe the change.
- Writeback: whether modified memory has been transferred toward its backing storage.
- Durability: whether the change survives the relevant failure conditions.
Synchronization operations such as msync, or Windows operations such as FlushViewOfFile and FlushFileBuffers, address different parts of this process. Their guarantees depend on the platform, file system, storage stack, and hardware. Unmapping a region or exiting a process is not a substitute for a defined durability protocol.
Memory mapping also does not provide transactions. Updating several fields may leave only some updates persistent after a failure. Systems requiring ACID properties may combine mapped storage with logging, copy-on-write file layouts, checksums, or other recovery mechanisms.
Synchronization for concurrent access and synchronization for persistence are separate concerns. A lock can coordinate processes without forcing data to stable storage, while flushing data does not by itself prevent concurrent modification.
Applications
Executable files and shared libraries
Operating-system loaders commonly map executable code and other program sections into process address spaces. Read-only code pages can be shared among processes, while writable regions receive suitable private or shared treatment. Demand paging allows execution to begin without loading every page immediately.
Databases and indexes
A database management system may use mapped files to access records or a database index through memory addresses. Mapping can simplify access to large on-disk structures and delegate caching to the operating system.
Other database systems prefer explicit input/output to retain tighter control over buffering, scheduling, error handling, and write ordering. Memory mapping is an architectural choice rather than an inherent database requirement.
Large data sets
Scientific computing, search systems, and machine learning applications can map arrays, model parameters, or other large binary data sets. Multiple processes may share read-only pages rather than maintaining independent copies.
Mapping is most directly useful when the file representation permits access to the needed data. Compressed or encoded content may still require decompression, decoding, or conversion before use.
Limitations and hazards
File truncation and access errors
A mapping does not guarantee that every address in its declared range remains backed by valid file data. If another process truncates the underlying file, later accesses to affected pages may fail. On POSIX systems, access to mapped pages beyond the end of the backing object can generate SIGBUS.
Storage failures can likewise surface during a memory access rather than as a return value from an explicit read operation. Error handling may therefore be less localized than with conventional file-input/output APIs.
Growing a file does not automatically enlarge an existing mapping. Access to additional content may require another mapping or a platform-specific resizing operation.
Data representation
A mapped file is a sequence of bytes, not automatically a valid collection of native programming-language objects. Directly treating those bytes as structures requires attention to alignment, byte order, padding, object-lifetime rules, and the application binary interface.
Ordinary memory pointers are generally unsuitable as persistent references because a file can be mapped at different addresses in different processes or executions. File-relative offsets or other position-independent representations avoid that dependence.
Protection and security
Mapping permissions restrict permitted accesses, but do not validate file contents. A malformed mapped file can still exploit unsafe parsing or bounds handling. Applications must also account for whether another actor can modify or truncate the file while it is being accessed.
These issues are especially important when a mapped file is treated as a trusted in-memory data structure: valid addresses and readable pages do not establish that the stored values are safe or internally consistent.