{
  "id": "write-ahead-logging",
  "title": "Write-Ahead Logging",
  "lang": "en",
  "source_lang": "en",
  "summary": "Write-ahead logging is a recovery technique that makes change records durable before the corresponding persistent data is overwritten.",
  "category": "computer-science",
  "subcategory": "software",
  "type": "concept",
  "aliases": [
    "WAL",
    "write-ahead log",
    "write-ahead log protocol"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/write-ahead-logging/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/write-ahead-logging/",
      "json": "https://aiwiki.page/zh/write-ahead-logging.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Write-ahead logging (WAL) is a technique for making changes to persistent data recoverable after a failure. Before a system writes modified data to its permanent location, it first makes the corresponding log records durable. If execution stops unexpectedly, recovery software uses those records to reconstruct required changes and, where necessary, reverse incomplete ones. WAL is fundamental to many database management systems and also appears in storage engines and other systems that maintain recoverable state.\n\nThe technique separates the durability of a transaction from the immediate persistence of every data page that the transaction changes. A transaction can commit after its necessary log records have reached durable storage, while its modified pages remain temporarily in memory. This arrangement supports efficient writes without requiring all data files to be updated atomically. It helps implement the atomicity and durability components of the ACID properties, but does not by itself provide transaction isolation, enforce application constraints, or protect against the destruction of every copy of the data and log.\n\nWAL denotes an ordering discipline rather than one universal file format or recovery algorithm. Systems differ in whether their logs contain old values, new values, physical page changes, logical operations, or complete page images. They also differ in checkpointing, concurrency control, replication, and storage assumptions. These choices determine how recovery works and what a successful commit actually guarantees."
    },
    {
      "heading": "The write-ahead rule and commit durability",
      "text": "The defining rule of WAL concerns the order in which information becomes persistent, not merely the order in which software issues write requests. For any modified data page, the log information needed to recover that modification must be durable before the page itself can become durable in its updated form. Writing a log record into a process buffer is insufficient: a crash could discard that buffer while an independently written data page survives.\n\nMany implementations identify positions in the log using a monotonically increasing log sequence number, or LSN. A page commonly stores a page LSN identifying the most recent logged modification represented in that page. If \\(L_{\\mathrm{durable}}\\) is the greatest log position known to be durable and \\(L_{\\mathrm{page}}\\) is the position associated with the page being written, the ordering requirement is commonly expressed as\n\n\\[L_{\\mathrm{durable}} \\geq L_{\\mathrm{page}}.\\]\n\nThe comparison is an implementation model rather than a requirement that every WAL system expose numerical LSNs. The essential property is that recovery information cannot be lost while the corresponding overwrite survives.\n\nA second requirement governs durable transaction completion. In a conventional synchronous-commit implementation, the system does not report a transaction as durably committed until the transaction's required log records, including its commit indication, have reached durable storage. The modified data pages need not be flushed at that point. After a crash, the log provides the information needed to reproduce their committed state.\n\nConsider a transfer that subtracts 100 units from one account and adds 100 to another. Both changed pages might remain in memory when the transaction commits. If the log contains sufficient recovery information and a durable commit record, restart recovery can reproduce both changes. Conversely, if an incomplete transaction has already changed persistent pages, an undo-capable recovery design can reverse those changes. WAL supplies the ordering foundation; the record contents and recovery algorithm determine which actions are possible.\n\nA crash can occur after a commit becomes durable but before the client receives its acknowledgment. The transaction is then committed even though the client does not know the outcome. This uncertainty is not a violation of WAL. It arises because recording a decision and communicating that decision are separate events. Applications that retry operations may therefore need transaction identifiers, deduplication, or other mechanisms to avoid performing a business operation twice.\n\nWAL consequently distinguishes three events that are often confused: modifying a page in memory, making the recovery record durable, and making the modified page durable. Correct implementations permit substantial freedom between these events while preserving the required dependencies."
    },
    {
      "heading": "Log records and representations of change",
      "text": "A WAL is usually organized as an append-oriented sequence, often divided into reusable or archived segments. Records may identify a transaction, the affected page or object, the type of operation, the record's length, and its position in the sequence. Additional fields can link records belonging to the same transaction or indicate how to interpret variable-length payloads. The exact layout is part of the storage engine's internal recovery interface.\n\nThe fundamental distinction is between information used to redo a change and information used to undo it. Redo information describes how to reproduce a modification that may not yet appear in the persistent data files. Undo information describes how to reverse a modification that should not remain. A before-image records the previous contents of some data; an after-image records the new contents. A log can contain either, both, or an operation-specific encoding that serves the required purpose.\n\nPhysical logging describes changes in terms of storage locations, such as replacing bytes at an offset within a page. Logical logging describes operations at a higher level, such as inserting a particular key into an index. Physical information generally permits direct replay, whereas logical information may remain meaningful even if the precise physical placement changes. Logical replay can nevertheless depend on indexes, object identity, execution semantics, and the state encountered during recovery.\n\nA widely used intermediate approach is physiological logging: a record identifies a physical page but describes the operation within that page in a more structured way. For example, a record might identify a leaf page of a B-tree and describe insertion into a slot rather than contain an entire replacement page. This combines explicit page targeting with a compact operation representation. Recovery schemes may use different representations for redo and undo because their correctness requirements differ.\n\nLogs also contain control records. These can mark transaction commit or abort, establish checkpoints, record changes to storage structures, or describe recovery actions. Systems with nested internal operations may distinguish structural work, such as an index-page split, from the user-level transaction that triggered it. Otherwise, reversing one transaction could incorrectly destroy a structure already used by other transactions.\n\nRecord framing must tolerate interrupted writes. Length fields, checksums, block headers, and sequence information help recovery distinguish complete records from an incomplete or invalid tail. A checksum can detect many corruptions, but it does not establish transaction commitment or guarantee that a device actually preserved a write. Some engines also include full-page images to recover pages damaged by partial writes.\n\nCompactness matters because every logged change consumes bandwidth and retained storage. However, a shorter encoding is useful only if replay remains unambiguous and correct. WAL format design therefore balances log volume, replay cost, compatibility, corruption detection, and the information necessary to recover under the engine's supported failure model."
    },
    {
      "heading": "Buffer management and the need for undo and redo",
      "text": "The relationship between WAL and recovery becomes clearer through two independent buffer-management choices: whether uncommitted changes may reach persistent data files, and whether committed changes must be written there immediately. These choices are traditionally described as steal versus no-steal and force versus no-force.\n\nA steal policy allows the buffer manager to write a page containing uncommitted changes to its persistent location. This can be necessary when memory is scarce or a transaction modifies more pages than the buffer pool can hold. If the transaction later aborts, the storage engine must ensure that those changes do not remain part of the database's valid state. In an in-place update architecture, this generally requires undo information or an equivalent mechanism for locating an earlier valid version.\n\nA no-steal policy prevents uncommitted modifications from being written into their permanent database locations. It simplifies one aspect of recovery because those locations cannot contain the prohibited uncommitted updates. The restriction can, however, require private copies, additional storage, or tighter constraints on buffer replacement. Copy-on-write and versioned designs can preserve earlier committed state without using precisely the same machinery as an in-place undo log.\n\nUnder a force policy, committing a transaction requires its modified data pages to be made durable. Under no-force, commit may finish before those pages reach their persistent locations. No-force improves flexibility and can avoid numerous scattered data writes at each commit, but it creates a need for redo: after a crash, committed work may exist only in the log. WAL makes this arrangement safe by preserving the information needed to reconstruct that work.\n\nThe commonly used combination of steal and no-force therefore calls for both undo and redo capabilities in conventional in-place transactional storage. Undo addresses uncommitted modifications that escaped to data files; redo addresses committed modifications that did not. The log need not be the sole repository for both kinds of information. Some engines maintain separate undo structures while using a redo-oriented WAL to protect changes to those structures as well as ordinary data pages.\n\nThese policies apply to pages, not cleanly separated transaction-sized objects. One page may contain changes from several transactions, some committed and some still active. Flushing that page cannot simply be equated with committing all its contents. The recovery machinery and concurrency-control rules must distinguish the transactions involved.\n\nThis explains why “the database files contain the latest data” is not a sufficient description of a running transactional system. Persistent pages may be ahead of some transaction decisions and behind others. The recoverable database state is determined by the interaction of those pages, durable recovery information, and transaction-status rules—not by the data files considered in isolation."
    },
    {
      "heading": "ARIES and restart recovery",
      "text": "ARIES, short for Algorithms for Recovery and Isolation Exploiting Semantics, is an influential recovery method associated with C. Mohan and colleagues at IBM. Its major description appeared in 1992 in ACM Transactions on Database Systems. ARIES provided a detailed framework for recovery with fine-grained locking, partial rollbacks, and flexible buffer management, including steal and no-force policies.\n\nIts best-known restart procedure has three phases: analysis, redo, and undo. Analysis reconstructs information about the state of the system at failure. Starting from an appropriate checkpoint, it examines log records to identify transactions that were active and pages that might require redo. The resulting transaction table and dirty-page table guide later work. Analysis does not normally require reading every database page.\n\nRedo follows the principle of repeating history. Rather than immediately replaying only transactions that ultimately committed, ARIES reconstructs logged actions that may be missing from the data files, including actions of transactions that will subsequently be undone. This reestablishes a state in which the logged history and page contents can be interpreted consistently. Redo also includes previously logged recovery actions when necessary.\n\nPage LSNs help prevent changes from being applied twice. In the standard page-oriented case, if a page already records an LSN at least as recent as the record being considered, that record's effect is already represented and need not be repeated. The dirty-page table provides further tests that avoid unnecessary page reads. These mechanisms allow recovery to accommodate data pages that were flushed at different times before the crash.\n\nUndo then reverses the effects of transactions that must not remain committed. Transaction log records can be linked backward, enabling recovery to follow a transaction's history without repeatedly scanning unrelated records. Crucially, undo work is itself logged using compensation log records, or CLRs. A CLR describes the redoable effect of an undo action and includes information that guides any remaining undo.\n\nCLRs make recovery robust against another crash during recovery. On the next restart, completed compensation actions can be redone, while undo traversal avoids reversing the same original action again. A CLR is not itself undone in the ordinary ARIES procedure. This is more precise than treating recovery as a one-time script that is assumed to finish uninterrupted.\n\nARIES also accommodates logical undo and internal structural operations. An index insertion, for example, may need to be undone even after page splits have changed the index's physical arrangement. Its recovery principles remain influential, but WAL and ARIES are not synonyms: a system can obey write-ahead ordering while using a different record format, transaction model, or restart procedure."
    },
    {
      "heading": "Checkpoints, log retention, and recovery time",
      "text": "A checkpoint establishes a recoverable reference point and records information that limits how far restart recovery must search. It does not necessarily mean that every dirty page has been written, every transaction has finished, or all earlier log records can be discarded. Those properties depend on the checkpoint protocol and on the other uses of the log.\n\nA simple checkpoint can temporarily restrict activity, flush relevant pages, and record a well-defined restart position. This is straightforward but may produce substantial pauses. Busy database engines commonly use fuzzy checkpoints, which allow transactions and page writes to continue while checkpoint information is collected. The checkpoint captures enough state, or enough recoverable information about state, for analysis to reconstruct what was happening.\n\nIn an ARIES-style system, checkpoint records can include the transaction table and dirty-page table. Each dirty-page entry tracks a recovery LSN: a conservative indication of the earliest logged change that might need redo for that page. The minimum relevant recovery LSN helps determine where redo scanning must begin. A long-lived dirty page can therefore keep the recovery starting point older than the newest checkpoint record.\n\nCheckpoint frequency creates a resource trade-off. More aggressive page flushing can shorten recovery and release pressure on finite log space, but consumes input/output bandwidth and may cause repeated writes to pages that are modified again soon afterward. Less aggressive flushing can improve foreground efficiency while increasing the amount of work required after a crash. Many systems spread checkpoint writes over time rather than writing a large burst at one instant.\n\nA rough model separates recovery into log scanning and page-access costs:\n\n\\[T_{\\mathrm{recovery}} \\approx \\frac{V_{\\mathrm{scan}}}{B_{\\mathrm{scan}}} + T_{\\mathrm{page\\ access}} + T_{\\mathrm{undo}},\\]\n\nwhere \\(V_{\\mathrm{scan}}\\) is the log volume examined and \\(B_{\\mathrm{scan}}\\) is effective scanning throughput. The expression is illustrative rather than predictive: parallelism, caching, page locality, and overlap between stages can change observed time considerably.\n\nLog retention is broader than local restart recovery. An old segment may still be needed by an active transaction, a backup, a replica, or a change-data consumer. A system cannot safely recycle a segment merely because its associated data pages have been flushed. It must account for every supported consumer whose recovery or progress depends on that segment.\n\nThis distinction explains how log storage can grow even when checkpointing appears healthy. A stalled replica or retained logical-decoding position can preserve old log segments independently of local dirty pages. Checkpoint completion, crash-recovery starting position, archive availability, and log-recycling eligibility are related but separate operational states."
    },
    {
      "heading": "Concurrency, group commit, and write performance",
      "text": "WAL often improves performance by replacing immediate scattered page writes with sequential or append-oriented log writes. A transaction that modifies many pages can become durable through a relatively compact stream of records. Data pages can then be written later in an order chosen for buffer management and storage efficiency. This benefit does not imply that logging is free: log bandwidth, synchronization, and durable-write latency can become central bottlenecks.\n\nThe simplest commit implementation flushes the log separately for every transaction. If a durable flush takes \\(t\\) seconds and only one transaction completes per flush, throughput along that serialized path is bounded approximately by\n\n\\[R \\leq \\frac{1}{t}.\\]\n\nFor an illustrative flush latency of 1 millisecond, that path permits about 1,000 such commits per second. This is not a general hardware performance figure; it demonstrates why the number of flushes matters independently of the number of log bytes written.\n\nGroup commit allows multiple transactions to share a durable flush. Transactions append their records and wait until the durable log position covers their respective commit positions. One flush can satisfy all those waiters without combining the transactions into a single atomic transaction. Each retains its own commit decision and transaction identity.\n\nBatching improves throughput but interacts with latency. A deliberate batching delay can collect more transactions per flush while increasing response time when load is low. Under sustained concurrency, naturally overlapping commit requests may provide large groups without an explicit delay. Engines use log-writer threads, queues, and notification mechanisms to coordinate this work.\n\nConcurrent log construction introduces another distinction: reserving a position is not the same as completing a record at that position. Several threads may reserve regions and fill them independently, but a flush cannot expose a valid-looking log beyond an unfinished gap. Implementations must track record completion, maintain safe publication rules, and ensure that the durable prefix contains interpretable records. Shared counters and log-buffer locks can themselves create contention under parallel workloads.\n\nLogging cost also depends on amplification. Updating a small field may require multiple records, index maintenance, transaction metadata, and occasionally full-page images. Conversely, repeated changes to one in-memory page may result in many log records but only one eventual data-page write. Compression and operation-specific encodings can reduce volume at the cost of processing and recovery complexity.\n\nWAL ordering is not a substitute for thread synchronization. Locks, latches, or other concurrency mechanisms protect live data structures and define how transactions interact. The log preserves recoverability across failures. A system needs both correct in-memory coordination and correct persistence ordering; satisfying either one alone does not establish transactional correctness."
    },
    {
      "heading": "Storage durability and failure assumptions",
      "text": "A correct WAL algorithm relies on a storage stack that honors its persistence requests. There may be several volatile layers between a database process and nonvolatile media: database buffers, operating-system page caches, controller caches, and device write caches. A successful ordinary write can mean only that data reached one of these layers. It does not necessarily mean that the data will survive power loss.\n\nOn systems with suitable semantics, database engines use synchronization operations such as fsync, fdatasync, or platform-specific equivalents to request durable completion. Direct input/output can bypass some buffering, but does not automatically solve persistence ordering or eliminate volatile device caches. Storage commands, barriers, force-unit-access mechanisms, and power-loss protection can all affect the guarantee actually delivered.\n\nThe failure model matters. A process crash may leave operating-system caches intact, whereas an operating-system crash or power failure may lose them. A device failure may destroy both data and log if they share the same failure domain. WAL primarily supplies a method for recovering from interrupted execution and incomplete propagation of writes, assuming the necessary durable records remain available and trustworthy.\n\nA torn page is a particularly important case. A database page may span several lower-level write units, and failure may leave a mixture of old and new contents. An ordinary byte-change redo record might assume that the rest of the page remains valid, so the write-ahead rule alone does not necessarily repair the damage. Engines therefore use additional mechanisms, including full-page images, doublewrite buffering, checksums, or storage guarantees strong enough for their page format.\n\nFull-page images place a recoverable copy of a page in the log under specified conditions. Doublewrite schemes first persist pages in an auxiliary location before writing their final locations, allowing recovery from incomplete final writes. Checksums help detect corruption, but detection alone does not reconstruct the missing bytes. The mechanism must match both the failure being considered and the available recovery information.\n\nLogs themselves can end with partially written records after a crash. Recovery normally identifies a valid prefix through framing and integrity checks, but arbitrary corruption in the middle of a log is a different problem from an interrupted final append. Hardware faults, software defects, and accidental overwrites can exceed the guarantees of ordinary crash recovery.\n\nSome systems offer settings that acknowledge commits before their logs are durably flushed. Such settings can improve latency while allowing recently acknowledged transactions to disappear after certain failures. The engine may still preserve recoverable structural consistency, but the acknowledgment no longer carries the same durability promise. “Uses WAL” therefore does not, by itself, specify a system's complete failure guarantees."
    },
    {
      "heading": "Implementation differences in major database systems",
      "text": "PostgreSQL uses WAL as the basis for crash recovery, physical replication, and recovery from archived logs. Its records describe changes needed for redo, including changes to indexes and transaction-status information. With full-page writes enabled, the first relevant modification to a page after a checkpoint can cause a full-page image to be logged, protecting recovery against partially written pages.\n\nPostgreSQL does not use an ARIES-style global undo pass to reverse all ordinary tuple changes made by transactions that were incomplete at the crash. Its multiversion concurrency control design stores tuple-version information and uses transaction status to determine visibility. Uncommitted tuple versions can remain physically present without being visible as committed data; later maintenance reclaims obsolete storage. This illustrates why WAL need not contain a conventional undo description for every logical row change.\n\nIn MySQL, the InnoDB storage engine distinguishes redo logging from undo logging. Redo records support recovery of changes to persistent pages. Undo records support transaction rollback and the reconstruction of older versions for consistent reads. Because undo information is itself stored in managed pages, changes to undo structures also require appropriate crash protection. The division is therefore functional, not a claim that undo data lies outside the recovery system.\n\nMySQL's binary log is another distinct component. It records database changes in forms used for replication and point-in-time recovery, but it is not interchangeable with InnoDB's internal redo log. Coordinating the storage engine's transaction outcome with the binary log is necessary to avoid inconsistent recovery decisions. The existence of multiple logs in one database product reflects their different consumers and correctness roles.\n\nSQLite introduced its WAL journal mode in version 3.7.0 in 2010. Rather than overwriting the main database file for each transaction, this mode appends frames containing database-page contents to a separate WAL file. Commit boundaries identify which frames form committed transactions. Readers can obtain a consistent snapshot by consulting the relevant WAL contents together with the main database file.\n\nSQLite checkpointing transfers appropriate committed page contents from the WAL into the main file. A reader may still need an older snapshot, so reader activity can constrain checkpoint progress and WAL reuse. SQLite WAL mode permits readers and a writer to operate concurrently, but it still allows only one writer at a time. Its coordination also ordinarily depends on shared-memory facilities available to processes on the same host.\n\nThese implementations share the write-ahead principle but differ in page propagation, version visibility, rollback, and log consumption. Features described as “WAL” should consequently be interpreted within a particular engine's architecture rather than assumed to imply ARIES, a specific concurrency model, or an identical durability configuration."
    },
    {
      "heading": "Replication, backups, and related logging techniques",
      "text": "Because WAL records describe recoverable changes, they can also transport state to another machine. In physical database replication, a primary sends log records that a standby replays against a compatible database image. The standby's received, durable, and replayed positions can differ. A record may have arrived over the network without being persistent, or be persistent without yet being reflected in pages available to queries.\n\nSynchronous replication adds acknowledgment conditions to transaction completion. Depending on configuration and implementation, the primary may wait for a remote system to receive log data, flush it durably, or apply it. These are distinct guarantees. A promise that a transaction exists durably on a standby does not necessarily mean that a query on that standby can already observe it.\n\nWAL is not itself distributed consensus. A local log orders recoverable work within one storage engine. A consensus protocol additionally coordinates agreement among participants under a defined failure model, including rules for leadership and committed decisions. Protocols such as Raft commonly rely on persistent logs, but their consensus properties arise from their replication, election, and commitment rules, not merely from appending records before updating data.\n\nArchived WAL can combine with a base backup to support point-in-time recovery. The backup supplies a starting database image, and replay advances it through subsequent logged changes. A backup taken while the database is running must be coordinated with the engine's backup protocol so that its pages and required log range form a recoverable set. Copying arbitrary live files and retaining an arbitrary collection of log segments is not equivalent.\n\nContinuous log availability is essential. A missing segment can prevent recovery past the gap, even if later segments are intact. Retention policy must therefore account for the age of usable backups, archive continuity, and the intended recovery interval. WAL archives complement backups; they do not eliminate the need for an initial recoverable image or protection against the loss of the archive itself.\n\nSeveral related techniques should be distinguished from WAL. A file-system journal may protect metadata operations, and sometimes file data, but generally does not understand a database transaction that updates several logical records. An application event log records domain events under application-defined semantics; it need not contain enough information to repair partially written storage pages. Logical change-data capture extracts committed changes for external consumers rather than necessarily reproducing the engine's internal recovery operations.\n\nCopy-on-write and shadow-paging designs instead preserve an older valid structure and publish new references when updates are ready. Log-structured storage organizes data placement around append-oriented structures. These approaches can coexist with WAL, but append-oriented storage layout, transactional event history, and write-ahead recovery ordering are separate architectural choices."
    }
  ],
  "links": [
    "database-management-system",
    "database-transaction",
    "acid-properties",
    "b-tree",
    "ibm",
    "parallel-computing",
    "synchronization-computing",
    "operating-system",
    "database-replication",
    "distributed-consensus",
    "raft-consensus-algorithm",
    "data-backup",
    "file-system"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}