{
  "id": "iteration",
  "title": "Iteration",
  "lang": "en",
  "source_lang": "en",
  "summary": "Iteration is the repeated execution of a process, used to traverse data, refine approximations, solve computational problems, and develop software incrementally.",
  "category": "computer-science",
  "subcategory": "algorithms-theory",
  "type": "concept",
  "aliases": [
    "Iterative computation",
    "iterative repetition"
  ],
  "generated_at": "2026-11-03",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/iteration/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/iteration/",
      "json": "https://aiwiki.page/zh/iteration.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Iteration is the repeated application of an operation or sequence of operations, usually with each repetition acting on an updated state. In computer science, it is a basic means of expressing computation: a program may process successive records, repeat an instruction until a condition changes, or progressively improve an approximate solution. An individual repetition is also called an iteration. The term therefore denotes both a repeated process and one step within that process.\n\nIteration connects programming practice with mathematics, numerical analysis, and software development. A loop implements repetition operationally; a mathematical recurrence describes how successive states relate; an iterative numerical method seeks a solution through successive approximations. These meanings share the idea of repeated transformation, but they have different criteria for success. A data-processing loop may require exact completion, a numerical method may stop at a specified tolerance, and a development iteration may deliver a revised product rather than a mathematically convergent result."
    },
    {
      "heading": "Mathematical foundations and historical development",
      "text": "A simple deterministic iteration can be written as\n\n\\[x_{k+1}=F(x_k), \\qquad k=0,1,2,\\ldots,\\]\n\nwhere \\(x_0\\) is the initial state and \\(F\\) is an update rule. After \\(k\\) updates, the state is \\(F^k(x_0)\\), where the superscript denotes repeated composition, not multiplication. A function may act on a number, a vector, a collection, or an entire machine state. Algorithms whose updates change with time are more generally written \\(x_{k+1}=F_k(x_k)\\); randomized processes also depend on sampled inputs.\n\nRepeated calculation predates electronic computers by millennia. Old Babylonian mathematical tablets document highly accurate numerical work, including an approximation to \\(\\sqrt{2}\\) on the tablet commonly designated YBC 7289. The tablet records a result rather than an explicit sequence of iterations, so it does not by itself establish the procedure used to obtain that value. A method explicitly described by Hero of Alexandria for square roots repeatedly replaces an estimate \\(x_k\\) of \\(\\sqrt{a}\\) by\n\n\\[x_{k+1}=\\frac{1}{2}\\left(x_k+\\frac{a}{x_k}\\right).\\]\n\nFor positive \\(a\\) and a positive starting estimate, this produces increasingly accurate approximations.\n\nIteration became central to early modern numerical mathematics. Isaac Newton developed methods for approximating roots in the seventeenth century; Joseph Raphson published a related algebraic procedure in 1690. The modern derivative-based formulation associated with their names emerged through subsequent developments. Such methods made it possible to obtain numerical answers when direct symbolic formulas were unavailable or inconvenient.\n\nProgrammable machines turned repetition from a manual calculating procedure into a control mechanism. Designs for Charles Babbage’s Analytical Engine included repeated operations and conditional control. In twentieth-century electronic computing, stored instructions and conditional branches made loops fundamental to practical programming. Fortran, first released in 1957, provided the DO construct for expressing repeated numerical calculations.\n\nThe sequence generated by iteration need not approach a solution. It may reach a fixed state, enter a cycle, grow without bound, or exhibit complicated nonperiodic behavior. Moreover, the machine representation matters: any deterministic iteration confined to a finite set of states must eventually repeat a state if execution continues indefinitely. Its subsequent behavior is then periodic. An ideal mathematical process over the real numbers can behave differently because its state space is not finite."
    },
    {
      "heading": "Loops and control flow in programming",
      "text": "In a programming language, iteration is commonly expressed through a loop: a control structure that repeatedly executes a body of statements. A pre-test loop, such as while, checks its condition before executing the body and may therefore execute zero times. A post-test loop checks after execution and consequently runs at least once when entered normally. A counted loop expresses repetition through a control variable or a specified sequence of values.\n\nA basic summation illustrates these components:\n\ntotal := 0\ni := 0\nwhile i < n:\n    total := total + a[i]\n    i := i + 1\n\nHere initialization establishes the initial state, the condition determines whether another repetition is permitted, the body updates the accumulated result, and the increment advances the index. If \\(n\\) is nonnegative and the array contains at least \\(n\\) elements, the body executes exactly \\(n\\) times. Changing the condition to i <= n would ordinarily attempt to access one element beyond the intended range.\n\nLoop syntax varies across languages. In the C programming language, a for statement groups initialization, a controlling expression, and an update expression in one construct. In JavaScript, for...of consumes values supplied by an iterable object, whereas for...in enumerates enumerable string-keyed properties. These constructs cannot be treated as interchangeable simply because both express repetition.\n\nControl-transfer statements modify the ordinary sequence. A break exits a loop; a continue skips the remainder of the current body and proceeds according to the loop’s rules for beginning another repetition. A function return, exception, or cancellation may also interrupt execution. Their interaction with resource cleanup is part of a language’s semantics: leaving a loop can require closing an iterator, releasing a lock, or executing a cleanup block.\n\nNested loops execute one repeated process within another. Two independent loops each ranging over \\(n\\) values perform \\(n^2\\) inner-body executions. Bounds need not be independent, however. If an inner loop runs \\(i\\) times for outer indices \\(i=1,\\ldots,n\\), the total is \\(n(n+1)/2\\). If the loops operate over two different collections, the relevant count may instead be their size product.\n\nNot all loops are intended to terminate. An operating system service or event-processing program may repeatedly wait for input throughout its useful lifetime. In such cases, correctness concerns include responsiveness, resource use, and appropriate shutdown rather than completion after a predetermined number of repetitions."
    },
    {
      "heading": "Iteration, recursion, and computational expressiveness",
      "text": "Iteration and recursion are alternative ways of organizing repeated computation. An iterative description explicitly advances a state through repeated updates. A recursive description defines a computation in terms of calls to the same function or to mutually dependent functions. Both can express factorial:\n\n\\[n!=\\prod_{i=1}^{n}i, \\qquad n!=n(n-1)!,\\quad 0!=1.\\]\n\nThe first expression suggests an accumulator loop; the second suggests a recursive function with a base case.\n\nFor ordinary sequential algorithms, recursive computation can be simulated iteratively by representing pending calls explicitly. Each record on a stack stores the information needed to resume an unfinished call, such as local values and the next operation. Conversely, a loop can be expressed recursively by passing its updated state into another call. These transformations preserve computational capability, but not necessarily execution cost, memory use, or readability.\n\nA recursive factorial implementation that multiplies after its recursive call returns normally retains one pending multiplication per level. Its call-stack use grows with \\(n\\). An accumulator-based loop can keep only the current index and accumulated product, apart from the storage required for the numerical values themselves. A tail-recursive version passes the accumulated product into its final call and has no remaining calculation after that call returns.\n\nWhether tail recursion uses constant stack space depends on language guarantees and implementation. Some languages require proper tail-call behavior; others may optimize particular cases but provide no general guarantee. Replacing a loop with a tail-recursive function is therefore not automatically a memory-neutral transformation. Conversely, converting a recursive tree traversal into a loop often still requires an explicit stack proportional to the depth of the traversal.\n\nThe distinction is especially visible in functional programming. Operations such as map, filter, and fold express recurring patterns without requiring a programmer to write an explicit loop. A fold combines elements with an accumulating state; a map transforms each element; a filter retains those satisfying a predicate. Their implementations may use loops, recursion, or combinations of the two.\n\nDynamic programming provides another useful comparison. A top-down implementation may use recursion with memoized results, while a bottom-up implementation fills a table iteratively in dependency order. The underlying subproblems can be identical even when control flow differs.\n\nIteration therefore describes an organization of computation, not a distinct level of computational power. The practical choice depends on the shape of the problem, language facilities, stack limits, optimization opportunities, and the clarity with which dependencies and termination conditions can be expressed."
    },
    {
      "heading": "Correctness, invariants, and termination",
      "text": "Reasoning about a loop requires more than checking that its body performs a plausible update. A proof must relate initialization, repeated execution, and the state at exit. A loop invariant is a property that holds before every execution of the loop body. It connects the changing machine state to the intended result.\n\nFor the summation loop, a useful invariant is\n\n\\[0\\le i\\le n \\quad\\text{and}\\quad \\text{total}=\\sum_{j=0}^{i-1}a[j].\\]\n\nInitially \\(i=0\\), so the sum is empty and equals zero. If the invariant holds and \\(i<n\\), adding \\(a[i]\\) and incrementing \\(i\\) preserves it. When the loop condition becomes false, the invariant implies \\(i=n\\), and the accumulated value is the sum of the intended elements.\n\nThis argument establishes partial correctness: if the loop terminates normally, its result satisfies the stated property. Total correctness additionally requires termination. A standard technique identifies a variant, or ranking function, that decreases on each repetition and belongs to a well-founded ordered set. In this example, \\(n-i\\) is a nonnegative integer that decreases by one whenever the body runs. It cannot decrease indefinitely.\n\nThe mathematical model must match the program’s actual operations. A proof over unbounded integers may fail for fixed-width arithmetic if an index overflows. A statement about exact sums may not describe floating-point accumulation, where rounding occurs after individual operations. Mutable shared data, exceptions, and external inputs can also invalidate assumptions that appear obvious in an isolated fragment.\n\nAutomated formal verification uses these ideas to establish loop properties through proof obligations, symbolic reasoning, and abstract interpretation. Invariants can express array bounds, resource ownership, relationships among variables, or permitted protocol states. Discovering a sufficiently strong invariant is often harder than verifying that a proposed invariant is preserved.\n\nTermination is not universally decidable. The halting problem rules out a general algorithm that correctly determines whether every arbitrary program-input pair eventually stops. Practical analyses therefore work with restricted classes, require annotations, or return an inconclusive result for some programs.\n\nFor intentionally persistent loops, a different family of properties becomes important. A server may need to guarantee that every accepted request is eventually handled, subject to scheduling and environmental assumptions. Such a liveness requirement does not assert that the server terminates. It asserts progress within a continuing computation, distinguishing productive repetition from deadlock or endless activity that accomplishes no required work."
    },
    {
      "heading": "Iterators, generators, and data traversal",
      "text": "An iterator is an abstraction for obtaining successive elements of a collection or sequence. It separates the consumer’s repeated processing from the mechanism that locates or produces each element. An array can advance through adjacent positions; a tree traversal may maintain a stack; a database cursor may fetch records from remote storage. Client code can use a similar traversal interface despite these different implementations.\n\nA typical iterator protocol provides an operation that advances the iterator and either yields a value or indicates exhaustion. Some interfaces expose these as separate operations. Others combine them in a result containing a value and a completion indicator. Whether an iterator can move backward, be copied, be restarted, or access an arbitrary position depends on its contract.\n\nAn iterable object is not necessarily itself an iterator. It may instead provide a way to create one. A stored collection can commonly produce multiple independent traversals, whereas a stream-reading iterator may be single-pass and consumable only once. Confusing these cases can cause a program to process no data on a second traversal or unexpectedly repeat expensive work.\n\nA generator defines successive values through a suspended computation. At a yield point, execution pauses while preserving the local state required for resumption. A generator producing successive integers need not construct an entire collection in advance. This supports lazy evaluation and potentially unbounded sequences, provided that consumers request only a finite amount of work at a time.\n\nLazy traversal can reduce intermediate storage. A pipeline that reads records, filters them, transforms selected fields, and accumulates a result may process one record at a time instead of allocating a complete collection after every stage. Laziness also changes when errors and side effects occur: creating a pipeline may perform little work, while requesting the next element triggers file access, parsing, or computation.\n\nMutation during traversal requires explicit rules. Inserting or removing elements may invalidate a cursor, shift positions, or alter which values remain visible. Some collection implementations reject certain concurrent changes; others use snapshots or define limited guarantees. The behavior belongs to the collection’s contract rather than to iteration in general.\n\nAsynchronous iteration extends traversal to values that become available later. In asynchronous programming, advancing an iterator may suspend until another network message or input chunk arrives. This is distinct from processing all elements concurrently. A consumer can await each value sequentially, thereby limiting outstanding work, although actual buffering and backpressure depend on the surrounding interfaces."
    },
    {
      "heading": "Numerical iteration and convergence",
      "text": "An iterative method constructs a sequence of approximations to a mathematical solution. Unlike a loop over a fixed collection, its required number of repetitions may not be known in advance. The central questions are whether the sequence converges, which solution it approaches, how quickly the error decreases, and how finite-precision arithmetic affects the process.\n\nFor fixed-point iteration,\n\n\\[x_{k+1}=g(x_k),\\]\n\na desired limit \\(x^\\ast\\) satisfies \\(g(x^\\ast)=x^\\ast\\). A major sufficient condition comes from the Banach fixed-point theorem: a contraction mapping from a nonempty complete metric space into itself has a unique fixed point, and repeated application converges to it from any starting point in that space. If its contraction factor is \\(q<1\\), the error obeys\n\n\\[d(x_k,x^\\ast)\\le q^k d(x_0,x^\\ast).\\]\n\nThis provides both a convergence guarantee and a geometric error bound under the theorem’s assumptions.\n\nNewton’s method instead uses local derivative information to solve \\(f(x)=0\\):\n\n\\[x_{k+1}=x_k-\\frac{f(x_k)}{f'(x_k)}.\\]\n\nNear a simple root, with sufficient smoothness and an appropriate initial estimate, it has quadratic convergence: the new error is approximately proportional to the square of the previous error. This strong local behavior is not a global guarantee. A zero derivative, a poor starting point, or an unsuitable function can cause failure, divergence, or cycling.\n\nLarge linear systems provide another major application. For \\(Ax=b\\), a stationary iteration often has the form\n\n\\[x_{k+1}=Bx_k+c.\\]\n\nFor a consistent fixed-point formulation, convergence from every starting vector occurs when the spectral radius of \\(B\\), the largest absolute value of its eigenvalues, is less than one. Jacobi and Gauss–Seidel methods use different decompositions of the coefficient matrix. Krylov-subspace methods, including conjugate gradients for symmetric positive-definite systems, construct approximations in expanding subspaces rather than applying one fixed update matrix.\n\nA stopping rule commonly tests a residual, a change between successive iterates, or both. The residual \\(r_k=b-Ax_k\\) measures how well the current estimate satisfies the equations. It is not identical to the error \\(x_k-x^\\ast\\): an ill-conditioned problem can have a small residual and still have a substantial solution error.\n\nFinite precision can eventually prevent further improvement. Updates may become too small to alter stored values, while rounding errors can accumulate or disrupt theoretical relationships. Practical algorithms therefore combine tolerances with iteration limits and checks for breakdown. A maximum iteration count bounds work; reaching it does not by itself establish that the requested accuracy has been achieved."
    },
    {
      "heading": "Optimization and machine learning",
      "text": "Iteration is a principal mechanism of mathematical optimization, where the objective is to find a state minimizing or maximizing a function. Gradient descent updates a parameter vector by\n\n\\[\\theta_{k+1}=\\theta_k-\\eta_k\\nabla L(\\theta_k),\\]\n\nwhere \\(L\\) is the objective and \\(\\eta_k\\) is the step size. The negative gradient gives a direction of steepest local decrease under the Euclidean norm, but the amount moved determines whether the update is useful or overshoots.\n\nConvergence depends on assumptions about the objective and step sizes. For a differentiable convex objective with a Lipschitz-continuous gradient, suitable step sizes support standard convergence guarantees. Strong convexity permits stronger conclusions. For nonconvex objectives, reducing the gradient norm does not generally establish that a global minimum has been found; stationary points can have different mathematical character.\n\nIn machine learning, the objective is often an average loss function over examples. Computing its full gradient can be expensive, so stochastic gradient descent and mini-batch variants use subsets of the training data. The resulting updates contain sampling noise. This can make progress less smooth than in deterministic optimization, and convergence analysis must account for how samples and step sizes are chosen.\n\nAn iteration, an optimization step, and an epoch are related but distinct units. An optimization step ordinarily means one parameter update. An epoch conventionally means one pass through a finite training dataset. With 10,000 examples and batches of 100, a complete pass requires 100 updates if every example is used once and each batch produces one update. Gradient accumulation, repeated sampling, dropped final batches, and distributed execution can alter that relationship.\n\nFor an artificial neural network, a training step commonly includes a forward evaluation, loss calculation, gradient computation through backpropagation, and an optimizer update. Adam additionally maintains moving averages of gradients and squared gradients. These auxiliary values are part of the iterative state, so restarting from model weights alone does not reproduce the same optimizer trajectory.\n\nTraining loss and predictive performance need not improve together. Continued optimization can produce overfitting, in which performance on training examples improves while performance on unseen examples deteriorates. Early stopping uses a specified monitoring criterion, commonly based on a validation set, to end training before a fixed computational budget is exhausted.\n\nIteration counts consequently provide an incomplete measure of training effort. A step involving a larger model, a larger batch, or longer input sequences can require much more computation. Comparisons also depend on data exposure, numerical precision, optimizer state, and the stopping criterion, not merely on how many updates occurred."
    },
    {
      "heading": "Performance, parallelism, and distributed execution",
      "text": "The cost of an iterative algorithm depends on both its number of repetitions and the work within each repetition. If \\(K\\) iterations each cost \\(C\\), a first approximation is \\(KC\\). More generally, total work is\n\n\\[\\sum_{k=0}^{K-1}C_k,\\]\n\nbecause the cost may change as a search frontier expands, a candidate set shrinks, or an approximation becomes more demanding. Computational complexity studies how such costs scale with input size and other relevant parameters.\n\nNested syntax alone does not determine asymptotic complexity. Repeatedly halving a search interval gives logarithmically many steps, as in binary search. A loop that occasionally performs an expensive operation can still have low amortized cost. Conversely, a single loop statement may call a procedure that performs substantial hidden work.\n\nA compiler can transform loops through unrolling, vectorization, fusion, interchange, or other optimizations. Unrolling reduces the relative frequency of control instructions and can expose independent operations. Fusion combines traversals and may improve locality by reusing data while it remains nearby in the memory hierarchy. Transformations must preserve the behavior required by the language and its compilation rules.\n\nDependencies determine which repetitions can execute simultaneously. If each iteration writes to a distinct output element and reads only unchanged input, iterations may be independent. A recurrence such as \\(x_{k+1}=F(x_k)\\) has an explicit dependence on the preceding state. A reduction, such as summation, can often be reorganized into a tree, but floating-point addition is not associative, so a parallel reduction may produce a different rounded result from a sequential one.\n\nParallel computing frequently accelerates each numerical iteration rather than eliminating the sequence of iterations. A matrix operation may be divided among many processors, followed by synchronization before the next update. The benefit depends on whether arithmetic work outweighs scheduling, communication, and data-transfer costs.\n\nIn distributed computing, synchronization introduces additional constraints. A synchronous algorithm may wait for every worker before advancing. An asynchronous algorithm may proceed using delayed information, reducing waiting while changing the mathematical update process. Its convergence can require assumptions about delay bounds, update frequency, and consistency.\n\nFailures also affect iterative state. A checkpoint may preserve parameters, counters, random-number-generator state, and auxiliary variables so that computation can resume. Repeating an interrupted iteration is safe only when its effects are appropriately controlled. An update that writes an external record or charges an account cannot be treated like a purely internal numerical calculation: recovery must distinguish work that never occurred from work that occurred but was not acknowledged."
    },
    {
      "heading": "Iterative software development and feedback cycles",
      "text": "In software engineering, iteration also describes repeated cycles of development activity. A team implements or revises part of a system, evaluates the result, and uses the findings to determine subsequent work. The state being updated includes requirements, designs, code, tests, and understanding of user needs rather than only numerical variables.\n\nIterative development is distinct from incremental development. An increment adds usable capability; an iteration revisits or refines a result. A project may do both by delivering a small feature, evaluating it, and revising it in a later cycle. It can also be incremental without substantially revisiting earlier decisions, or iterative without producing a newly deployable capability on every cycle.\n\nHistorically influential methods organized these cycles differently. Barry Boehm’s spiral model, described in a 1988 paper, structured development around repeated consideration of objectives, alternatives, risks, engineering work, and planning. Its defining emphasis was risk-driven activity, not merely repeating the same fixed sequence. Scrum organizes work into time-bounded Sprints; the 2020 Scrum Guide specifies that these last one month or less. Such a Sprint is a management and development interval, not an execution count within an algorithm.\n\nFeedback can come from automated tests, performance measurements, security analysis, operational incidents, or observation of actual use. These forms of evidence answer different questions. Passing a regression test shows consistency with a specified expected behavior; it does not establish that the requirement is useful. Faster execution does not establish correctness. Repeated revision is informative only insofar as the evaluation exposes relevant differences between the current system and its intended behavior.\n\nVersion control records successive changes and supports comparison, reversal, and collaboration. Automated build and test systems evaluate revisions, while continuous delivery organizes software so that release remains feasible through frequent integration and validation. A development iteration, a build, a commit, and a production release are therefore different events, even when a particular workflow closely connects them.\n\nIteration in development lacks the general convergence guarantees available for some numerical methods. Requirements may change, evaluation criteria may conflict, and revisions can introduce new defects. Repeating a process does not establish that each result is better than its predecessor. Meaningful assessment depends on explicit objectives and observable properties, such as defect rates under a defined measurement procedure, response times under a stated workload, or completion of specified acceptance tests."
    }
  ],
  "links": [
    "computer-science",
    "mathematics",
    "mathematical-function",
    "charles-babbage",
    "fortran",
    "programming-language",
    "c-programming-language",
    "javascript",
    "operating-system",
    "recursion",
    "functional-programming",
    "dynamic-programming",
    "loop-invariant",
    "formal-verification",
    "halting-problem",
    "asynchronous-programming",
    "banach-fixed-point-theorem",
    "newtons-method",
    "matrix",
    "mathematical-optimization",
    "gradient-descent",
    "machine-learning",
    "loss-function",
    "stochastic-gradient-descent",
    "training-data",
    "artificial-neural-network",
    "backpropagation",
    "adam-optimizer",
    "overfitting",
    "early-stopping",
    "validation-set",
    "computational-complexity",
    "binary-search",
    "compiler",
    "memory-hierarchy",
    "parallel-computing",
    "distributed-computing",
    "software-engineering",
    "version-control"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}