{
  "id": "optimal-control",
  "title": "Optimal Control",
  "lang": "en",
  "source_lang": "en",
  "summary": "Optimal control determines how a dynamical system should be influenced over time to minimize a cost or maximize a specified performance criterion.",
  "category": "mathematics",
  "subcategory": "applied",
  "type": "concept",
  "aliases": [
    "Optimal control theory"
  ],
  "generated_at": "2026-11-03",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/optimal-control/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/optimal-control/",
      "json": "https://aiwiki.page/zh/optimal-control.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Optimal control is the branch of mathematical optimization concerned with choosing inputs to a dynamical system so that its evolution best satisfies a specified objective. The inputs, called controls, may represent forces, voltages, production rates, investment decisions, or other quantities that can be adjusted over time. Unlike a static optimization problem, an optimal control problem links present decisions to future possibilities through equations of motion, resource balances, or transition probabilities.\n\nThe subject supplies a mathematical language for problems such as steering a spacecraft with limited fuel, regulating an industrial process while respecting safety limits, and operating an energy-storage system under changing demand. Its central methods connect the calculus of variations, differential equations, optimization, and probability. Two foundational approaches are the Pontryagin maximum principle, which characterizes optimal trajectories through auxiliary differential equations, and dynamic programming, which characterizes optimal decisions through a value function.\n\nAn optimal solution is always relative to a model, an objective, constraints, and available information. Minimizing travel time need not minimize energy consumption, and a control that is optimal for a perfectly known model need not perform well under disturbances. Consequently, optimal control encompasses not only analytical solutions but also questions of existence, numerical approximation, feedback, uncertainty, and implementability."
    },
    {
      "heading": "Mathematical formulation and information structure",
      "text": "A standard continuous-time problem describes a state \\(x(t)\\in\\mathbb{R}^n\\) and a control \\(u(t)\\in U\\subseteq\\mathbb{R}^m\\) over a time interval \\([t_0,T]\\). The state obeys an ordinary differential equation,\n\n\\[\\dot{x}(t)=f(t,x(t),u(t)),\\qquad x(t_0)=x_0.\\]\n\nThe objective is commonly written in Bolza form:\n\n\\[J(u)=\\phi(x(T))+\\int_{t_0}^{T}L(t,x(t),u(t))\\,dt.\\]\n\nHere \\(L\\) is a running cost and \\(\\phi\\) a terminal cost. The problem is to minimize \\(J\\) over admissible controls. Maximization problems can be converted into minimization problems by changing the sign of the objective. Constraints may restrict the control, require \\(g(t,x(t),u(t))\\leq0\\), or impose a terminal condition such as \\(x(T)\\in X_f\\).\n\nThe state must contain enough information to determine future evolution once future controls are specified. In a vehicle model it might include position, velocity, and fuel mass; in an inventory model it might be stock on hand. A control is a decision variable, whereas a disturbance is an external influence that is not chosen by the controller. This distinction matters even when the two enter the same equation. The objective function also requires careful interpretation: a weighted sum of energy, time, and tracking error implicitly specifies trade-offs, and changing the units or weights can change the resulting policy.\n\nAn open-loop control is specified as a function of time, usually after the initial state has been given. A feedback control instead takes the form \\(u(t)=\\pi(t,x(t))\\), allowing decisions to change when the observed state changes. For a deterministic model with a known initial state, a feedback policy generates a particular open-loop trajectory. The distinction becomes essential when disturbances, uncertain initial conditions, or measurement errors are present. A precomputed trajectory does not by itself prescribe how to recover after a deviation.\n\nDiscrete-time problems replace the differential equation by\n\n\\[x_{k+1}=F_k(x_k,u_k),\\qquad J=\\phi(x_N)+\\sum_{k=0}^{N-1}L_k(x_k,u_k).\\]\n\nThey arise both intrinsically, as in annual planning, and through numerical discretization of continuous-time models. The horizon may be fixed, free, or infinite. Free terminal time introduces an additional decision variable; infinite-horizon formulations generally require discounting, stability conditions, or long-run average criteria to make the objective meaningful. Problems with only an integral cost are called Lagrange problems, while those with only a terminal cost are Mayer problems. Introducing an accumulated-cost state converts an integral objective into a terminal one."
    },
    {
      "heading": "Historical development",
      "text": "Optimal control grew from problems in mechanics and the calculus of variations, but its modern formulation emerged when explicit controls, constrained inputs, and feedback became central. In 1696 Johann Bernoulli posed the brachistochrone problem: finding the curve along which a particle slides between two points in the least time under gravity. Eighteenth-century work by Leonhard Euler and Joseph-Louis Lagrange developed systematic necessary conditions for optimizing integral functionals. The resulting Euler–Lagrange equations became a principal foundation for later trajectory optimization.\n\nClassical variational methods typically treat a path and its derivatives as the primary unknowns. Control problems often differ because the rate of change of the state is constrained by a separate input, and that input may have hard bounds. A rocket engine, for example, cannot supply an arbitrary thrust merely because a variational calculation requests it. Constantin Carathéodory’s work on variational methods and differential equations helped connect local differential conditions with broader optimization principles. During the twentieth century, engineering developments in servomechanisms, aviation, and communications sharpened the distinction between choosing an ideal trajectory and building a controller capable of maintaining it.\n\nTwo major approaches took shape in the 1950s. Richard Bellman developed dynamic programming and published Dynamic Programming in 1957. Its principle of optimality expressed a recursive property: after an optimal decision has led to a new state, the remaining decisions must solve the corresponding continuation problem. In parallel, Lev Pontryagin and collaborators in the Soviet Union developed the maximum principle. Their monograph The Mathematical Theory of Optimal Processes appeared in Russian in 1961 and in English in 1962. The theory directly accommodated bounded controls and helped explain switching behavior in time- and fuel-optimal systems.\n\nAround 1960, Rudolf Kalman’s work established influential connections among linear systems, quadratic optimization, state estimation, and feedback. These developments supported a systematic theory in which optimal gains could be computed from matrix equations rather than guessed from trial and error. Subsequent research extended the subject to stochastic dynamics, distributed systems, nonsmooth value functions, and constrained numerical optimization. From the late 1970s onward, receding-horizon methods became increasingly important in process industries. Later improvements in computing, sparse optimization, and derivative calculation made repeated online solution practical for many mechanical and energy applications. These developments supplemented rather than displaced the classical theory: modern software still relies heavily on its differential equations, multipliers, and recursive structure."
    },
    {
      "heading": "Existence, admissibility, and the meaning of an optimum",
      "text": "Before deriving an optimality condition, a problem must specify what counts as a permissible control and whether a minimizer actually exists. Controls are often taken to be measurable functions, while the corresponding state trajectory is absolutely continuous and satisfies the dynamics almost everywhere. Conditions such as continuity in the control and local Lipschitz continuity in the state help ensure that the state equation has a well-defined solution for a given control. Integrability and growth assumptions are also needed to prevent finite-time blow-up or an undefined objective.\n\nA finite infimum does not guarantee an optimal control. Consider\n\n\\[\\dot{x}=u,\\quad x(0)=0,\\quad x(1)=1,\\qquad J(u)=\\int_0^1x(t)^2\\,dt,\\]\n\nwith no common bound on control amplitude. A control can keep \\(x=0\\) until time \\(1-\\varepsilon\\), then use \\(u=1/\\varepsilon\\) to reach \\(x(1)=1\\). Its cost is \\(\\varepsilon/3\\), which tends to zero. But an absolutely continuous trajectory with zero cost must be zero everywhere, contradicting the terminal condition. The infimum is therefore zero and is not attained. An amplitude bound rules out this particular concentration mechanism.\n\nExistence theorems typically combine compactness of admissible trajectories with lower semicontinuity of the cost and suitable closure properties of the dynamics. A compact control set is helpful, but it is not sufficient by itself in every nonlinear problem. Convexity of the attainable velocity set, or an appropriate velocity–cost set, often supplies the missing closure. With unbounded controls, coercive costs can provide compactness by penalizing large control magnitudes. The relevant mathematical setting may involve functional analysis, weak convergence, and spaces of integrable functions rather than ordinary finite-dimensional compactness.\n\nOne way to address nonconvexity is to allow relaxed controls: instead of choosing one control value at each instant, a relaxed control chooses a probability distribution over control values. The effective velocity is an average of the velocities generated by those controls. Such formulations can possess minimizers even when an ordinary control does not, and rapidly switching ordinary controls may approximate the relaxed solution under suitable assumptions. This construction is mathematical; a physical actuator may be unable to execute arbitrarily fast switching.\n\nExistence is also separate from uniqueness and global optimality. A nonlinear problem can have several local minima, and a trajectory satisfying necessary conditions may be a saddle point or even a maximizer. By contrast, linear dynamics together with convex costs and convex constraints can produce a convex optimization problem. In that setting, local minima are global, and strict convexity may imply uniqueness of the control up to distinctions on sets of measure zero. These structural properties determine how much can be concluded from either an analytical derivation or a numerical result."
    },
    {
      "heading": "The Pontryagin maximum principle",
      "text": "The Pontryagin maximum principle converts a trajectory optimization problem into conditions involving the state and an auxiliary variable called the costate. For a normal minimization problem, a convenient convention defines the Hamiltonian\n\n\\[H(t,x,u,p)=L(t,x,u)+p^\\mathsf{T}f(t,x,u).\\]\n\nUnder appropriate regularity assumptions, an optimal control \\(u^*\\), state \\(x^*\\), and costate \\(p\\) satisfy\n\n\\[\\dot{x}^*=H_p,\\qquad \\dot{p}=-H_x,\\qquad u^*(t)\\in\\operatorname*{arg\\,min}_{u\\in U} H(t,x^*(t),u,p(t))\\]\n\nalmost everywhere. Although this convention uses minimization, it expresses the same principle as the customary maximum formulation obtained by changing signs.\n\nThe costate measures the marginal effect of changing the state on the optimized objective, subject to qualifications concerning differentiability and normality. Its equation propagates information about future costs backward through the dynamics. If the final state is unconstrained and the terminal time is fixed, then\n\n\\[p(T)=\\nabla\\phi(x^*(T)).\\]\n\nTerminal constraints introduce additional multipliers. A free terminal time yields a further transversality condition; in the simple autonomous case without explicit terminal-time cost, this includes \\(H(T)=0\\). Thus the state has initial data while the costate often has terminal data, producing a two-point boundary-value problem rather than an ordinary initial-value simulation.\n\nThe general principle includes a scalar multiplier \\(\\lambda_0\\geq0\\), with\n\n\\[H=\\lambda_0L+p^\\mathsf{T}f,\\]\n\nand requires the multipliers not to vanish simultaneously. Normal extremals have \\(\\lambda_0>0\\), which can be normalized to one. Abnormal extremals have \\(\\lambda_0=0\\): their first-order conditions are governed by the constraints and dynamics without an explicit contribution from the objective. Omitting this possibility can lose legitimate candidates. Likewise, replacing Hamiltonian minimization by the equation \\(H_u=0\\) is valid only for appropriate interior, differentiable cases; it fails for controls whose optimum lies on a bound.\n\nThe principle gives necessary conditions, not a universal certificate of global optimality. For an affine system with a convex running cost and convex terminal cost, suitable Hamiltonian inequalities can establish sufficiency. For general nonlinear systems, different extremals must be compared, and second-order conditions may be needed to distinguish local minima from other stationary solutions. The costate also connects the principle to Lagrangian duality: it acts as a time-dependent multiplier enforcing the state equation. Numerically computed multipliers can therefore indicate which state components or constraints have the greatest marginal influence on the objective."
    },
    {
      "heading": "Dynamic programming and the Hamilton–Jacobi–Bellman equation",
      "text": "Dynamic programming studies the optimal cost of continuing from every relevant state, rather than only a single candidate trajectory. Its central object is the value function\n\n\\[V(t,x)=\\inf_u\\left\\{ \\int_t^T L(s,x(s),u(s))\\,ds+\\phi(x(T)) \\right\\},\\]\n\nwhere \\(x(t)=x\\). The principle of optimality states that, for a short interval of length \\(h\\),\n\n\\[V(t,x)=\\inf_u\\left\\{ \\int_t^{t+h}L(s,x(s),u(s))\\,ds+ V(t+h,x(t+h)) \\right\\}.\\]\n\nThe controls on the short interval determine the state at which the continuation problem begins. This decomposition is the essential reason a multi-stage decision problem can be solved recursively.\n\nIf the value function is sufficiently smooth, expanding the recursion gives the Hamilton–Jacobi–Bellman equation:\n\n\\[-V_t(t,x)= \\min_{u\\in U} \\left\\{L(t,x,u)+\\nabla_xV(t,x)^\\mathsf{T}f(t,x,u)\\right\\}, \\qquad V(T,x)=\\phi(x).\\]\n\nA minimizing selector provides a feedback law. A verification theorem then supplies sufficient conditions: if a suitably regular function solves the equation and an admissible policy attains its minimum, that policy is optimal. This is stronger than merely identifying an extremal, although the required regularity and boundary conditions cannot be ignored.\n\nValue functions often fail to be differentiable. Two distinct routes to a target may have equal cost along a switching surface, creating a corner even when the system dynamics are smooth. Viscosity solutions provide a generalized interpretation of first-order nonlinear partial differential equations that handles such nonsmoothness. Rather than requiring classical derivatives everywhere, the definition uses smooth test functions touching the candidate value function from above or below. Comparison principles can then establish uniqueness in the appropriate class. State constraints and exit-time problems require carefully formulated boundary behavior, not an arbitrary boundary value on every part of the domain.\n\nWhere the value function is differentiable along an optimal trajectory, the costate in the minimum-Hamiltonian convention is typically \\(p(t)=\\nabla_xV(t,x^*(t))\\). The maximum principle and dynamic programming are therefore related descriptions of the same optimization structure: one follows characteristics along a trajectory, while the other describes optimal costs across state space. Their computational burdens differ sharply. A grid with \\(M\\) points in each of \\(n\\) state coordinates has \\(M^n\\) points, illustrating the curse of dimensionality. Dynamic programming is exceptionally powerful in low-dimensional problems and specially structured models, but direct gridding becomes impractical for many large systems."
    },
    {
      "heading": "Linear–quadratic control and optimal feedback",
      "text": "The linear–quadratic regulator is a central exactly solvable class. For\n\n\\[\\dot{x}=Ax+Bu,\\]\n\nconsider the objective\n\n\\[J=\\frac12x(T)^\\mathsf{T}Sx(T) +\\frac12\\int_0^T \\left(x^\\mathsf{T}Qx+u^\\mathsf{T}Ru\\right)\\,dt,\\]\n\nwhere \\(Q\\) and \\(S\\) are symmetric positive semidefinite and \\(R\\) is symmetric positive definite. The state penalty discourages deviation from the desired equilibrium, while the control penalty discourages excessive actuation. The problem has no control or state bounds in its standard form. Its importance lies partly in its explicit feedback solution and partly in its role as a local approximation to nonlinear problems.\n\nSubstituting a quadratic value function,\n\n\\[V(t,x)=\\frac12x^\\mathsf{T}P(t)x,\\]\n\ninto the Hamilton–Jacobi–Bellman equation yields\n\n\\[u^*(t)=-R^{-1}B^\\mathsf{T}P(t)x(t)\\]\n\nand the Riccati differential equation\n\n\\[-\\dot{P} =A^\\mathsf{T}P+PA-PBR^{-1}B^\\mathsf{T}P+Q, \\qquad P(T)=S.\\]\n\nThe optimal gain is time-dependent on a finite horizon. For an infinite horizon with constant system matrices, a stabilizing steady solution may satisfy the algebraic Riccati equation obtained by setting \\(\\dot P=0\\). Stabilizability of \\((A,B)\\) and detectability of the state penalty are standard conditions supporting this solution. They exclude, respectively, uncontrollable unstable behavior and unstable behavior invisible to the objective.\n\nA scalar example makes the trade-off explicit. For \\(\\dot{x}=ax+bu\\) with positive weights \\(q,r\\), the infinite-horizon Riccati equation is\n\n\\[0=2aP-\\frac{b^2}{r}P^2+q.\\]\n\nIf \\(b\\neq0\\), its stabilizing solution is\n\n\\[P=\\frac{r}{b^2} \\left(a+\\sqrt{a^2+\\frac{b^2q}{r}}\\right).\\]\n\nThe feedback \\(u=-(bP/r)x\\) makes the closed-loop coefficient equal to\n\\(-\\sqrt{a^2+b^2q/r}\\). Increasing the state penalty relative to the control penalty generally demands stronger corrective action. The formula also shows why the appropriate Riccati solution matters: an algebraic equation can possess more than one root, but not every root produces a stabilizing controller.\n\nWhen the state is not directly observed, linear–quadratic–Gaussian control combines this regulator with a Kalman filter. Under the standard linear model with Gaussian process and observation noise, quadratic cost, and no additional constraints, the separation principle allows the estimation and feedback gains to be designed separately. The controller applies the full-state optimal gain to the conditional state estimate. This result does not mean that estimation is irrelevant, nor that arbitrary nonlinear or constrained problems separate in the same way. Even in the linear setting, actuator saturation changes the optimization problem: simply clipping an unconstrained optimal input generally does not preserve global optimality."
    },
    {
      "heading": "Constraints, switching, and singular behavior",
      "text": "Constraints frequently determine the qualitative form of an optimal control. If the Hamiltonian is affine in a scalar bounded control, it can be written\n\n\\[H=H_0+\\sigma(t)u,\\qquad u_{\\min}\\leq u\\leq u_{\\max},\\]\n\nwhere \\(\\sigma(t)\\) is the switching function. Under the minimum convention, the minimizing control is \\(u_{\\min}\\) when \\(\\sigma>0\\) and \\(u_{\\max}\\) when \\(\\sigma<0\\). Such bang-bang control uses actuator extremes rather than intermediate settings. It commonly appears in minimum-time problems because, without a penalty on effort, there is no reason to move more slowly than necessary except to satisfy future constraints.\n\nFor the minimum-time double integrator,\n\n\\[\\dot{x}=v,\\qquad \\dot v=u,\\qquad |u|\\leq a,\\]\n\nthe cost is elapsed time, so the normal Hamiltonian is\n\n\\[H=1+p_xv+p_vu.\\]\n\nThe costate equations give \\(\\dot p_x=0\\) and \\(\\dot p_v=-p_x\\). Thus the switching function \\(p_v\\) is affine in time and, for a nontrivial normal transfer, has at most one zero. To move a distance \\(d>0\\) from rest to rest, the optimal input is \\(+a\\) followed by \\(-a\\), with equal acceleration and braking durations. The total time is\n\n\\[T=2\\sqrt{\\frac{d}{a}}.\\]\n\nThis simple result depends on the absence of a speed limit, drag, or further path constraints.\n\nA vanishing switching function at an isolated instant merely marks a possible switch. If it vanishes throughout an interval, Hamiltonian minimization may fail to determine the input directly. Such an interval is called a singular arc. Differentiating the switching function along the dynamics can reveal the control at a higher derivative order, but additional optimality conditions are needed to determine whether the resulting arc is minimizing. Some problems also exhibit chattering: infinitely many switches accumulate in finite time. This is a mathematical property distinct from numerical oscillation, although coarse discretizations can make the two difficult to distinguish.\n\nState constraints introduce further structure. A bound such as \\(v(t)\\leq v_{\\max}\\) can force an optimal trajectory to remain on a boundary arc. Remaining on that boundary requires tangency conditions, such as \\(\\dot v=0\\), until departure becomes optimal or necessary. General maximum-principle formulations may require measure-valued multipliers and corresponding jumps in the costate at constraint junctions. Integral constraints, such as a fixed fuel budget, can often be represented by an additional state tracking cumulative consumption. Adding a large penalty for constraint violation is not always equivalent to imposing the constraint itself: a finite penalty can still make some violation optimal, and very large penalties can make numerical computations poorly conditioned."
    },
    {
      "heading": "Numerical solution and receding-horizon implementation",
      "text": "Analytical solutions are exceptional, so most realistic problems are solved numerically. Indirect methods first derive the maximum-principle conditions and then solve the resulting state–costate boundary-value problem. A shooting method guesses missing initial costates, integrates the equations, and adjusts the guesses to satisfy terminal conditions. It can be highly accurate when initialized well, but the shooting map may be sensitive to small changes in the guess. Unstable dynamics, long horizons, and unknown switching times can make convergence difficult. Multiple shooting reduces some of this sensitivity by dividing the trajectory into intervals and enforcing matching conditions between them.\n\nDirect methods instead parameterize controls, states, or both, and transcribe the problem into a finite-dimensional optimization problem. Direct single shooting optimizes control parameters while obtaining states by integration. Direct collocation introduces state values at selected nodes and imposes discretized dynamics as equality constraints. Polynomial and pseudospectral methods can achieve high accuracy for smooth solutions, while adaptive meshes allocate more points near switches or sharp transitions. These methods connect optimal control to numerical analysis, sparse nonlinear programming, and the Karush–Kuhn–Tucker conditions. Under suitable discretizations, optimization multipliers approximate continuous-time costates.\n\nNumerical success requires more than obtaining a solver’s success flag. A computed trajectory must be checked for dynamic residuals, terminal errors, path-constraint violations, and sensitivity to mesh refinement. Constraints enforced only at nodes can be violated between nodes. A local nonlinear solver ordinarily supplies no global optimality guarantee, and different initial guesses can converge to different solutions. Scaling states and constraints to comparable magnitudes can improve conditioning. Automatic differentiation supplies accurate derivatives of discretized models, while sparse linear algebra exploits the predominantly local coupling between neighboring time steps.\n\nModel predictive control turns repeated trajectory optimization into feedback. At each sampling instant, it estimates the current state, solves a finite-horizon problem, applies only the first control segment, and then solves again using new measurements. This arrangement naturally handles changing references and explicit actuator or state constraints. For linear dynamics with quadratic costs and linear constraints, each optimization is often a convex quadratic program; nonlinear models generally require more expensive nonlinear programming.\n\nA finite prediction horizon does not automatically guarantee stability or continued feasibility. Terminal costs, terminal sets, and invariant-set constructions can provide such guarantees under specified assumptions. Robust or tube-based formulations account explicitly for bounded disturbances rather than assuming the nominal predicted trajectory will occur exactly. In real-time applications, the numerical algorithm must also meet a deadline: a more accurate solution arriving after the control action is needed may be less useful than a timely feasible approximation."
    },
    {
      "heading": "Stochastic control, partial observation, and learning",
      "text": "In stochastic optimal control, the dynamics contain random influences and the objective commonly minimizes an expected value. A continuous-time diffusion model is\n\n\\[dX_t=f(t,X_t,u_t)\\,dt+\\sigma(t,X_t,u_t)\\,dW_t,\\]\n\nwhere \\(W_t\\) is a Wiener process. Controls must be nonanticipating: they may depend on information available at the decision time but not on future noise. This restriction is expressed through a filtration and suitable measurability conditions. Without it, a mathematical optimization could exploit information no physical decision-maker possesses.\n\nFor a fully observed diffusion, the smooth stochastic Hamilton–Jacobi–Bellman equation becomes\n\n\\[-V_t=\\inf_u\\left\\{ L+f^\\mathsf{T}\\nabla V+ \\frac12\\operatorname{tr}\\!\\left(\\sigma\\sigma^\\mathsf{T}D^2V\\right) \\right\\}.\\]\n\nThe second-order term reflects the effect of diffusion on expected future cost. State-dependent or control-dependent noise can therefore change the optimal action even when its mean is zero. In discrete time, the corresponding recursion is a Bellman equation involving a conditional expectation over the next state. Finite-state models lead naturally to the Markov decision process, with algorithms such as value iteration and policy iteration.\n\nWhen observations are incomplete, the controller generally must act on an information state rather than the physical state itself. A common information state is the conditional probability distribution of the hidden state given past observations and controls. The resulting belief-state problem connects optimal control with the partially observable Markov decision process. Even when the hidden physical state is finite-dimensional, its probability distribution can be an infinite-dimensional object. Linear–Gaussian estimation is a special tractable case because the conditional distribution remains Gaussian and is determined by a mean and covariance.\n\nMinimizing expected cost is only one treatment of uncertainty. Risk-sensitive objectives may penalize the distribution of losses, while chance constraints limit the probability of specified violations. Robust control instead often seeks performance against all disturbances in a prescribed set. A worst-case formulation can become a differential game, linking the subject to game theory. These approaches encode different attitudes toward rare events and model error; none is interchangeable with the others without changing the problem.\n\nReinforcement learning overlaps with optimal control when transition laws or cost functions are learned from interaction rather than fully specified in advance. The Bellman framework remains relevant, but data introduce approximation error, exploration requirements, and uncertainty about unvisited states. In an unknown system, an action may both influence the physical state and reveal information useful for later decisions, a phenomenon called dual control. A policy learned from data is therefore not automatically optimal for the physical system, especially when operating conditions differ from those represented in the data."
    },
    {
      "heading": "Applications, model limitations, and continuing research",
      "text": "Aerospace engineering supplies characteristic optimal control problems: minimum-fuel orbit transfer, launch ascent, atmospheric entry, and spacecraft attitude maneuvering. The state can combine position, velocity, orientation, and propellant mass, while constraints include thrust limits, heating, and allowable accelerations. Different propulsion models lead to markedly different solutions. A high-thrust maneuver may be approximated by short impulses, whereas low-thrust electric propulsion requires sustained control over long intervals. Free terminal time and moving target states couple trajectory design to the timing of arrival, and gravitational models can produce many competing local solutions.\n\nRobotics and mechanical systems use trajectory optimization to coordinate motion under actuation, collision, and contact constraints. Walking introduces hybrid dynamics: continuous motion is interrupted by impacts or changes in contact mode. Optimizing over those modes is more difficult than optimizing a smooth trajectory with a fixed sequence of events. In industrial processes and energy systems, the principal issue is often constrained economic operation rather than minimum-time movement. Storage dispatch, for example, balances charging losses, capacity, power limits, and future prices. A terminal valuation of remaining stored energy is crucial; without it, a finite-horizon calculation can empty a store merely because costs beyond the horizon are absent from the model.\n\nIn economics, a standard example is choosing consumption and investment over time. A resource or capital stock evolves dynamically, and the costate represents the marginal value of an additional unit of that stock. Discounting changes both the relative weight of future outcomes and the form of the costate equations when current-value variables are used. Similar mathematical structures occur in resource management and production planning, but the interpretation of the objective remains domain-specific. An optimum based on a single aggregate criterion does not by itself settle how competing benefits, costs, or distributional effects should be valued.\n\nA recurring limitation is the difference between mathematical optimality and performance outside the model. Unmodeled actuator delays, inaccurate friction, uncertain demand, and imperfect observations can change the ranking of candidate policies. Sensitivity analysis examines how solutions change with parameters, while robust and adaptive approaches address specified forms of uncertainty. There is also a distinction between finding a feasible trajectory and certifying that every implemented trajectory remains safe. Tightening constraints can provide margins, but guarantees depend on the accuracy of the assumed disturbance bounds and the implementation.\n\nContinuing research addresses high-dimensional value-function approximation, efficient nonlinear receding-horizon computation, control of systems governed by partial differential equations, and optimization with discrete events or contact. Data-driven models create an additional challenge: uncertainty in the learned dynamics must be propagated into performance and constraint assessments. Computation and communication can themselves become decision resources, as in event-triggered or networked control. In these settings, admissible policies may need to account not only for physical dynamics but also for delayed measurements, limited messages, and the time required to compute the next action."
    }
  ],
  "links": [
    "mathematical-optimization",
    "dynamical-system",
    "calculus-of-variations",
    "ordinary-differential-equation",
    "objective-function",
    "leonhard-euler",
    "euler-lagrange-equations",
    "richard-bellman",
    "measurable-function",
    "absolute-continuity",
    "lipschitz-continuity",
    "functional-analysis",
    "convex-optimization",
    "boundary-value-problem",
    "lagrangian-duality",
    "dynamic-programming",
    "value-function",
    "hamilton-jacobi-bellman-equation",
    "curse-of-dimensionality",
    "positive-definite-matrix",
    "kalman-filter",
    "bang-bang-control",
    "numerical-analysis",
    "karush-kuhn-tucker-conditions",
    "automatic-differentiation",
    "model-predictive-control",
    "expected-value",
    "wiener-process",
    "filtration-probability",
    "bellman-equation",
    "markov-decision-process",
    "value-iteration",
    "policy-iteration",
    "partially-observable-markov-decision-process",
    "robust-control",
    "game-theory",
    "reinforcement-learning",
    "economics",
    "sensitivity-analysis",
    "partial-differential-equation"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}