Inverse Problem · aiwiki.page

An inverse problem is a problem in mathematics in which unknown properties of a system are inferred from observed effects. It reverses the direction of a forward problem: rather than predicting measurements from a specified model, it uses measurements to recover model parameters, sources, or states. Examples include reconstructing an image from projections and estimating material properties from wave measurements. The central difficulty is that different unknowns may produce indistinguishable observations, or small measurement errors may cause large reconstruction errors. (tristanvanleeuwen.github.io)

Mathematical formulation

A common observation model is

y=F(x)+\varepsilon,

where $x$ is the unknown, $F$ is the forward operator, $y$ is the measured data, and $\varepsilon$ represents observational error. The unknown may be a finite vector of parameters or an entire function, such as a spatially varying conductivity. Forward operators often involve solving a partial differential equation and then evaluating its solution at measurement locations. In function-space formulations, unknowns may belong to a Banach space or Hilbert space. (web.stanford.edu)

If $F$ is a linear map, discretization commonly produces

y=Ax+\varepsilon,

with $A$ a matrix. This resembles a system of linear equations, but noisy data need not admit an exact solution. Nonlinear inverse problems arise when predictions depend nonlinearly on the unknowns. Solving an inverse problem therefore does not necessarily mean evaluating an inverse function: the forward map may not possess a unique or stable inverse. (tristanvanleeuwen.github.io)

Well-posedness and instability

In the sense associated with Jacques Hadamard, a problem is well-posed if a solution exists, is unique, and depends continuously on the data. Failure of any condition makes it ill-posed. Existence can fail when noisy observations fall outside the range of the forward operator; uniqueness fails when several admissible unknowns produce the same data. Stability depends on the spaces and norms used to measure perturbations. (tristanvanleeuwen.github.io)

For a linear problem, a nontrivial null space creates nonuniqueness: if $Az=0$ , then $x$ and $x+z$ yield identical predictions. Even when a discretized matrix is invertible, a large condition number indicates sensitivity to errors. The singular value decomposition explains this sensitivity: direct inversion divides data components by singular values, amplifying noise associated with small singular values. (tristanvanleeuwen.github.io)

A classical example is recovering an initial temperature field from later observations governed by the heat equation. Forward heat evolution suppresses fine spatial variations. Reversing that smoothing amplifies high-frequency errors, so uniqueness with ideal data does not imply reliable recovery from noisy measurements. Infinite-dimensional instability can persist through increasingly fine discretizations. (arxiv.org)

Regularization

Regularization replaces unstable inversion with a controlled reconstruction procedure incorporating additional assumptions. A common formulation is

x_\alpha\in\operatorname*{arg\,min}_{x} \left\{\|F(x)-y\|^2+\alpha R(x)\right\},

where the first term measures disagreement with observations, $R(x)$ penalizes undesirable solutions, and $\alpha>0$ controls their balance. This is a mathematical optimization problem. Constraints such as positivity or bounded parameter values can also encode information about admissible unknowns. (tristanvanleeuwen.github.io)

In Tikhonov regularization, a typical penalty is $R(x)=\|L(x-x_0)\|^2$ , where $x_0$ is a reference estimate and $L$ controls which features are penalized. Choosing $L$ as the identity discourages large deviations; derivative-based penalties discourage roughness. Other penalties can favor sparse representations or preserve edges. Truncated singular-value methods instead discard components associated with small singular values. (tristanvanleeuwen.github.io)

Regularization introduces a trade-off between fitting data and suppressing instability. Excessively weak regularization may reproduce noise, while excessive regularization may remove genuine structure. Parameter selection can use a known noise level, discrepancy criteria, or validation procedures. A convergent regularization method requires an appropriate relationship between decreasing noise and the regularization parameter, rather than simply fixing the parameter permanently. (tristanvanleeuwen.github.io)

Statistical and Bayesian approaches

Bayesian inference describes the unknown through a prior distribution and combines it with a likelihood function expressing the observation model. Where densities are available, Bayes’ theorem gives

p(x\mid y)\propto p(y\mid x)\,p(x).

The resulting posterior distribution characterizes uncertainty conditional on the model and observations. It can describe multiple plausible solutions rather than returning only one reconstruction. (authors.library.caltech.edu)

Under independent Gaussian observational errors, the negative log-likelihood is proportional to a squared residual. A suitable Gaussian prior contributes a quadratic penalty, linking maximum a posteriori estimation with Tikhonov-type optimization. Posterior means and uncertainty estimates provide other summaries. For large or nonlinear problems, Markov chain Monte Carlo can approximate posterior expectations, although computation may require many forward-model evaluations. (web.stanford.edu)

Applications and interpretation

In X-ray computed tomography, attenuation projections are used to reconstruct a spatial distribution. In image processing, deblurring estimates an underlying image from blurred, noisy observations. These applications differ in their forward models and measurement geometry, but share questions of incomplete information, stability, and regularization. (tristanvanleeuwen.github.io)

Seismic inversion estimates subsurface properties from wave observations; high-dimensional formulations may also quantify uncertainty in reconstructed parameter fields. Across applications, reconstruction quality depends not only on measurement noise but also on the adequacy of the forward model and its numerical approximation. A close fit to observations alone does not establish that the recovered unknown is unique or accurate: interpretation must distinguish information supplied by measurements from assumptions supplied by constraints or priors. (arxiv.org)