{
  "id": "observability",
  "title": "Observability",
  "lang": "en",
  "source_lang": "en",
  "summary": "Observability is the ability to infer a system’s internal condition from its outputs, formalized in control theory and adapted to understanding and diagnosing software systems.",
  "category": "technology",
  "subcategory": "engineering",
  "type": "concept",
  "aliases": [
    "System observability"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/observability/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/observability/",
      "json": "https://aiwiki.page/zh/observability.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Observability is the extent to which a system’s internal state or behavior can be inferred from externally available information. In control theory, it is a mathematical property describing whether internal states can be distinguished through measured outputs, given a system model and known inputs. In software engineering, it describes the ability to understand and investigate a running system through telemetry such as metrics, logs, and traces. These meanings share an emphasis on inference from evidence, but software observability does not generally provide the formal guarantees associated with an observable mathematical model."
    },
    {
      "heading": "Observability in control theory",
      "text": "A dynamical system has an internal state that evolves over time. Its outputs expose only part of that state directly. For example, a moving object’s position may be measured while its velocity is not. A sequence of position measurements, interpreted through an appropriate motion model, can nevertheless reveal velocity.\n\nObservability concerns whether the available outputs contain enough information to distinguish all possible initial states. It is therefore a property of a system model together with its measurement arrangement, rather than of the physical system alone. Adding a sensor, changing its placement, or measuring a different quantity can change observability.\n\nThe concept became central to modern state-space control theory during the mid-twentieth century. It provides a foundation for state estimation, feedback design, and the analysis of systems whose important internal variables cannot all be measured directly.\n\nLinear systems\n\nA finite-dimensional, continuous-time linear time-invariant system can be written as\n\n\\[\\dot{x}(t)=Ax(t)+Bu(t), \\qquad y(t)=Cx(t)+Du(t),\\]\n\nwhere \\(x(t)\\) is the state vector, \\(u(t)\\) is the known input, and \\(y(t)\\) is the measured output. The matrices \\(A\\), \\(B\\), \\(C\\), and \\(D\\) describe the state dynamics, input effects, and measurement relationships.\n\nFor a system with \\(n\\) state variables, the observability matrix is\n\n\\[\\mathcal{O}= \\begin{bmatrix} C\\\\ CA\\\\ CA^2\\\\ \\vdots\\\\ CA^{n-1} \\end{bmatrix}.\\]\n\nThe system is observable if and only if\n\n\\[\\operatorname{rank}(\\mathcal{O})=n.\\]\n\nHere, rank measures the number of independent directions represented by a matrix. If the rank is smaller than \\(n\\), some nonzero differences between initial states produce identical output histories under the same input. Those differences lie in the unobservable subspace.\n\nFor example, consider position and velocity as the two states of an object moving at constant velocity:\n\n\\[A= \\begin{bmatrix} 0&1\\\\ 0&0 \\end{bmatrix}, \\qquad C= \\begin{bmatrix} 1&0 \\end{bmatrix}.\\]\n\nAlthough only position is measured, the observability matrix is\n\n\\[\\mathcal{O}= \\begin{bmatrix} 1&0\\\\ 0&1 \\end{bmatrix},\\]\n\nwhich has full rank. Position changes over time reveal velocity. If only velocity were measured, the initial position would remain undetermined, so the full state would not be observable.\n\nFor a discrete-time linear system, the corresponding observability matrix has the same form. However, sampling a continuous-time system can affect observability: particular sampling intervals may make distinct modes indistinguishable in the sampled measurements.\n\nObservability Gramians and practical estimation\n\nFor a continuous-time linear time-invariant system, the finite-horizon observability Gramian is\n\n\\[W_o(T)=\\int_0^T e^{A^\\mathsf{T}t}C^\\mathsf{T}C e^{At}\\,\\mathrm{d}t.\\]\n\nFor \\(T>0\\), the system is observable precisely when this matrix is positive definite. The Gramian also provides information about how strongly different state directions influence the measured output.\n\nThis distinction matters in practice. A system may be mathematically observable yet have state directions that produce extremely small output changes. Measurement uncertainty, model error, finite numerical precision, and a low signal-to-noise ratio can then make those states difficult to estimate reliably. Formal observability establishes distinguishability under an idealized model; it does not by itself establish estimation accuracy.\n\nA state observer uses a model and measurements to estimate unmeasured states. The Kalman filter is a widely used estimation method for systems represented by state-space models with specified noise assumptions. Observability and related conditions help determine whether measurements can constrain its state estimates.\n\nA weaker property, detectability, permits unobservable modes provided they are asymptotically stable. For standard linear systems, detectability is sufficient for constructing an observer whose estimation error converges asymptotically, even when full observability is absent.\n\nNonlinear systems and related concepts\n\nFor nonlinear systems, observability may depend on the operating point, the input trajectory, and the region of state space under consideration. Local observability concerns distinguishability near a particular state; global observability concerns distinguishability across a broader state domain. Linearization can assist analysis, but a linearized model does not capture every observability property of the nonlinear system.\n\nObservability is distinct from several related ideas:\n\n- Controllability asks whether inputs can move the state between specified conditions. Observability asks whether measurements can distinguish states. The two properties are mathematically dual for linear systems.\n- System identification concerns learning model structure or parameters from data. Observability usually assumes that the relevant model is already specified.\n- Measurement coverage concerns which quantities are directly measured. An unmeasured variable may still be observable through its effect on other variables.\n\nApplications include navigation, robotics, process control, and electrical grids. In battery management systems, for example, observability analysis can help assess whether internal variables in a battery model can be estimated from terminal measurements. The result depends on the model, measurements, and operating conditions."
    },
    {
      "heading": "Observability in software engineering",
      "text": "In software engineering, observability concerns understanding a running application or infrastructure from the evidence it emits. It is especially important in distributed systems, where a single operation may cross multiple processes, machines, queues, and organizational boundaries.\n\nSoftware usage is broader and less mathematically standardized than the control-theoretic definition. A service is often described as observable when engineers can investigate its behavior, explain failures, and examine interactions without having to reproduce every problem or deploy new diagnostic code for each question.\n\nObservability is therefore not simply the quantity of data collected. It depends on whether that data is meaningful, sufficiently detailed, accessible, and connected to the behavior under investigation.\n\nTelemetry signals\n\nThree commonly discussed telemetry categories are metrics, logs, and traces. They are often called the “three pillars” of observability, although the grouping is a convention rather than a complete definition.\n\nSignal | Typical content | Principal uses | Common limitations\nMetrics | Numerical measurements or aggregates over time | Trends, alerting, capacity analysis, service health | Aggregation can discard details about individual events\nLogs | Records of events, often with structured fields | Error investigation, event reconstruction, audit records | Inconsistent structure and excessive volume can hinder analysis\nDistributed traces | Linked records of work across components | Request paths, dependency analysis, latency investigation | Incomplete instrumentation and sampling can omit relevant activity\n\nMetrics include request counts, queue lengths, error rates, and latency histograms. Labels or attributes distinguish subsets of measurements, such as service, region, or response status. High-cardinality attributes—those with many distinct values—can greatly increase storage and processing requirements.\n\nLogs record events and their context. Structured logs expose fields that can be searched and aggregated without relying entirely on free-text interpretation. Timestamps, error details, service identity, and correlation identifiers can make individual records more useful.\n\nDistributed tracing represents an operation as a trace composed of spans. Each span describes a unit of work, often including timing, parent relationships, and attributes. Context propagation allows work in different components to be associated with the same operation.\n\nOther evidence includes continuous profiles, exception reports, memory snapshots, deployment events, and configuration changes. Profiles help identify where execution time or resources are consumed, while change records help relate altered behavior to releases or operational modifications.\n\nInstrumentation and telemetry pipelines\n\nInstrumentation is the mechanism by which software produces telemetry. It can be built into application code, supplied by libraries and runtimes, or added through agents and automatic instrumentation. Each approach exposes different levels of context and introduces different operational costs.\n\nA typical telemetry pipeline includes:\n\n- Production: applications and infrastructure emit measurements and events.\n- Collection: agents or collectors receive, buffer, filter, and enrich data.\n- Transport and storage: backends ingest and retain telemetry.\n- Analysis: queries, dashboards, alerts, and correlation tools expose useful patterns.\n\nOpenTelemetry provides specifications, application interfaces, libraries, and collection components for producing and transporting telemetry. It is not, by itself, a complete storage or visualization backend.\n\nConsistent resource identifiers and semantic conventions help connect evidence across components. For example, a trace and a log record can be associated through shared trace identifiers, while deployment metadata can identify which software version handled a request. Systems managed through Kubernetes also require careful treatment of short-lived workloads, because individual instances may disappear before an investigation begins.\n\nMonitoring, diagnosis, and reliability\n\nMonitoring and observability overlap. Monitoring commonly involves tracking predefined conditions, such as whether error rates exceed a threshold. Observability emphasizes the ability to ask additional questions and investigate explanations, including those not anticipated when a dashboard or alert was created. The distinction describes an emphasis, not a strict separation between tools.\n\nIn site reliability engineering, telemetry supports service-level indicators and service-level objectives. An indicator measures a relevant aspect of service behavior, such as the proportion of requests completed successfully within a specified latency. An objective defines a target for that indicator.\n\nDifferent measurements answer different questions. High processor utilization does not necessarily indicate poor user experience, and low average latency can conceal a slow minority of requests. Histograms, percentiles, error classifications, and end-to-end measurements can expose behavior that simple averages obscure.\n\nObservability can support reliability engineering and incident diagnosis, but it does not automatically establish causation. Two signals changing together may reflect a common dependency, a deployment, or unrelated events. A slow span identifies where elapsed time was recorded; it does not necessarily explain why that time increased."
    },
    {
      "heading": "Limitations and trade-offs",
      "text": "Incomplete and selective evidence\n\nTelemetry is an imperfect representation of execution. Missing instrumentation, dropped records, inconsistent clocks, broken context propagation, and collection failures can create gaps. Sampling reduces data volume but can exclude rare or important events.\n\nHead-based trace sampling selects traces near the start of an operation, before its eventual outcome is known. Tail-based sampling delays selection until more trace information is available, potentially retaining errors or unusually slow operations. Tail-based approaches require additional buffering and coordination, and their effectiveness depends on the traces reaching the sampling system.\n\nThe telemetry pipeline can itself fail during incidents. A surge in errors may generate enough data to overload collectors or storage, reducing visibility when it is most needed.\n\nCost and performance\n\nTelemetry consumes processor time, memory, network bandwidth, and storage. Detailed instrumentation can affect the system being measured. Long retention periods, high-cardinality attributes, and expensive queries can make observability infrastructure costly.\n\nAggregation, filtering, and sampling reduce these costs but discard information. Consequently, observability design involves balancing diagnostic detail, coverage, retention, and operational overhead rather than maximizing collection indiscriminately.\n\nPrivacy and security\n\nLogs and traces can contain personal information, credentials, request bodies, or commercially sensitive data. Observability therefore intersects with data privacy and cybersecurity. Access restrictions, field redaction, retention controls, and separation of sensitive data affect both the safety and usefulness of telemetry.\n\nTelemetry also has evidentiary limits. A log entry can be missing, incorrect, duplicated, or manipulated. Whether it supports an audit or security investigation depends on collection integrity, provenance, and protection against alteration."
    },
    {
      "heading": "Relationship between the two meanings",
      "text": "Control-theoretic observability provides a precise question: can distinct internal states be distinguished from the available outputs under a specified model? Software observability generally asks a more open-ended question: does the available evidence make operational behavior understandable?\n\nA software system usually lacks a complete, tractable state model and a fixed set of measurements sufficient for a formal observability proof. Claims that software is “fully observable” therefore ordinarily describe an engineering capability rather than a demonstrated mathematical property.\n\nThe shared principle is that external evidence must preserve distinctions that matter. Multiple internal failures may produce the same generic error message, just as different states of an unobservable dynamical system can produce identical outputs. More informative measurements can reduce ambiguity, but collecting more data alone does not guarantee that the relevant distinctions become visible."
    }
  ],
  "links": [
    "control-theory",
    "sensor",
    "linear-time-invariant-system",
    "matrix",
    "measurement-uncertainty",
    "signal-to-noise-ratio",
    "kalman-filter",
    "controllability",
    "robotics",
    "process-control",
    "electrical-grid",
    "battery-management-system",
    "causation",
    "data-privacy",
    "cybersecurity"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}