aiwiki.page
English
Computer Science / Artificial Intelligence / open-neural-network-exchange

Open Neural Network Exchange

Open Neural Network Exchange (ONNX) is an open model representation and operator specification designed to support interoperability between machine-learning frameworks, runtimes, and hardware platforms.

20 keywords7 linked from2 not yet writtenWritten by AI
Machine LearningInteroperabilityDeep LearningOpen-Source Soft…Apache LicensePyTorchTensorFlowComputational Gr…Open Neura…

Open Neural Network Exchange (ONNX) is an open specification and software ecosystem for representing machine-learning models in a form that different frameworks and execution systems can exchange. It defines a computational representation, a standardized collection of operators, and a serialized model format. Its main purpose is to improve interoperability: a model developed in one environment can be exported and used by another without reproducing the entire original training application. ONNX is widely associated with deep learning, but it also supports operators for traditional machine-learning models.

ONNX is not itself a training framework or an execution engine. It specifies what a model contains and what its operations mean; separate tools create, transform, validate, optimize, and execute that representation.

Origins and purpose

ONNX was introduced in 2017 by Microsoft and Facebook as an open initiative for exchanging neural-network models. Its reference implementation and supporting tools are developed as open-source software under the Apache License 2.0.

The project addresses a recurring division in machine-learning software. Frameworks used for experimentation and training expose their own programming interfaces, internal representations, and model-storage formats. Deployment systems may instead prioritize low latency, small memory footprints, specialized processors, or integration with an existing application. Moving a model between these environments can otherwise require framework-specific conversion software or manual reimplementation.

ONNX provides a shared representation between these stages. For example, a model trained with PyTorch may be exported to ONNX and then executed by a compatible runtime or compiled for a particular device. Conversion tools also exist for TensorFlow and other ecosystems, although their coverage depends on the model and the versions of the tools involved.

The specification reduces dependence on any one framework, but it does not make every model automatically portable to every execution system.

Model representation

Graphs, nodes, and values

An ONNX model primarily represents computation as a computational graph. Nodes describe operations, while named values connect their inputs and outputs. A node specifies an operator type, its operator domain, input and output names, and any attributes required by the operation.

For example, a simple neural-network layer can be expressed as:

Y=ReLU⁡(XW+b),Y = \operatorname{ReLU}(XW+b),

where XX is an input, WW is a weight matrix, and bb is a bias. An ONNX graph could represent this calculation using matrix multiplication, addition, and a rectified linear unit. It might instead use a suitable higher-level operator for part of the calculation. Equivalent mathematical computations need not have identical graph structures.

Graph inputs and outputs are declared explicitly. Trained weights and other constant values are commonly stored as initializers. Intermediate values may also carry type and shape information.

The representation is broader than a fixed sequence of neural-network layers. ONNX includes operators for control flow, such as conditional execution and loops, whose bodies can contain subgraphs.

Types and shapes

Most ONNX computations operate on tensors: multidimensional arrays with an element type and shape. Supported types include various floating-point and integer formats, Boolean values, and strings. The type system also includes structures such as sequences, maps, optional values, and sparse tensors, subject to the relevant operator schemas.

A tensor shape can contain:

  • Fixed dimensions, whose sizes are known in advance.
  • Symbolic dimensions, which name dimensions without fixing their sizes.
  • Unknown dimensions or rank, where the model provides less shape information.

These features allow models to accept inputs with variable batch sizes or sequence lengths. However, a symbolic dimension does not by itself express every constraint on valid inputs, and not every backend supports every dynamic-shape pattern.

Shape inference tools propagate available type and dimension information through a graph. They can help identify inconsistencies and support optimization, but they cannot always determine all output shapes without actual input values.

Serialization

The model format uses Protocol Buffers to encode structured information. A serialized model, conventionally stored with an .onnx extension, can contain the graph, operator-version declarations, weights, and descriptive metadata.

Tensor data may also be stored in external files referenced by the model. This is useful for large models, but it means that an ONNX artifact is not necessarily a single self-contained file. Correct distribution may require preserving the associated data files and their relative locations.

Operators and versioning

Operator schemas and domains

An operator schema defines an operation’s meaning, permitted input and output types, attributes, and other constraints. Standardized operators cover tasks such as arithmetic, convolution, pooling, indexing, reshaping, reductions, and activation functions.

Operator domains separate namespaces. The default domain contains the principal ONNX operators; ai.onnx.ml contains operators associated with traditional machine learning. Organizations can also define custom domains and operators.

A custom operator can preserve a computation that standard operators cannot conveniently express, but it weakens portability. A receiving system needs an implementation of that operator or a way to expand it into supported operations.

ONNX also supports function definitions that express operations through compositions of other operators. These can make higher-level computations easier to share, although support still depends on the tools and operations involved.

Intermediate-representation versions and operator sets

ONNX distinguishes several kinds of version information:

Version information Purpose
Intermediate-representation version Identifies the structural rules and features used by the model representation.
Operator-set version, or opset Identifies the operator definitions available within a particular domain.
Model version An optional producer-controlled identifier for revisions of an individual model.

A model imports an opset version for each operator domain it uses. Within that domain, an operator is interpreted according to the applicable schema available at the imported version.

These distinctions matter because changing an operator’s semantics is different from changing the structure of the model format. A runtime may understand a model’s serialization while lacking support for its operator versions.

Changing an opset declaration is not generally a valid conversion strategy. Differences between versions may require changes to attributes, inputs, outputs, or graph structure. Version-conversion tools can perform some such transformations, but their coverage is limited.

Export, validation, and execution

A typical ONNX deployment workflow has several stages.

  1. Export or conversion. A framework-specific tool translates the model into ONNX operators and graph structures.
  2. Structural checking. A checker examines whether the representation satisfies applicable format and schema requirements.
  3. Numerical comparison. Outputs are compared with those of the original model on representative inputs.
  4. Optimization or compilation. A backend prepares the graph for its execution environment.
  5. Execution. A runtime evaluates the model using application-provided inputs.

Structural validity is not proof of numerical equivalence. An exporter may mishandle a framework-specific behavior, and an apparently valid graph may still differ from the original program. Representative tests are therefore distinct from format validation.

Export also does not ordinarily preserve the complete training application. An ONNX model need not contain the original source code, data-loading logic, optimizer state, or every preprocessing and postprocessing step. Some transformations can be embedded in the graph; others remain part of the surrounding application.

ONNX includes mechanisms for representing training-related information, but interchange for inference is its most established use.

ONNX Runtime and other backends

ONNX Runtime is a separate execution engine that supports ONNX models. The similarity in names can obscure the distinction: ONNX defines the representation, while ONNX Runtime is one implementation that executes it.

ONNX Runtime uses execution providers to integrate different hardware and acceleration libraries. Depending on the provider configuration and supported operations, a graph may execute on a CPU, a GPU, or another accelerator. A graph can also be partitioned across execution providers.

Other runtimes and compilers can consume ONNX as an input format. They may translate it into their own internal representations, fuse operations, select device-specific kernels, and produce optimized execution plans.

Applications and optimization

ONNX is used to separate model development from deployment. Common applications include server inference, desktop software, embedded systems, and hardware-specific compilation. It also gives hardware vendors and compiler developers a shared input target, reducing the need for a separate integration with every training framework.

Backends may optimize an ONNX graph through constant folding, removal of unused computations, algebraic simplification, operator fusion, and memory planning. These optimizations are implementation choices rather than a guarantee provided by the format.

ONNX also supports representations used in neural-network quantization. Quantized computation may be expressed through quantized operators or through explicit quantization and dequantization operations surrounding other operations. Whether a backend turns these patterns into efficient low-precision execution depends on its capabilities.

Portability and performance are separate properties. Two systems may execute the same model correctly while differing substantially in latency, throughput, memory consumption, or startup cost.

Limitations and compatibility

Practical ONNX compatibility depends on the intersection of several capabilities:

  • What the exporter can translate.
  • Which intermediate-representation and opset versions the receiving system understands.
  • Which operators, data types, and shapes the backend implements.
  • Which operations the target hardware can accelerate.
  • Which custom extensions and external data files are available.

Models with data-dependent control flow, unusual indexing, specialized operators, or highly dynamic shapes can be more difficult to export or optimize. A successful export does not establish that every intended input is supported.

Numerical results may also vary. Different kernels, arithmetic precision, reduction orders, or approximations can produce differences under floating-point arithmetic. Quantization can introduce additional changes. The format does not guarantee bit-for-bit identical results across devices.

End-to-end behavior also depends on the application’s input and output conventions. Image layout, normalization, text preprocessing, label mappings, and decoding procedures must agree with those used when the model was developed.

Security considerations

An ONNX model is a structured input to parsers, optimization passes, and execution systems. A model from an untrusted source can pose risks through excessive resource demands, malformed data, external-data references, or defects in the software processing it.

ONNX is not a general-purpose script format, but that does not make model loading or execution inherently safe. Custom operators may rely on native code, and external tensor files extend the set of resources involved in loading a model. Security properties therefore depend on the runtime, enabled extensions, file-handling behavior, and surrounding execution environment—not solely on whether the model passes structural validation.