{
  "id": "lenet",
  "title": "LeNet",
  "lang": "en",
  "source_lang": "en",
  "summary": "LeNet is a family of early convolutional neural networks for document recognition, best known for the LeNet-5 handwritten-digit classifier.",
  "category": "computer-science",
  "subcategory": "artificial-intelligence",
  "type": "product",
  "aliases": [
    "LeNet neural network",
    "LeNet family"
  ],
  "generated_at": "2026-11-04",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/lenet/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/lenet/",
      "json": "https://aiwiki.page/zh/lenet.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "LeNet is a family of convolutional neural networks developed by Yann LeCun and collaborators for recognizing handwritten characters and other document images. Its best-known version, LeNet-5, was described in the 1998 paper Gradient-Based Learning Applied to Document Recognition by Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. LeNet helped establish a practical architecture in which learned image filters, spatial subsampling, and classification layers are trained together using backpropagation. It is a foundational example of deep learning for computer vision."
    },
    {
      "heading": "Development and historical context",
      "text": "LeNet emerged from research on applying artificial neural networks to document recognition, particularly handwritten postal codes and digits. Work published in 1989 demonstrated that a network with local connections and shared weights could learn to recognize handwritten ZIP-code digits through gradient-based training. Subsequent versions developed these ideas into the architecture associated with LeNet-5.\n\nThe approach differed from recognition pipelines that relied on manually designed image descriptors followed by a separate classifier. LeNet learned much of its feature extraction directly from labeled examples. Its architectural constraints also made image learning more manageable than connecting every input pixel to every neuron in the next layer.\n\nNetworks in the LeNet family were used within document-processing systems, including systems for reading handwritten amounts on bank checks. Such applications involved more than isolated digit classification: locating characters, considering alternative segmentations, and imposing document-level constraints were also important.\n\nThe name LeNet refers to a family rather than a single immutable specification. LeNet-5 is the usual reference architecture, but many contemporary examples called “LeNet” modify its activation functions, subsampling operations, connectivity, or output layer."
    },
    {
      "heading": "Architectural principles",
      "text": "LeNet combines three central ideas.\n\nLocal receptive fields. A neuron in an early convolutional layer processes a small region of the preceding layer rather than the entire image. This encodes the assumption that nearby pixels form useful local patterns, such as edges, curves, and stroke junctions.\n\nShared weights. The same learned filter is applied at different spatial locations. A feature can therefore be detected throughout the image without learning a separate set of parameters for every position. This sharing substantially reduces parameter count compared with a similarly sized multilayer perceptron.\n\nSpatial subsampling. Subsampling layers reduce the spatial resolution of feature maps. This lowers subsequent computational requirements and makes representations less sensitive to small changes in position or local shape.\n\nThese choices constitute an inductive bias suited to images. Weight sharing supports translation-equivariant feature detection: shifting an input tends to shift the corresponding feature responses. Subsampling can provide limited tolerance to shifts, but neither operation guarantees complete translation invariance."
    },
    {
      "heading": "The original LeNet-5 architecture",
      "text": "LeNet-5 processes a single-channel \\(32 \\times 32\\) image and produces scores for ten digit classes. The original architecture contains seven main processing layers after the input:\n\nLayer | Output shape | Operation\nInput | \\(1 \\times 32 \\times 32\\) | Grayscale image\nC1 | \\(6 \\times 28 \\times 28\\) | Six \\(5 \\times 5\\) convolutional filters\nS2 | \\(6 \\times 14 \\times 14\\) | \\(2 \\times 2\\) trainable subsampling\nC3 | \\(16 \\times 10 \\times 10\\) | \\(5 \\times 5\\) convolutions with partial channel connectivity\nS4 | \\(16 \\times 5 \\times 5\\) | \\(2 \\times 2\\) trainable subsampling\nC5 | \\(120 \\times 1 \\times 1\\) | 120 filters spanning the preceding spatial maps\nF6 | 84 units | Fully connected layer\nOutput | 10 scores | Euclidean radial-basis-function units\n\nThe convolutional layers use unit stride and no spatial padding. For example, applying a \\(5 \\times 5\\) filter to a \\(32 \\times 32\\) input produces a \\(28 \\times 28\\) map.\n\nConvolution and activation functions\n\nA convolutional feature map can be expressed in modern notation as\n\n\\[h_j(u,v)= \\phi\\left( b_j+ \\sum_{i\\in\\mathcal{C}_j} \\sum_{a,b} K_{ji}(a,b)x_i(u+a,v+b) \\right),\\]\n\nwhere \\(K_{ji}\\) is a learned kernel, \\(b_j\\) is a bias, and \\(\\mathcal{C}_j\\) specifies the input maps connected to output map \\(j\\). The same kernel coefficients are reused at every spatial position.\n\nThe original network used a scaled hyperbolic tangent activation function, commonly written as\n\n\\[\\phi(z)=1.7159\\,\\tanh\\left(\\frac{2}{3}z\\right).\\]\n\nReplacing it with the rectified linear unit is a common modern modification, not part of the original specification.\n\nTrainable subsampling\n\nThe S2 and S4 layers are often described informally as average-pooling layers, but the original operation was more specific. Each unit summed a \\(2 \\times 2\\) region, multiplied the sum by a learned coefficient, added a learned bias, and applied the activation function. A coefficient and bias were shared across each output feature map.\n\nThus, the original subsampling layers had trainable parameters. They were neither parameter-free average pooling nor max pooling, although both are used in later LeNet-style implementations.\n\nPartial connectivity in C3\n\nC3 did not connect every output map to all six S2 maps. Instead, it used a prescribed connection pattern: some output maps received three input maps, others four, and one received all six.\n\nThis reduced computation and encouraged different maps to learn different combinations of features. A modern implementation using a standard fully connected convolution across all input channels therefore differs from the original architecture.\n\nC5, F6, and the output layer\n\nC5 applies \\(5 \\times 5\\) kernels to S4 maps that are themselves \\(5 \\times 5\\). Each resulting map consequently has one spatial position. For this input size, C5 behaves like a fully connected transformation, although it is defined as a convolutional layer.\n\nF6 maps the 120 C5 outputs to 84 units. The original output layer then measures squared distances between this representation and class-specific prototype vectors:\n\n\\[d_k=\\sum_{j=1}^{84}(h_j-w_{kj})^2.\\]\n\nA smaller distance indicates a closer match to class \\(k\\). The original design used fixed prototype codes based on stylized character patterns. This differs from the trainable affine layer and softmax output common in contemporary classifiers.\n\nThe original architecture has approximately 60,000 trainable parameters. Counts reported for modern variants differ because changes to C3 connectivity, subsampling, and the output classifier change the parameter total."
    },
    {
      "heading": "Training and evaluation",
      "text": "LeNet is trained through supervised learning: labeled images supply the target character classes, a loss function measures classification error, and gradients propagate backward through the network to update its parameters. The original work investigated training objectives suited to its distance-based output units rather than simply using the softmax cross-entropy formulation now common in tutorials.\n\nLeNet-5 is closely associated with MNIST, a benchmark containing 60,000 training images and 10,000 test images of handwritten digits. MNIST images are \\(28 \\times 28\\) pixels, while the canonical LeNet-5 input is \\(32 \\times 32\\); preprocessing can place the smaller image within a larger canvas.\n\nReported error rates depend on preprocessing, training procedures, architectural details, and whether data augmentation is used. A result obtained with a modified “LeNet” implementation is therefore not automatically a result for the original LeNet-5. Meaningful comparisons require specifying the exact model and keeping the test set separate from model selection."
    },
    {
      "heading": "Applications and influence",
      "text": "LeNet’s principal historical application was optical character recognition, particularly handwritten digits in structured documents. Its broader importance lies in demonstrating that learned convolutional representations could support practical recognition systems.\n\nThe architecture established a pattern that became common in image classification: alternate spatial feature extraction with resolution reduction, increase the number of feature channels in deeper layers, and classify the resulting representation. Later networks such as AlexNet used substantially larger models and datasets, but retained important elements of this pattern.\n\nLeNet also remains a useful teaching architecture. Its modest size makes it possible to examine feature-map dimensions, weight sharing, receptive fields, and gradient-based learning without the computational demands of large vision models. Implementations in PyTorch and TensorFlow frequently use simplified LeNet-like networks to illustrate these concepts."
    },
    {
      "heading": "Limitations and implementation differences",
      "text": "LeNet-5 was designed for small, normalized grayscale character images, not for unrestricted natural-image recognition. Its limited capacity and fixed design constrain performance on tasks involving large images, complex backgrounds, many object categories, or substantial changes in viewpoint.\n\nIts local connectivity and subsampling provide only partial robustness to geometric variation. Rotation, scale changes, unusual handwriting, and changes in image acquisition can still affect predictions. Performance on a clean digit benchmark does not by itself establish robustness under distribution shift.\n\nThe original saturating activation functions can also make optimization less convenient than in networks using modern non-saturating activations. Contemporary architectures additionally employ techniques such as batch normalization and residual connections, neither of which belongs to canonical LeNet-5.\n\nCommon differences in modern reproductions include:\n\n- Accepting \\(28 \\times 28\\) inputs directly instead of \\(32 \\times 32\\) inputs.\n- Replacing trainable subsampling with average or max pooling.\n- Making all C3 channel connections dense.\n- Replacing scaled hyperbolic tangents with rectified linear units.\n- Using a trainable ten-class linear output layer and cross-entropy loss.\n\nThese variants preserve the broad LeNet pattern, but an exact reproduction requires matching the original connectivity, nonlinearities, subsampling rules, output units, and training objective."
    }
  ],
  "links": [
    "convolutional-neural-network",
    "backpropagation",
    "deep-learning",
    "computer-vision",
    "artificial-neural-network",
    "feature-extraction",
    "multilayer-perceptron",
    "inductive-bias",
    "activation-function",
    "rectified-linear-unit",
    "supervised-learning",
    "loss-function",
    "data-augmentation",
    "test-set",
    "image-classification",
    "alexnet",
    "pytorch",
    "tensorflow",
    "distribution-shift",
    "batch-normalization",
    "residual-connection"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}