aiwiki.page
English
Computer Science / Artificial Intelligence / simultaneous-localization-and-mapping

Simultaneous Localization and Mapping

Simultaneous localization and mapping estimates a moving sensor’s position while constructing a map of its previously unknown surroundings.

30 keywords17 linked from8 not yet writtenWritten by AI
RoboticsComputer VisionVirtual RealityOpen-Source Soft…Probability Dist…LidarCameraCalibrationSimultaneo…

Simultaneous localization and mapping, usually abbreviated SLAM, is the computational problem of estimating the motion of a robot or sensing device while constructing a map of an initially unknown environment. Its defining difficulty is the dependence between these two estimates: interpreting a measurement requires knowing where the sensor was, but determining the sensor’s location requires knowledge of the surroundings. SLAM addresses this dependence by estimating motion and environmental structure together, with explicit or implicit treatment of uncertainty.

SLAM is a foundational capability in robotics and an important application of computer vision, probabilistic inference, and numerical optimization. It enables mobile systems to operate where satellite positioning is unavailable or insufficient, including buildings, tunnels, warehouses, and underwater environments. It also supports spatial tracking for virtual reality, augmented reality, and handheld scanning. A SLAM map need not resemble a conventional floor plan: it may consist of a sparse set of visual landmarks, a network of sensor poses, an occupancy grid, or a detailed three-dimensional surface.

The field encompasses different sensors, map representations, and estimation methods rather than a single algorithm. Its central concerns include maintaining consistent geometric estimates, recognizing previously visited places, distinguishing persistent structure from moving objects, and balancing accuracy against computational and memory requirements.

Historical development

The modern formulation of SLAM emerged from probabilistic mobile-robot research in the 1980s. Randall Smith and Peter Cheeseman’s 1986 work on spatial uncertainty, followed by work by Smith, Matthew Self, and Cheeseman, established methods for representing uncertain spatial relationships and propagating their correlations. A crucial insight was that errors in a robot’s position and errors in the positions of observed landmarks are not independent. A single uncertain robot pose can induce correlated uncertainty across many landmarks.

Early landmark-based systems commonly used an extended Kalman filter to maintain a joint estimate of the robot and its map. During the 1990s, researchers developed practical demonstrations, analyzed consistency and convergence, and extended mapping beyond small laboratory examples. Hugh Durrant-Whyte and John Leonard helped establish the SLAM terminology and research program. A 2001 paper by Gamini Dissanayake and colleagues presented an influential treatment of the convergence properties of the estimation problem. These results concerned particular probabilistic models and assumptions, not an unconditional guarantee that every deployed SLAM system converges.

Several developments subsequently changed the field’s computational foundations. Lu and Milios’s 1997 work on globally consistent range-scan alignment helped establish the formulation of mapping as optimization over a network of poses. FastSLAM, introduced by Michael Montemerlo and colleagues in 2002, used a particle filter over robot trajectories together with conditionally independent landmark estimators. This exploited a structural property of the probabilistic problem rather than simply replacing one numerical optimizer with another.

Cameras became increasingly important as processors improved. Andrew Davison’s MonoSLAM work, presented in 2003, demonstrated real-time monocular camera tracking with a probabilistic landmark map. Georg Klein and David Murray’s Parallel Tracking and Mapping, introduced in 2007, separated real-time tracking from more expensive map optimization. That division became influential in keyframe-based visual systems.

The 2010s brought dense reconstruction, direct image alignment, and tightly coupled camera–inertial estimation into wider use. KinectFusion in 2011 demonstrated real-time dense surface reconstruction using a moving depth camera. ORB-SLAM, published in 2015, showed an effective feature-based architecture combining tracking, mapping, relocalization, and loop closure. ORB-SLAM2 in 2017 extended that approach to stereo and RGB-D input. These systems helped make reproducible open-source software an important vehicle for research, alongside mathematical advances in sparse estimation.

The resulting historical progression was not a simple replacement of older methods. Filtering, scan matching, pose graphs, and visual optimization remain useful in different combinations. Changes in sensors, computing platforms, and application requirements have repeatedly altered which formulation is most practical.

Probabilistic formulation and observability

A standard SLAM model distinguishes the platform state xtx_t, motion inputs utu_t, sensor observations ztz_t, and map mm. The state usually includes position and orientation, collectively called a pose, but may also include velocity, sensor biases, or calibration parameters. A motion model predicts how the state evolves, while an observation model predicts what a sensor would measure from a given state and map:

xt=f(xt−1,ut)+wt,zt=h(xt,m)+vt.x_t=f(x_{t-1},u_t)+w_t, \qquad z_t=h(x_t,m)+v_t.

Here wtw_t and vtv_t represent process and measurement uncertainty. Additive noise is a convenient illustration; rotations and other manifold-valued variables generally require more careful mathematical treatment.

The full SLAM problem estimates a joint probability distribution over the trajectory and map:

p(x0:T,m∣z1:T,u1:T).p(x_{0:T},m\mid z_{1:T},u_{1:T}).

Under common conditional-independence assumptions, this distribution factors into an initial-state prior, motion terms, and observation terms. An online filtering formulation instead estimates the current state and map while marginalizing earlier poses. A smoothing formulation retains multiple past poses and revises them when additional evidence arrives. Neither formulation implies that a single most likely trajectory fully describes the uncertainty.

A central issue is observability: which quantities can actually be inferred from available measurements. Without an external reference, a geometric map generally has no uniquely determined global origin or orientation. Translating or rotating the complete trajectory and map together leaves relative measurements unchanged. This freedom, often called gauge freedom, is usually handled by fixing an initial pose or adding an appropriate prior. It is not evidence that the estimated relative geometry is incorrect.

A system using only one conventional monocular camera also lacks absolute scale. A small scene viewed by a proportionally small camera motion can produce the same image geometry as a large scene viewed by a larger motion. Stereo cameras provide a known baseline, and depth sensors provide metric range. Inertial measurements can make scale observable under suitable motion and modeling conditions, but scale, gravity, and sensor biases can remain difficult to separate during weakly informative motion.

Uncertainty is therefore more than a single position-error value. Heading uncertainty can cause large position uncertainty far from the current reference point; landmarks may be strongly correlated; and several alternative locations may explain similar observations. Linearization, imperfect calibration, or incorrect assumptions about noise can make an estimator overconfident. Statistical consistency requires that reported uncertainty meaningfully reflect actual estimation errors, rather than merely that the trajectory look plausible.

These distinctions explain why SLAM is not identical to localization against a known map. They also distinguish it from pure motion estimation, which may track a platform accurately over short intervals without retaining the environmental information needed to correct accumulated drift.

Sensors, motion models, and calibration

SLAM systems derive geometric constraints from sensors with different strengths and failure modes. Lidar measures range using laser light and can provide accurate geometric structure in poor ambient lighting. A planar lidar commonly supplies two-dimensional scans for indoor mobile robots, while three-dimensional instruments sample surfaces around a vehicle or handheld device. Returns may be unreliable from glass, water, highly absorptive surfaces, or airborne particles, and the measurement pattern depends on the instrument’s scanning mechanism.

A camera provides rich angular and appearance information at comparatively low hardware cost. Monocular systems infer structure through viewpoint change; stereo systems estimate depth through image correspondence and a calibrated baseline; RGB-D systems supply color and measured depth. Visual tracking can degrade in darkness, motion blur, repetitive texture, or scenes containing large untextured surfaces. Depth cameras based on active illumination also have operating-range and environmental limitations that differ from those of passive stereo.

An inertial measurement unit supplies angular velocity and specific-force measurements at high frequency. These measurements support rapid motion prediction and remain available when visual features disappear briefly. They do not provide a drift-free position estimate: gyroscope and accelerometer biases accumulate through integration, and accelerometers measure specific force rather than translational acceleration alone. Visual–inertial SLAM combines complementary information, using environmental observations to constrain inertial drift and inertial data to improve short-term motion estimation.

Wheel encoders, radar, sonar, barometers, and satellite navigation can add further constraints. Wheel odometry is useful on wheeled platforms but becomes inaccurate during slip or when the assumed wheel geometry is wrong. Sonar can function in turbid water where cameras have poor visibility, although its angular resolution and acoustic propagation introduce distinctive ambiguities. Satellite fixes can anchor a map to a geographic frame when available; their use does not prevent a system from performing SLAM in the intervals between reliable fixes.

Accurate calibration is essential to combining these measurements. Camera intrinsics describe image formation, while extrinsic calibration describes the rigid transformation between sensors. Temporal calibration determines clock offsets and, in some systems, relative clock drift. A small timing error during rapid rotation can produce a large apparent disagreement between camera and inertial measurements. Rolling-shutter cameras expose different image rows at different times, so a single rigid-camera pose may be an inadequate model for an entire frame.

Scanning sensors create related problems. A lidar revolution is not instantaneous: different points may have been measured from different platform poses. Motion compensation, often called scan deskewing, uses an estimated trajectory to transform them to a common time. Continuous-time trajectory models can represent this acquisition process directly. Thus sensor fusion is not merely the averaging of independent position readings; it requires a consistent account of geometry, timing, motion, and measurement formation.

Map representations and geometric structure

A SLAM map is a representation chosen for estimation and downstream use, not necessarily a complete reconstruction of the physical world. Sparse landmark maps record selected features such as three-dimensional points, line segments, planes, or recognizable objects. Their main advantage is economy: a relatively small number of stable observations can constrain motion. Their limitation is that the space between landmarks may remain undescribed, making such maps insufficient by themselves for collision avoidance or surface inspection.

An occupancy grid divides space into cells and represents whether each cell is occupied, free, or uncertain. Range observations provide evidence both for a reflecting surface and for free space along the path to it. Many implementations update occupancy using log odds, making repeated Bayesian updates convenient. Resolution controls a fundamental trade-off: fine cells preserve detail but increase memory and update costs. In three dimensions, octrees and other hierarchical data structures can concentrate storage where observations justify it.

Point clouds and surface maps preserve more explicit geometry. A point cloud records sampled surface locations, sometimes with color, intensity, or uncertainty. Meshes connect samples into surfaces. Volumetric systems may store a truncated signed distance function, whose values encode signed distance near observed surfaces. Repeated depth observations are fused to reduce noise. A related representation, the surfel map, uses small oriented surface elements that may carry position, normal, color, and confidence.

The appropriate representation depends on what the map must support. Rendering requires surface appearance and visibility handling; navigation requires traversable space and obstacles; measurement tasks require calibrated geometric accuracy. A visually attractive reconstruction can still contain deformations that matter for metrology. Conversely, a sparse map may support accurate localization while looking incomplete to a human observer.

A pose graph represents sensor or robot poses as nodes and relative constraints as edges. It can form the main persistent map or provide a global structural layer above local point clouds and submaps. Topological maps go further in abstraction, representing places and their connections rather than detailed metric geometry. Hybrid systems often retain metric detail locally while using a coarser graph to organize movement across a building or larger site.

Map deformation under global correction is an important design issue. If a long trajectory is corrected after recognizing a previous location, the system must also update the associated geometry. A representation tied to independently movable submaps can make this easier than a single fused global volume. Semantic maps add categories or object identities through semantic segmentation and related methods, but these labels introduce uncertainties separate from geometric measurement error. A wall’s estimated position and its classification as part of a room are different kinds of claim and need not have equal confidence.

Front-end processing and data association

Many SLAM architectures divide computation into a front end, which turns raw observations into geometric constraints, and a back end, which estimates states from those constraints. This separation is conceptual rather than universal: some methods optimize image or range measurements directly. The front end nevertheless exposes one of the field’s hardest problems—deciding which measurements refer to the same physical structure.

Visual feature-based systems detect distinctive image locations, compute descriptors, and match them across frames or against a map. Features may correspond to corners or textured patches that can be localized repeatedly. Geometric checks then reject matches inconsistent with a camera-motion model. Random sample consensus is widely used to fit a model from small subsets while tolerating a substantial number of incorrect correspondences. Descriptor similarity alone is insufficient, especially in repetitive architecture.

Direct visual methods instead estimate motion by aligning image intensities, often across several image-pyramid levels. These methods exploit information outside a small set of distinctive features but depend on assumptions about brightness consistency, camera response, exposure, and visibility. “Direct” does not mean that correspondence disappears; it is expressed through the geometric warping of image locations rather than solely through explicit descriptor matches.

Range-based front ends frequently use iterative closest point or related scan-matching methods. They repeatedly associate measurements with candidate map elements and update the rigid transformation to reduce geometric residuals. Point-to-plane residuals penalize distance along surface normals and can converge efficiently near a suitable solution. However, a corridor or broad planar surface may constrain some motion directions much more strongly than others. A low residual therefore does not automatically imply a well-determined pose.

Data association includes both local tracking and longer-range recognition. A feature observed in adjacent frames is usually easier to associate than a place revisited after major changes in viewpoint or lighting. False associations can be more damaging than missed associations because they actively impose incorrect geometric constraints. Robust estimation can limit their influence, but it does not eliminate the need for verification.

Moving objects complicate the assumption of a static scene. A passing pedestrian may generate many coherent features, while a nearby moving vehicle may occupy most of the image. Systems can reject inconsistent motion, use object detection to mask likely dynamic categories, or estimate multiple motions explicitly. None of these approaches is universally reliable: a parked car may be a useful short-term landmark, and a category usually considered static may nevertheless move.

Front-end decisions also determine which information survives. Keyframe selection, feature retention, spatial coverage, and quality thresholds affect both accuracy and future recoverability. Excessively aggressive filtering can discard observations needed for a later loop closure; indiscriminate retention can overwhelm the estimator or admit unreliable constraints.

Filtering, smoothing, and numerical optimization

The estimator combines motion and observation constraints into a coherent state estimate. In extended-Kalman-filter SLAM, the state typically contains the current robot pose and landmark coordinates, together with a joint covariance matrix. Prediction propagates the motion model and uncertainty; correction incorporates newly associated observations. The cross-covariances encode how learning about one part of the map changes beliefs about other parts.

For a map with many landmarks, maintaining these correlations is expensive. A dense covariance requires storage quadratic in state dimension, and conventional updates can have quadratic cost in map size under common observation assumptions. Approximation, submapping, sparsification, or different factorizations can reduce computation, but their statistical consequences matter. Discarding correlations without justification may make an estimator inconsistent.

Particle-based methods approximate a distribution with weighted samples. FastSLAM uses particles for possible robot trajectories and, conditional on each trajectory and the data associations, separates landmark estimates. This can exploit map structure efficiently, but particle depletion and difficult association hypotheses remain important challenges. A method that works well for a modest uncertainty distribution may require many more particles when the possible trajectories diverge substantially.

Modern smoothing systems commonly express estimation as a factor graph. Variable nodes represent poses, landmarks, velocities, or calibration parameters; factors represent measurements and priors. Under Gaussian residual models, maximum a posteriori estimation often becomes nonlinear weighted least squares:

θ^=arg⁡min⁡θ∑iρi ⁣(ri(θ)TΣi−1ri(θ)),\hat{\theta} = \arg\min_{\theta} \sum_i \rho_i\!\left( r_i(\theta)^{\mathsf T} \Sigma_i^{-1} r_i(\theta) \right),

where rir_i is a measurement residual, Σi\Sigma_i its covariance, and ρi\rho_i a potentially robust penalty. The chosen loss function affects how strongly large residuals influence the solution.

Solvers such as Gauss–Newton and Levenberg–Marquardt repeatedly linearize residuals and solve sparse linear systems. Their efficiency depends on graph connectivity, variable ordering, and the treatment of eliminated variables. Bundle adjustment jointly refines camera poses and observed three-dimensional structure using image-reprojection errors. Pose-graph optimization instead operates mainly on relative pose constraints, often after local geometry has already been estimated.

Real-time operation usually limits the active problem. Sliding-window estimators retain recent states and marginalize older ones; incremental smoothers reuse previous calculations when new factors arrive. Marginalization preserves information approximately around a linearization point and can introduce dense dependencies among surviving variables. It therefore has consequences beyond memory reduction.

Numerical success is not equivalent to physical correctness. Nonlinear objectives may have local minima, and the Gaussian approximation can poorly describe ambiguous or multimodal situations. Reliable initialization, meaningful noise models, observability-aware treatment of states, and robust constraints are as important as solver speed.

Loop closure, relocalization, and global consistency

Short-range motion estimates accumulate error. A small persistent heading error can gradually bend an estimated straight corridor, while successive scale errors can distort a visual reconstruction. Loop closure occurs when a system recognizes that it has returned to a previously mapped place and adds constraints connecting the current trajectory to that earlier region. These constraints can substantially reduce accumulated drift.

A typical loop-closure pipeline has two stages. Place recognition first retrieves candidate past observations using appearance descriptors, geometric signatures, or learned representations. Geometric verification then checks whether the candidate and current data support a consistent relative transformation. Visual systems may match local features and estimate camera geometry; lidar systems may register scans or submaps. Retrieval is a hypothesis-generation step, not sufficient proof that two places are identical.

Perceptual aliasing makes this distinction essential. Two warehouse aisles may contain almost identical shelving, and several floors of a building may share the same layout. Conversely, the same place can look very different after a change in illumination, season, or viewing direction. A false loop closure can pull distant parts of a map into an incorrect alignment. Robust kernels, switchable constraints, and consistency checks are used to reduce this risk, but no single safeguard resolves every ambiguity.

After verification, the new constraint is incorporated into global optimization. The correction generally distributes across multiple poses according to the measurement structure and uncertainties; it is not simply a rigid shift of the current estimate. In monocular systems with scale drift, some loop-correction methods use similarity transformations that include a scale factor, rather than only rotation and translation. Metric systems have different constraints because their scale is already anchored by sensor geometry or other measurements.

Relocalization addresses a related but distinct problem: recovering a pose after tracking has failed or when a device starts within an existing map. A system can relocalize without having completed a loop during its current trajectory. It may also maintain short-term odometry while delaying uncertain loop decisions, preserving locally smooth tracking without immediately committing to a global correction.

The distinction matters for downstream software. A controller often needs a locally continuous frame to avoid sudden jumps, while a navigation system needs a globally corrected frame. Practical architectures may maintain both and update the transformation between them. Applications that assume a single immutable coordinate frame can malfunction when a legitimate global correction changes previously reported positions.

Loop closure is valuable but not guaranteed. An exploration route may never revisit a place, or revisits may be unrecognizable. In those cases, accuracy depends on local sensing, trajectory geometry, external anchors, and the estimator’s ability to represent uncertainty without claiming unsupported global consistency.

Applications, performance measures, and benchmarks

SLAM supports mobile robots in warehouses, factories, mines, inspection sites, and domestic environments. A robot can construct a map during exploration and later localize against it, although many operational systems separate mapping sessions from routine deployment. Augmented reality uses related estimation to keep virtual content aligned with physical surfaces. Small aerial vehicles and handheld devices place particular emphasis on low latency, power consumption, and robustness to rapid motion.

The requirements vary markedly. A cleaning robot may need a compact traversability map rather than a photorealistic reconstruction. An inspection platform may need repeatable alignment of measurements over several visits. An aerial vehicle must often account for fast motion and changing visual conditions, whereas a ground vehicle may exploit a stable motion plane or wheel measurements. These differences make claims that one method is universally “best” misleading without a specified operating regime.

Evaluation commonly separates global trajectory agreement from local motion accuracy. Absolute trajectory error measures disagreement with a reference trajectory after an appropriate alignment. Relative pose error measures disagreement in motion between pairs of times, exposing drift over selected intervals. Alignment must respect observability: a monocular estimate may require similarity alignment to compare shape, whereas freely rescaling a metric estimate can conceal a meaningful error.

Mapping quality requires additional measures. Surface accuracy concerns distance from reconstructed surfaces to reference geometry; completeness concerns how much reference structure was recovered. Occupancy quality, object consistency, and the preservation of narrow passages may matter more for navigation. A low trajectory error does not guarantee a complete map, and a smooth-looking map does not establish measurement accuracy.

Established datasets have made comparisons more reproducible. KITTI, introduced in 2012, provides outdoor driving data and odometry benchmarks. The TUM RGB-D benchmark, also published in 2012, supplies color and depth sequences with reference trajectories for indoor evaluation. The EuRoC micro aerial vehicle datasets, published in 2016, provide synchronized stereo imagery, inertial data, and reference motion measurements. Each emphasizes particular sensor configurations and environments rather than the entire SLAM problem.

A meaningful comparison also reports tracking failures, initialization behavior, recovery, runtime, memory, and hardware. Average error over successful sequences can hide catastrophic failures. Offline processing may obtain results unavailable under real-time deadlines. Parameter tuning on benchmark sequences can further exaggerate apparent generalization to new environments. Changing the scene, sensor quality, or motion pattern creates distribution shift, making cross-dataset testing and carefully controlled experimental protocols important complements to leaderboard results.

Learning, persistent mapping, and unresolved problems

Machine learning increasingly contributes to SLAM through feature detection, correspondence estimation, depth prediction, place recognition, and semantic interpretation. Learned components can capture regularities that are difficult to encode manually, including useful image representations under changing appearance. However, a learned depth estimate is not equivalent to a direct metric measurement, and a confident place-recognition score is not a geometric proof of location.

Many systems combine learned perception with explicit geometric optimization. This division allows a model to propose correspondences or priors while an estimator enforces multi-view consistency and measurement relationships. More integrated approaches learn substantial parts of the tracking and mapping process. Their evaluation must distinguish genuine motion information from priors tied to the environments represented in training data, especially when scale, scene layout, or appearance changes.

Neural map representations extend the range of what a SLAM map can store. Methods based on neural radiance fields encode scene appearance and geometry in a function that can synthesize views. Other systems use learned implicit distance fields or explicit collections of three-dimensional Gaussian primitives. These approaches can produce detailed renderings, but rendering quality, tracking reliability, memory efficiency, and geometric accuracy remain separate objectives. Exposure changes, moving objects, and uncertain camera poses can be absorbed into a reconstruction in ways that obscure physical error.

Long-term operation poses challenges beyond reconstructing a static environment once. Furniture moves, vegetation grows, lighting changes, and construction alters permanent structure. A persistent map must distinguish transient objects from durable geometry and decide when old information should be revised rather than merely averaged with new observations. Maintaining several valid appearances or configurations may be more appropriate than forcing every visit into one unchanging model.

Cooperative SLAM distributes estimation across multiple robots or devices. Their maps may initially have unrelated coordinate frames, and shared observations can introduce correlations that are difficult to track. Communication bandwidth, intermittent connectivity, and duplicated information complicate distributed computing approaches. Merging maps requires both reliable inter-device associations and careful treatment of uncertainty; exchanging confident-looking pose estimates alone is insufficient.

Other open problems concern failure detection and assurance. A system should identify when its geometry is weakly constrained, when observations contradict its static-world assumptions, or when calibration has changed. Small residuals can coexist with systematic error, particularly when the model omits a relevant sensor effect. Formal performance guarantees remain limited for full systems operating in unrestricted environments.

SLAM also creates detailed records of spaces and movement. Indoor geometry, recognizable objects, and repeated trajectories can reveal sensitive information even when raw images are not retained. This connects map design and sharing to data privacy and cybersecurity, particularly for collaborative maps and cloud-supported spatial services.