Experimental design is the systematic planning of an experiment: selecting treatments, assigning them to experimental units, and organizing measurements so that the resulting data answer a defined question. It connects statistics with practical investigation by determining which effects can be estimated and how precisely. Its objectives include comparing alternatives, identifying influential variables, and developing models of processes. Design therefore precedes data collection rather than merely specifying how observations will later be analyzed. (itl.nist.gov)
Historical development
Modern statistical experimental design developed substantially through agricultural research. Ronald Fisher joined Rothamsted Experimental Station in 1919, where long-running crop trials provided problems involving environmental variability and treatment comparisons. His work helped establish randomization, replication, and structured allocation as foundations of experimental investigation. These developments connected the physical arrangement of experiments with formal methods for estimating effects and experimental error. (rothamsted.ac.uk)
Units, factors, and responses
A factor is a variable whose influence is investigated; its levels are the settings or categories used. A treatment may be one factor level or a combination of levels across several factors. The response is the measured outcome. For example, an experiment might vary processing temperature and material composition while measuring product strength. Objectives determine which variables and levels belong in the design. (itl.nist.gov)
An experimental unit is the entity to which a treatment is independently applied. It need not coincide with the entity measured. If different water treatments are assigned to aquaria, fish measured within an aquarium are sampling units, while the aquarium is the experimental unit. Treating those fish as independent treatment replicates creates pseudoreplication, artificially increasing the apparent information available for comparing treatments and inflating error degrees of freedom. (online.stat.psu.edu)
Fundamental principles
Randomization assigns treatments, or orders experimental runs, through a specified chance mechanism. It reduces systematic associations between treatments and uncontrolled influences and supplies a basis for statistical comparisons. Random assignment strengthens causal inference, but differs from randomly sampling a population: assignment concerns treatment comparability, whereas sampling concerns representativeness. A randomized experiment does not automatically justify conclusions about every population or setting. (online.stat.psu.edu)
Replication applies treatments to multiple experimental units. It provides information about experimental variability and can improve precision. Repeated readings from a single unit may improve measurement precision, but do not replace independent treatment applications. The distinction is essential when calculating sample sizes and evaluating uncertainty. (online.stat.psu.edu)
Blocking groups units or runs according to important nuisance variables, such as production batch, operator, or location. Treatments are compared within relatively homogeneous blocks, reducing unwanted contributions to error variance. Randomization then operates within the restrictions imposed by the block structure. Blocking addresses identified sources of variation; it does not eliminate every possible source of confounding. (itl.nist.gov)
Major design families
A completely randomized design assigns treatments across all available experimental units without block restrictions. A randomized complete block design includes every treatment within each block, with treatment allocation randomized inside blocks. These arrangements concern allocation and can be combined with different treatment structures. (online.stat.psu.edu)
A factorial design investigates several factors together. A full factorial includes every combination of the selected levels; with (k) factors at two levels each, it has (2^k) treatment combinations before replication. It permits estimation of main effects and interactions—situations in which the effect of one factor depends on the level of another. Varying factors only one at a time does not generally reveal these joint effects. (itl.nist.gov)
A fractional factorial design uses a selected subset of combinations to reduce experimental cost. This economy introduces aliasing: certain effects cannot be distinguished without additional assumptions or observations. Screening designs emphasize identifying influential factors, while response surface designs support models containing curvature, often through quadratic terms. Experiments can be augmented with further runs when initial results indicate that additional effects require investigation. (itl.nist.gov)
A split-plot design accommodates factors applied at different scales or with different practical constraints. A hard-to-change setting may be assigned to whole plots, while another factor varies among subplots. Because the design contains more than one size of experimental unit, its analysis must distinguish the corresponding sources of variation. (itl.nist.gov)
Precision and statistical analysis
Sample-size planning relates the effect of interest to expected variability and the intended comparison. Statistical power is the probability that a specified hypothesis test rejects its null hypothesis under a particular alternative. Power calculations can determine the sample size needed to detect a specified difference; they depend on assumptions about effect magnitude, variability, and the testing procedure. (online.stat.psu.edu)
The analysis must reflect the design. Analysis of variance and linear regression can represent treatment effects, blocks, and interactions. A model for a full factorial may include all interaction orders, whereas fractional designs require explicit attention to their alias structure. Residual analysis assesses whether the fitted model adequately represents the observations. Statistical modeling cannot recover separately identifiable effects when the design provides no information to distinguish them. (itl.nist.gov)
Optimal designs and practical constraints
Optimal experimental design formulates the selection of runs as a mathematical optimization problem. A-optimality minimizes average parameter-estimation variance, while D-optimality minimizes generalized variance. Such methods accommodate constraints on the number of runs or allowable settings that standard designs may not handle. “Optimal” is conditional on the chosen model, candidate settings, and criterion: a design favorable for estimating parameters may be less favorable for predicting responses. Computational searches also need not guarantee the global optimum. (itl.nist.gov)