Publication bias is the systematic dependence of the dissemination or availability of research on the nature of its findings. Studies reporting statistically significant, novel, favorable, or otherwise striking results may be more likely to become publicly accessible than studies reporting inconclusive or unfavorable results. Consequently, the published literature can differ systematically from the research actually conducted. Publication bias is a form of selection bias operating on the evidence base rather than necessarily on the participants within an individual study.
The problem is important because scientific conclusions usually depend on accumulated evidence. A meta-analysis may combine individually well-conducted studies yet produce a misleading estimate if the available studies are selected by their results. Publication bias can exaggerate benefits, obscure harms, create apparent effects where none exist, or conceal genuine effects that are unwelcome to particular audiences. Its direction depends on what kinds of findings are favored, not on a universal preference for numerically positive estimates.
The subject connects research institutions, editorial practices, statistical inference, and the economics of scientific communication. Understanding it requires distinguishing missing studies from missing results within studies, examining how selection changes observed distributions, and assessing both statistical diagnostics and institutional measures intended to make research accessible independently of its outcome.
Definition and boundaries
In its narrow sense, publication bias occurs when an entire study’s probability of publication depends on its results. The familiar “file drawer” example is a completed experiment that remains unpublished because its investigators consider the findings uninteresting. However, selection can operate before submission, during editorial assessment, or after publication through indexing and accessibility. It is therefore not simply a synonym for journal editors rejecting null findings. If investigators never prepare a manuscript, editorial decisions cannot account for that study’s absence.
The defining feature is dependence on results. A study may remain unpublished because its records were destroyed, its investigators changed employment, or its methods were inadequate. Such nonpublication does not necessarily constitute publication bias, although these circumstances can become result-dependent. For example, investigators may devote scarce time to writing up a favorable result while abandoning an equally rigorous inconclusive study. Conversely, an unpublished study with poor methods is not automatically evidence that the published literature is biased. What matters is whether the missingness systematically changes the evidence relevant to a research question.
Publication bias is closely related to, but distinguishable from, selective outcome reporting. In the latter, a study is accessible, but particular outcomes, measurement times, subgroups, or analyses are omitted or emphasized according to their findings. A trial might publish its symptom score while withholding an unfavorable functional outcome. Reporting an unplanned analysis as though it had been specified in advance creates another discrepancy between the research undertaken and the research presented. These processes are often grouped under the broader heading of reporting biases, but they require different investigative methods.
Other distinctions concern the timing and visibility of dissemination. Time-lag bias arises when favorable findings appear sooner than unfavorable ones, making the evidence base temporarily unrepresentative even if all studies are eventually published. Language bias occurs when the language of publication is related to results and evidence searches cover only selected languages. Citation bias concerns preferential citation rather than initial publication. Multiple publications of a favorable study can also inflate its apparent contribution unless reports are linked to the same underlying investigation.
“Positive,” “negative,” and “null” findings require care. A result favorable to a hypothesis may involve a reduction, such as lower mortality, rather than a positive numerical effect. A statistically nonsignificant result is not proof of no effect: its confidence interval may include both important benefit and important harm. Likewise, a p-value below a conventional threshold does not establish that an effect is large, practically important, or reproducible. Publication bias often exploits these distinctions by treating statistical significance as a categorical measure of scientific worth.
Historical development and empirical evidence
Concern about preferential reporting predates modern evidence synthesis. In 1959, Theodore Sterling examined the prevalence of statistically significant findings in psychology journals and argued that publication practices could produce a distorted record. The problem gained a memorable formulation in Robert Rosenthal’s 1979 discussion of the “file drawer problem”: journals could contain the small fraction of studies rejecting a null hypothesis, while the much larger collection of unsuccessful tests remained out of view. These arguments shifted attention from whether published studies were individually correct to whether the published collection represented the research population.
The development of quantitative research synthesis made that distinction increasingly consequential. Combining studies increases precision only for the evidence actually included; it does not automatically restore missing findings. Rosenthal’s fail-safe-number approach asked how many additional null studies would be required to make a combined result nonsignificant. Later work showed why this calculation was not a general solution: unpublished studies need not have exactly null effects, and the statistical significance of a pooled result is not the same question as the magnitude or practical importance of the underlying effect.
Stronger empirical evidence became possible when researchers identified studies before their publication outcomes were known. Cohorts assembled from ethics committees, funding bodies, conference submissions, and regulatory records made it possible to compare the subsequent dissemination of studies with different findings. Such designs avoid relying exclusively on the visible literature to infer what is invisible. They also reveal that publication is a process with several stages: completion, preparation of a report, submission, acceptance, and indexing can each introduce selection.
A prominent example is Erick Turner and colleagues’ 2008 comparison of antidepressant trials reviewed by the United States Food and Drug Administration with their journal publications. The investigation covered 74 studies involving 12,564 participants. Thirty-one percent of the studies were unpublished. The journal literature made 94% of the trials appear positive, whereas the FDA assessments classified 51% as positive. Comparing the published evidence with the FDA dataset indicated an overall increase of 32% in the estimated effect size. The discrepancy involved both missing studies and presentations of some studies that differed from their regulatory assessments.
Evidence outside clinical research has also exploited independently documented study cohorts. Annie Franco, Neil Malhotra, and Gabor Simonovits examined 221 survey-based experiments conducted through the Time-sharing Experiments for the Social Sciences program in a study published in 2014. Their analysis linked findings to subsequent write-up and publication, showing that selection could arise through investigators’ decisions about whether to prepare a paper. Such evidence supports a general mechanism without establishing a single universal publication-bias rate. Rates depend on the field, historical period, research design, dissemination system, and definition of a publishable result.
Mechanisms and incentives across the research process
Publication bias often begins with investigators’ expectations about what other actors will value. Preparing a manuscript requires time, and researchers may prioritize results believed likely to attract citations, journal acceptance, or further funding. A technically sound null result may be interpreted as a failed project rather than as information about the phenomenon under study. This response is particularly consequential when low statistical power makes statistically significant findings uncommon: the studies selected for dissemination can then be an unusually extreme subset of all completed studies.
Editors and reviewers introduce another layer of selection. Journals may prioritize novelty, perceived importance, or a clear narrative. These criteria need not explicitly mention statistical significance to favor it. A surprising association can appear more newsworthy than a precise estimate close to zero, while a complicated pattern of inconsistent outcomes may seem less publishable than a simple affirmative claim. Nevertheless, observed differences between published and unpublished studies do not by themselves establish editorial discrimination. Submission decisions and differences in study quality must also be considered.
Sponsors and organizations can influence dissemination through control of data, contractual arrangements, and decisions about which analyses to develop. Commercially favorable findings may receive greater support for publication than commercially unfavorable findings. Similar incentives can arise outside commercial settings when findings support an established theory, a policy commitment, or an institution’s reputation. These mechanisms are not equivalent to fabrication. Every reported numerical result could be calculated correctly while the available collection remains systematically unrepresentative.
Selection also interacts with p-hacking, the use of analytical flexibility to obtain a preferred statistical result. Choices about covariates, exclusion rules, outcome transformations, subgroups, and stopping points can generate many possible analyses. If only a favorable analysis is reported, the nominal error properties of the stated test need not describe the actual procedure. Publication bias then operates on an evidence base already selected within studies. These processes are conceptually distinct: publication bias can occur when every analysis was prespecified, and p-hacking can occur in a study that would have been published regardless of its findings.
The broader incentive system helps explain why selection can persist without a central decision to suppress evidence. Authors anticipate editors, editors anticipate readers, and institutions reward indicators partly produced by these interactions. Replications, routine measurements, and careful null findings may receive less recognition than novel claims. Yet a preference for significance is not universal. A dramatic contradiction of an accepted theory, a clear demonstration of harm, or an unexpected absence of benefit may itself be especially publishable. The relevant selection mechanism must therefore be specified in relation to the field and question rather than assumed from the sign of an estimate.
Statistical representation of result-dependent selection
A simple model separates the generation of study results from their availability. Let denote a study’s estimated effect size, its standard error, and an indicator that the result is observed in the evidence base. If the unselected density is , where is the underlying effect, and the selection probability is , then the observed density is
The denominator normalizes the selected distribution. When is constant with respect to results, selection does not distort this conditional distribution. When favors particular estimates or significance levels, the observed distribution differs from the distribution generated by the studies themselves.
Consider an idealized example with and complete selection for results satisfying . Writing and for the density and cumulative distribution function of the standard normal distribution, respectively, the selected mean is
This is the mean of a truncated normal distribution. With and , the mean among selected estimates is approximately , despite the true effect being zero. The threshold corresponds to retaining only significant results in the positive direction under a conventional two-sided 5% test. It is an illustrative selection rule, not a description of all publishing systems.
Direction matters. If selection favors statistical significance equally in either direction, the observed distribution under a true zero effect remains symmetric. Its signed mean can remain zero even though its tails are overrepresented and absolute effect sizes are exaggerated. Thus, publication bias does not invariably produce a positive average estimate. It can instead distort dispersion, the frequency of apparently convincing findings, or the estimated distribution of effects across settings.
For a nonzero effect, selection commonly creates a winner’s-curse pattern: estimates must overshoot the underlying effect to pass a publication threshold, especially when uncertainty is large. Larger studies can become significant with modest estimates, whereas smaller studies require more extreme estimates. This mechanism can generate an association between effect size and precision. It also explains why the published effect may decline when subsequent studies are larger or disseminated under less selective conditions.
The central difficulty is identification. The observed literature reveals the product of a research-generating process and a selection process, not each process separately. A large true effect with weak selection can resemble a smaller true effect with strong selection. Variation in underlying effects creates further ambiguity. Statistical correction therefore requires assumptions about selection, additional information about missing research, or both; the published estimates alone generally do not determine a unique correction.
Consequences for evidence synthesis and scientific inference
A conventional fixed-effect meta-analysis often combines study estimates using inverse-variance weights,
Under its assumptions, more precise estimates receive greater weight. But if studies enter the dataset partly because of their results, this procedure summarizes a selected collection. A narrow interval around the pooled estimate can indicate high precision about the wrong target. Increasing the number of selectively available studies need not remove the distortion; it can instead produce stronger apparent certainty.
Random-effects models allow underlying effects to vary across studies, but they do not automatically account for missing studies. In a common formulation, weights involve , where is the between-study variance. Introducing changes relative weighting and often gives small studies more influence than they receive under fixed-effect weighting. If small studies are especially selected for large effects, the random-effects estimate can consequently be larger. This is not an inherent defect of random-effects models: it reflects a mismatch between their assumptions and a selected dataset.
Publication bias can also alter the apparent pattern of effect modification. Suppose studies in one population are typically small, while studies in another are large. If small studies with modest findings are missing, the first population may appear more responsive even when population membership has no genuine modifying role. Study-level comparisons can then confuse selection with substantive differences. Such patterns complicate causal inference because an association between a study characteristic and its reported effect need not represent a causal difference between settings.
In applied research, the consequences extend beyond an inflated average. Missing adverse-event results can produce an incomplete account of harms. Missing comparisons can alter rankings among competing interventions. In network meta-analysis, where indirect comparisons connect treatments through shared comparators, absent studies or outcomes can affect several estimated contrasts. In econometrics, selective reporting of favorable coefficients may influence conclusions about policy effects. In laboratory research, the same process can direct subsequent experiments toward relationships whose published magnitude is difficult to reproduce.
A biased literature also changes the allocation of research effort. Overstated initial effects can lead investigators to plan replications with inadequate sample sizes, because those studies are powered for an exaggerated effect. The resulting mixture of failures and selectively successful replications may prolong uncertainty. Conversely, making well-conducted null results accessible can reveal that a proposed effect is small, context-dependent, or insufficiently measured. None of these consequences implies that every irreproducible finding results from publication bias: measurement problems, confounding, procedural differences, and ordinary sampling variation can produce similar observations.
Detecting missing evidence and interpreting funnel plots
The strongest assessments begin by looking beyond journal articles. A systematic review can compare publications with trial registrations, protocols, funding records, regulatory submissions, dissertations, conference abstracts, and reports obtained from investigators. These sources can establish that a study existed and sometimes recover results. Comparing a protocol with a final article can identify omitted outcomes or changes in outcome status. Such evidence is more direct than inferring missing studies solely from the shape of the published effect-size distribution, although records can themselves be incomplete or retrospectively altered.
The funnel plot is the best-known graphical diagnostic. It commonly places study effect estimates on the horizontal axis and standard errors, often with the direction reversed, on the vertical axis. Large, precise studies appear near the top, while small studies scatter more widely near the bottom. Under suitable assumptions, results form an approximately symmetric funnel around a common effect. If small studies with unfavorable results are absent, one side of the lower funnel may be sparsely populated.
Asymmetry is not specific to publication bias. Small studies may enroll different participants, use different doses, employ less rigorous methods, or measure outcomes differently from large studies. The effect estimate and its standard error may also be mathematically related for some effect measures. Chance produces substantial irregularity when few studies are available. A symmetric plot, meanwhile, does not establish absence of selection: large and small studies may be selected similarly, or opposing mechanisms may conceal one another.
Egger and colleagues introduced a widely used regression test in 1997. In a common version, standardized estimates are regressed on precision:
An intercept differing from zero is interpreted as evidence of asymmetry under the model’s assumptions. The test is better described as a test for small-study effects than as a direct test of publication bias. Its performance depends on the effect measure, between-study differences, and number of studies. A frequently used caution is that asymmetry tests are generally uninformative with fewer than about ten studies; even above that number, power may remain limited.
Contour-enhanced funnel plots add regions corresponding to statistical significance thresholds. Missingness concentrated where results would be nonsignificant is compatible with significance-driven selection, but it is not proof of that mechanism. Analyses of p-value distributions provide another approach. Methods such as p-curve examine the distribution of significant p-values under specified assumptions to investigate evidential value and selective reporting. They do not by themselves enumerate missing studies or reconstruct the complete evidence base. Diagnostic results are therefore interpreted alongside study characteristics and documentary evidence rather than treated as binary declarations that a literature is biased or unbiased.
Adjustment methods and sensitivity analysis
Statistical adjustment attempts to estimate what evidence synthesis might show under alternative assumptions about missing results. The most familiar graphical procedure is trim-and-fill, developed by Sue Duval and Richard Tweedie in 2000. It identifies asymmetry, temporarily removes studies to estimate a center, and then imputes mirror-image studies to produce a more symmetric funnel. Its appeal lies in its simplicity and visual interpretation. Its limitation is that symmetry is an assumption about the missing-data process, not a fact established by the observed studies.
If genuine differences between small and large studies create asymmetry, trim-and-fill can impute studies that never existed. Conversely, selection may leave a roughly symmetric distribution and escape adjustment. An adjusted estimate should therefore not be interpreted as the recovered truth. The number of “filled” studies is a model-dependent construction, not a count of documented unpublished investigations. Similar caution applies to fail-safe numbers: a large number of hypothetical null studies may be required to cross a significance threshold even when the published effect is already substantially exaggerated.
Selection models describe dissemination more explicitly. A weight-function model can assign different observation probabilities to intervals of p-values, such as values below and above 0.05, and estimate the effect distribution conditional on that selection. Other models relate publication to study precision and a latent component correlated with the observed estimate. The Copas selection model is a prominent example. Such approaches connect directly to the statistical mechanism of selection, but their estimates may be unstable when the literature is small or the selection process is only weakly identified.
Regression-based adjustments take another route. PET and PEESE procedures model relationships between estimated effects and their standard errors or sampling variances, extrapolating toward the estimate expected as sampling uncertainty approaches zero. Their rationale is that significance-driven selection often affects imprecise studies more strongly. However, the same relationship can reflect real differences between studies, and extrapolation beyond the observed precision range can be fragile. These methods are not interchangeable with finding and incorporating previously unavailable data.
Bayesian approaches can combine effect models with selection models and express uncertainty through a prior distribution and the resulting posterior. Model averaging can distribute weight across hypotheses about whether an effect exists and whether selection is present. Such methods make assumptions explicit, but the posterior can depend strongly on prior choices when the data provide little information about selection. A technically elaborate model cannot eliminate the absence of identifying information.
For these reasons, sensitivity analysis is often more informative than presenting one corrected number. Analyses can vary the relative publication probabilities of significant and nonsignificant findings, the assumed outcomes of known missing studies, or the strength of association between availability and effect size. The resulting range shows which conclusions survive plausible selection mechanisms. Documentary evidence constrains these scenarios: knowing that several large completed studies are missing provides more specific information than merely observing an asymmetric funnel.
Registration, reporting systems, and institutional prevention
Prevention aims to separate a study’s existence and dissemination from the attractiveness of its findings. Preregistration creates a time-stamped record of research questions, outcomes, and analysis plans before relevant results are known. In clinical research, a clinical trial registry can also document recruitment status, interventions, and planned enrollment. These records make it harder for completed studies to disappear without trace and allow reviewers to compare planned outcomes with published ones. Registration does not guarantee completion, accurate entries, or public results, but it creates an external record against which reporting can be assessed.
International policy has developed through several overlapping systems. The International Committee of Medical Journal Editors announced in 2004 that prospective trial registration would become a condition of consideration for publication, with implementation beginning in 2005. The World Health Organization’s International Clinical Trials Registry Platform supports access to records from participating registries. In the United States, the Food and Drug Administration Amendments Act of 2007 established registration and results-reporting requirements for defined categories of applicable clinical trials; a final rule issued in 2016 clarified and expanded implementation. These requirements differ in scope, and registration should not be equated with complete results disclosure.
Registered Reports change the publication decision more directly. Journals assess the importance of a question and the adequacy of proposed methods before outcomes are known, and may offer in-principle acceptance conditional on completing the approved work and meeting reporting requirements. The format became established through initiatives including the journal Cortex in 2013. Because acceptance is not supposed to depend on whether the final result is statistically significant, it addresses a principal incentive behind result-dependent publication. Its scope nevertheless depends on which projects enter the process and how compliance is evaluated.
Results repositories and regulatory disclosure systems provide routes that do not require a conventional journal article. Structured results can preserve basic numerical findings even when investigators do not produce a narrative manuscript. Preprints reduce some editorial barriers and can shorten dissemination delays, although investigators still choose which studies to post. Sharing data, analysis code, and protocols supports scrutiny of reported analyses, but data availability alone does not ensure that the existence of every study or outcome is discoverable. These practices address different points in the reporting process rather than constituting interchangeable solutions.
Institutional evaluation can also distinguish the quality of a research process from whether it yields a striking result. Funding decisions, hiring practices, and journal policies influence the opportunity costs of publishing replications and null findings. Within open science, an important operational distinction is between merely inviting all results and creating durable systems that record projects, track completion, require results, and preserve access. Assessments of these systems therefore examine prospective registration, completeness of outcome reporting, delays in dissemination, and discrepancies between registrations and final reports—not only the number of papers ultimately published.