{
  "id": "critical-assessment-of-protein-structure-prediction",
  "title": "Critical Assessment of Protein Structure Prediction",
  "lang": "en",
  "source_lang": "en",
  "summary": "The Critical Assessment of Protein Structure Prediction (CASP) is a recurring international experiment that independently evaluates computational methods against experimentally determined molecular structures withheld from predictors.",
  "category": "science",
  "subcategory": "biology",
  "type": "event",
  "aliases": [
    "CASP",
    "Critical Assessment of Techniques for Protein Structure Prediction"
  ],
  "generated_at": "2026-11-03",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/critical-assessment-of-protein-structure-prediction/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/critical-assessment-of-protein-structure-prediction/",
      "json": "https://aiwiki.page/zh/critical-assessment-of-protein-structure-prediction.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "The Critical Assessment of Protein Structure Prediction (CASP) is an international, community-organized experiment for evaluating methods that predict the three-dimensional structures of proteins and, in some editions, other biological molecules and their complexes. Established in 1994 and generally held every two years, CASP compares submitted predictions with experimentally determined structures that were not publicly available when the predictions were made. Its central contribution is a prospective, blind assessment: methods must demonstrate their performance on previously unseen targets rather than only on examples selected by their developers."
    },
    {
      "heading": "Scientific purpose",
      "text": "A protein’s sequence of amino acids constrains its three-dimensional structure, which in turn helps determine its biochemical activity and interactions. Predicting that structure from sequence is a central problem in molecular biology. Experimental structure determination can be demanding, while sequence data are available for far more proteins than have been structurally characterized.\n\nProtein structure prediction is related to, but distinct from, the problem of protein folding. A prediction method may accurately produce a final structure without reproducing the physical pathway or timescale by which a protein folds.\n\nBefore CASP, comparisons between prediction methods were often complicated by different test sets, inconsistent scoring, and the possibility that developers had already encountered information about the structures being predicted. CASP created a shared assessment framework with common targets, deadlines, and independent evaluation.\n\nIt is often described as a competition, but its broader purpose is methodological assessment. Rankings help identify strong approaches, while analysis of successes and failures reveals which structural problems remain difficult."
    },
    {
      "heading": "How the assessment works",
      "text": "Target collection and release\n\nExperimental researchers contribute structures that have been determined, or are expected to be determined, but have not yet been publicly released. These become assessment targets. Reference structures commonly come from X-ray crystallography, nuclear magnetic resonance, or cryo-electron microscopy.\n\nPredictors receive the target sequence and the information permitted for the relevant task. Depending on the category, this may include component sequences, chemical identities, or specified experimental information. They must submit predictions before a deadline and before the reference structure becomes publicly available.\n\nNot every released target necessarily contributes to the final assessment. Targets can be excluded if, for example, the experimental structure is unavailable in time or public disclosure compromises the blind evaluation.\n\nPrediction submission\n\nCASP has distinguished automated server predictions from submissions that may involve human intervention. Automated servers operate within prescribed submission rules and deadlines. Human-assisted groups may combine computational outputs, inspect candidate structures, or select and refine models.\n\nThese modes answer different questions. Server assessment tests an automated workflow, whereas human-assisted assessment can reveal what is achievable with expert interpretation and additional effort. A group’s preferred submitted model is also informative: generating a good candidate is not equivalent to reliably identifying it as the best prediction.\n\nIndependent evaluation\n\nAfter submissions close, assessors compare predictions with the experimental structures. Predictor identities can be concealed during assessment to reduce reputational bias.\n\nAssessors examine numerical scores alongside structural features and target difficulty. A single aggregate ranking cannot show whether a method succeeded on easy targets, captured an unfamiliar fold, placed separate domains correctly, or reconstructed an interaction interface.\n\nThe cycle concludes with community discussion and publication of assessment reports. These reports document performance, methodological developments, and unresolved problems."
    },
    {
      "heading": "Prediction tasks and categories",
      "text": "CASP’s categories have changed as methods and scientific priorities have developed. Categories from different editions should therefore not be treated as permanently fixed or directly interchangeable.\n\nIndividual protein structures\n\nA traditional distinction separates template-based modeling from free modeling.\n\nTemplate-based modeling uses experimentally determined structures of related proteins as guides. Difficulty depends not only on the existence of a template but also on recognizing the relationship, aligning sequences correctly, and modeling regions that differ from the template.\n\nFree modeling addresses targets for which useful structural templates are absent or difficult to identify. Historically, this was especially challenging because methods had to infer overall structure with much weaker direct structural guidance.\n\nThe boundary has become less clear as machine-learning systems combine sequence information, evolutionary relationships, and knowledge learned from large structural datasets. CASP categories describe assessment conditions and target difficulty, not necessarily a clean division between underlying algorithms.\n\nProtein assemblies and molecular complexes\n\nMany proteins function as assemblies rather than isolated chains. Predicting a complex requires identifying how its components are arranged, not merely predicting the shape of each component.\n\nAssessment can examine subunit folds, relative orientation, contact interfaces, and overall assembly architecture. A model may contain accurate individual chains yet reconstruct the complex incorrectly.\n\nInteractions involving antibodies and their binding partners are one example of the difficulty: successful prediction requires accurate recognition of the binding surface and its geometry, not just accurate component structures.\n\nAdditional tasks\n\nDifferent CASP editions have included or developed assessments of:\n\n- Model refinement: improving an existing structural model.\n- Model accuracy estimation: predicting which models or regions are likely to be correct without access to the experimental answer.\n- Contact and distance prediction: estimating spatial relationships between residues.\n- RNA structure prediction: extending blind assessment to RNA, whose structural organization differs from that of proteins.\n- Protein–ligand modeling: predicting structures involving bound ligands, including the placement of a ligand within a binding site.\n\nThese tasks require distinct evaluation criteria. A strong protein backbone prediction does not automatically imply accurate ligand geometry or RNA folding."
    },
    {
      "heading": "Measures of structural accuracy",
      "text": "CASP uses multiple metrics because structural accuracy has several dimensions. A score that describes the overall fold may conceal errors in an active site, while a local score may not fully capture the relative placement of large structural units.\n\nRoot-mean-square deviation\n\nThe root-mean-square deviation (RMSD) measures coordinate differences between corresponding atoms, usually after structural superposition:\n\n\\[\\mathrm{RMSD} = \\sqrt{\\frac{1}{N}\\sum_{i=1}^{N} \\left\\|\\mathbf{x}_i-\\mathbf{y}_i\\right\\|^2},\\]\n\nwhere \\(N\\) is the number of matched atoms and \\(\\mathbf{x}_i\\) and \\(\\mathbf{y}_i\\) are their coordinates in the superposed structures.\n\nRMSD is intuitive but can be dominated by a small number of large deviations. A flexible loop or displaced domain can produce a large value even when much of a model is accurate.\n\nGlobal Distance Test\n\nThe Global Distance Test total score, usually called GDT_TS, summarizes the proportions of corresponding residues that can be brought within distance thresholds of 1, 2, 4, and 8 ångströms through structural superposition. Their percentages are averaged, giving a score conventionally reported on a 0–100 scale.\n\nGDT_TS is less sensitive than RMSD to a few very large coordinate errors and became a prominent measure in CASP assessments. The related GDT_HA measure uses tighter thresholds to emphasize high-accuracy modeling.\n\nLocal accuracy\n\nThe local Distance Difference Test (lDDT) evaluates how well local interatomic distances are preserved without requiring a single global structural superposition. It is useful when local structural quality must be distinguished from the relative motion or placement of larger units.\n\nOther measures evaluate side-chain placement, stereochemistry, residue contacts, and molecular interfaces. Complex prediction requires interface-sensitive scores because correct monomer structures can coexist with an incorrect assembly.\n\nAccuracy versus confidence\n\nA model’s experimentally assessed accuracy is distinct from its predicted confidence. Confidence measures estimate expected reliability without knowing the reference structure.\n\nFor example, AlphaFold uses predicted lDDT, or pLDDT, as a local confidence measure. This is not the same as lDDT calculated by comparing a prediction with an experimental structure. Confidence assessment asks whether such estimates reliably distinguish correct and incorrect regions."
    },
    {
      "heading": "Historical development",
      "text": "Early assessments\n\nThe first CASP experiment took place in 1994. Early editions established blind assessment as a practical community standard and exposed the gap between successful modeling of proteins with close structural relatives and the much harder prediction of unfamiliar folds.\n\nThey also demonstrated that the presence of a useful template was not sufficient by itself: template detection, sequence alignment, and model construction could each limit accuracy.\n\nEvolutionary information and deep learning\n\nLater advances exploited patterns of correlated sequence variation. Residues that interact in a folded protein can show related changes across evolution, providing clues about spatial proximity.\n\nDeep learning substantially improved the extraction and use of these signals. Methods progressed from predicting contacts to predicting distance distributions, orientations, and increasingly complete structural representations. CASP supplied a recurring test of whether these advances generalized to unseen targets.\n\nAlphaFold at CASP13 and CASP14\n\nAt CASP13 in 2018, the first AlphaFold system demonstrated a major advance, particularly on difficult targets with limited template guidance.\n\nAt CASP14 in 2020, AlphaFold2 achieved a further substantial improvement. For many assessed protein domains, its predictions approached the accuracy of experimentally determined structures. Its performance established that highly accurate prediction from sequence was possible across a much broader range of targets than earlier methods had demonstrated.\n\nThis result did not show that every protein, molecular state, or interaction could be predicted reliably. It shifted attention toward remaining difficulties, including complexes, conformational variation, disordered regions, and the reliable interpretation of confidence.\n\nCASP15 in 2022 and CASP16 in 2024 continued assessment in this changed methodological setting, with increasing attention to assemblies and broader molecular modeling tasks."
    },
    {
      "heading": "Scientific influence",
      "text": "CASP has provided a common basis for evaluating methodological claims. Its prospective design makes it more difficult to demonstrate apparent progress by selecting favorable examples or repeatedly tuning methods to a known test set.\n\nIts influence extends beyond rankings:\n\n- Benchmark design: common targets and independent assessment improve comparisons between methods.\n- Research priorities: recurring failures reveal problems that are not captured by headline accuracy measures.\n- Accuracy estimation: assessments distinguish producing plausible structures from recognizing when those structures are reliable.\n- Experimental interpretation: good predictions can assist structure determination and the interpretation of experimental data.\n- Community coordination: predictors, assessors, and experimental structural biologists work within a shared evaluation framework.\n\nCASP complements rather than replaces the Protein Data Bank, which archives experimentally determined macromolecular structures. It is an assessment process, not a comprehensive repository of predictions or a certification system for individual models."
    },
    {
      "heading": "Limitations and interpretation",
      "text": "Experimental structures are conditional references\n\nAn experimental structure represents a molecule under particular conditions and within the limits of the measurement and modeling procedure. Proteins may adopt multiple conformations, and some regions may be disordered or unresolved.\n\nA disagreement with the reference can therefore reflect prediction error, a different conformational state, or ambiguity in the experimental model. This does not remove the value of comparison, but it requires attention to the reference structure’s context and uncertainty.\n\nTarget selection is not universal sampling\n\nCASP targets depend on what experimental groups can contribute during a particular cycle. They are not a random sample of all proteins, complexes, or biological conditions. Performance can vary with sequence family, molecular size, structural novelty, and interaction type.\n\nResults from one edition or category should therefore not be interpreted as a universal success rate.\n\nBlind testing does not eliminate every source of bias\n\nWithholding target structures reduces direct contamination, but related sequences and structures may already occur in public databases or training data. The degree of novelty differs among targets, and rapidly changing databases complicate comparisons between methods developed at different times.\n\nMeaningful interpretation requires attention to available information, target difficulty, and the rules of the particular assessment.\n\nStatic structural accuracy is not complete biological understanding\n\nA highly accurate model does not by itself establish biochemical function, binding affinity, reaction mechanism, or the effect of every mutation. Nor does it necessarily describe conformational populations, folding kinetics, or changes induced by the cellular environment.\n\nGlobal structural scores can also conceal local errors that matter disproportionately for catalysis or molecular recognition. CASP measures defined aspects of prediction performance; those measurements do not establish every downstream use of a model."
    }
  ],
  "links": [
    "protein",
    "amino-acid",
    "molecular-biology",
    "protein-structure-prediction",
    "protein-folding",
    "x-ray-crystallography",
    "nuclear-magnetic-resonance",
    "cryo-electron-microscopy",
    "machine-learning",
    "antibody",
    "rna",
    "ligand",
    "angstrom",
    "alphafold",
    "deep-learning",
    "protein-data-bank",
    "measurement-uncertainty",
    "training-data",
    "mutation",
    "catalysis",
    "molecular-recognition"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}