AI agent

An AI agent is a computational system that receives information about an environment and selects actions to pursue goals or satisfy a performance criterion. Its environment may be physical, as in robotics, or digital, as in software that searches information, modifies files, or interacts with online services. The agent perspective connects artificial intelligence with the study of action: intelligence involves not only producing answers but also choosing what to do in response to changing circumstances. Autonomy and learning vary between systems; an agent need not learn continuously or operate without human supervision. (aima.cs.berkeley.edu)

Agent research encompasses rule-based and deliberative architectures, reinforcement learning, and systems built around large language models. In language-model-based agents, a model can direct tool use and revise its actions using feedback from the environment, rather than merely follow a predetermined sequence. This connects language generation with task execution, while introducing questions about reliability, evaluation, authorization, and security. Understanding AI agents therefore requires attention both to their decision-making mechanisms and to the wider systems through which their actions take effect. (aima.cs.berkeley.edu)

Definition, scope, and terminology

An artificial agent is a computational system understood through its interaction with an environment: it receives perceptual inputs and selects actions that affect that environment. In Stuart Russell and Peter Norvig’s formulation, perception occurs through sensors and action through actuators, but neither requires a physical body. A robot may receive camera images and drive motors; a software agent may read files or network messages and act by writing files, displaying information, or sending requests. The environment can therefore be physical, digital, or a combination of both. This definition describes an input–action relationship, not a particular machine-learning technique or level of intelligence. (aima.cs.berkeley.edu)

The agent function is the abstract mapping from perceptual history to action, conventionally written $f:P^{*}\rightarrow A$, where $P^{*}$ denotes finite sequences of percepts and $A$ the available actions. The agent program is the concrete implementation of that mapping on a computational architecture. An implementation need not store its entire perceptual history explicitly. Agenthood is also an analytical perspective rather than an absolute classification: even a calculator can be described as selecting a display action from its inputs, although that description may explain little about its operation. Consequently, the broad formal definition does not by itself distinguish an AI agent from ordinary software. (aima.cs.berkeley.edu)

In their June 1995 survey, Intelligent Agents: Theory and Practice, Michael Wooldridge and Nicholas Jennings identified four influential properties: autonomy, meaning control over actions and internal state without direct intervention; reactivity, meaning timely responses to environmental changes; pro-activeness, meaning goal-directed initiative rather than response alone; and social ability, meaning interaction with other agents or humans through a communication language. These properties characterize one influential research usage, not a universally accepted checklist. The authors explicitly noted disagreement over the term’s definition. Autonomy in this usage concerns operational control: a system can receive a human-assigned task yet independently choose actions during its execution. (cs.ox.ac.uk)

A large language model is not synonymous with an agent: it can be a component of a larger system that supplies tools, executes selected operations, and returns observations. Likewise, a conversational interface describes how users communicate with software, not who controls the task’s execution. An ordinary service can execute a requested operation without selecting a continuing course of action. In its December 19, 2024 account, Anthropic distinguished workflows, whose model calls and tools follow predefined code paths, from agents, in which a model dynamically directs the process and tool use. A fixed sequence that generates marketing copy and then translates it illustrates the former; a coding system that selects edits and subsequent operations from execution feedback illustrates the latter. These categories can overlap: a conversational application may contain both fixed workflows and agent-controlled loops. Anthropic used agentic systems as an umbrella for both, illustrating why that expression requires an explicit architectural definition. (anthropic.com)

The related labels emphasize different aspects. Intelligent agent associates agency with capabilities for effective action; rational agent specifies a decision standard—choosing actions expected to achieve goals given available information—rather than human-like reasoning. Autonomous agent emphasizes independence in operation, not guaranteed competence. Russell and Norvig’s rational-agent approach permits effective reflexes as well as deliberation, and recognizes computational limits. Terms such as “belief,” “goal,” and “intention” can describe computational representations or explanatory models; their use does not establish subjective experience or consciousness. Technical agency therefore concerns behavior, control, and interaction, without requiring human-like mental experience. (aima.cs.berkeley.edu)

Historical development

Agent research developed from several traditions rather than a single founding invention. Control theory supplied mathematical methods for regulating systems through feedback, while cybernetics connected control and communication in machines and organisms. Norbert Wiener’s Cybernetics (1948) helped establish this interdisciplinary programme; related work by Wiener, Arturo Rosenblueth, and Julian Bigelow treated purposive behaviour as regulation that reduces the difference between a current state and a target state. Symbolic AI contributed a different approach: explicit descriptions of situations, goals, and actions that a program could manipulate through inference and search. These traditions supplied complementary foundations for systems that respond to observations and select actions, without establishing one universally adopted agent architecture. (aima.cs.berkeley.edu)

A milestone in symbolic action selection was STRIPS, introduced by Richard Fikes and Nils Nilsson in 1971. Its planner searched for sequences of operators that transformed an initial world model into one satisfying a specified goal. Operators described their applicability conditions and changes to the model through preconditions, add lists, and delete lists. This made action consequences explicit and supported planning for robot tasks, rather than merely deriving answers to logical questions. A contrasting direction emerged in robotics through Rodney Brooks’s subsumption architecture, described in a September 1985 MIT technical report and a March 1986 journal article. Brooks organized control into asynchronous behavioural layers: higher layers could suppress lower-layer outputs, while lower layers continued operating. The architecture challenged the requirement that action always follow construction of a comprehensive symbolic world model and centralized deliberation. (ai.stanford.edu)

Another architectural lineage drew on Michael Bratman’s Intention, Plans, and Practical Reason (1987). Bratman’s account emphasized the role of intentions in stabilizing practical reasoning instead of reconsidering every choice continuously. Anand Rao and Michael Georgeff developed this approach computationally in Modeling Rational Agents within a BDI-Architecture (1991), treating beliefs, desires, and intentions as distinct elements in a branching-time logical framework. Their 1995 paper, BDI Agents: From Theory to Practice, connected these formal ideas to executable systems, including the Procedural Reasoning System and its successor, dMARS. In this belief–desire–intention tradition, commitment to selected plans addressed the tension between maintaining a course of action and responding to environmental change; the terminology designated computational structures, not evidence of mental experience. (jmvidal.cse.sc.edu)

From the mid-1980s, distributed AI increased attention to cooperation and coordination among agents, extending the research emphasis from individual reasoning to interacting systems. In parallel, Yoav Shoham’s agent-oriented programming proposals, published in 1990 and 1993, made beliefs, commitments, and abilities programming abstractions; AGENT0 was an early implementation. Interface-agent research explored assistance within ordinary software applications. Pattie Maes’s work, discussed in 1994, included NEWT, a USENET news-filtering system trained with examples of articles a user would or would not read and subsequently refined through feedback. Michael Wooldridge and Nicholas Jennings’s June 1995 survey, Intelligent Agents: Theory and Practice, synthesized this already diverse research community under three headings—agent theories, architectures, and languages—and surveyed distributed, interface, and information-agent applications. (cs.ox.ac.uk)

Large-language-model agents introduced a new way to implement parts of the established perception–action cycle. Shunyu Yao and colleagues first posted ReAct: Synergizing Reasoning and Acting in Language Models on 6 October 2022; it subsequently appeared at ICLR 2023. ReAct interleaved generated reasoning-oriented text with task-specific actions and observations returned by external resources or environments. Its demonstrations included question answering and fact verification using a Wikipedia interface, simulated household tasks in ALFWorld, and shopping tasks in WebShop. The historical change lay in using a language model to generate and revise action trajectories from textual context, rather than in introducing the agent concept itself: planning, reactive control, commitment, and software assistance had already been studied for decades. (arxiv.org)

Task environments and rational action

The PEAS framework specifies a task through its performance measure, environment, actuators, and sensors. In a two-location vacuum world, the environment comprises squares A and B and their dirt status; actuators provide movement, suction, and inaction; sensors report location and local dirt. Performance can award one point per clean square per time step over 1,000 steps. Penalizing movement changes what counts as good behavior: once both squares remain clean, stopping outperforms continual patrol. (aima.cs.berkeley.edu)

Environment classification identifies constraints on design. Full observability exposes all decision-relevant state; partial observability requires tracking unseen conditions. Deterministic transitions follow uniquely from state and action; stochastic transitions introduce uncertainty. Episodic decisions are independent, whereas sequential actions affect later choices. Static environments remain unchanged during deliberation; dynamic ones require timely responses. Discrete or continuous states, actions, and time demand different representations. Multi-agent environments additionally require accounting for other decision-makers. A vacuum sensing only local dirt has partial observability even when movement and cleaning are deterministic: uncertainty about state differs from uncertainty about effects. (aima.cs.berkeley.edu)

Rational action maximizes expected performance given the agent’s observations, prior knowledge, and available actions—not the outcome that hindsight reveals would have been best. This connects agency with decision theory and expected value: possible consequences matter through both their desirability and their probability. An unfavorable result therefore does not, by itself, establish that the preceding decision was irrational. Information gathering can itself be rational when improved observations make subsequent choices more effective. Rationality also does not require a lengthy logical derivation: the rational-agent approach permits a rapid response when slower deliberation would produce a worse result. (aima.cs.berkeley.edu)

Bounded rationality addresses the difficulty of implementing this standard with finite processing time and memory. Computing an exact best decision may consume resources needed to act, particularly when the environment changes during calculation. Practical decision procedures consequently use approximations rather than exhaustively evaluating every possibility. A related mechanism is rational metareasoning: choosing computations according to their expected contribution to better external decisions. Beginning in 1988, Stuart Russell and Eric Wefald developed this approach around the expected value of computation. It treats deliberation as an activity with benefits and costs, rather than as a free preliminary to action; deciding how much to think can itself become a computationally difficult problem. (aima.cs.berkeley.edu)

In 1995, Russell and Devika Subramanian formalized bounded optimality, shifting the comparison from ideal individual actions to executable agent programs. A bounded-optimal program performs at least as well as alternative programs available on the specified computational architecture within the specified task environment. Their analysis constructed such agents for a restricted class of machines and real-time environments, illustrated with automated mail sorting. This is a stronger and more specific criterion than merely using an inexpensive heuristic: an approximation is not automatically bounded-optimal, and optimality remains relative to the machine’s constraints and the environment being considered. (arxiv.org)

Classical agent architectures

A simple reflex agent selects an action from the current observation using condition–action rules, without retaining the observation history. In a two-location vacuum environment, its rules can be: if the current location is dirty, clean it; otherwise, move to the other location. Such a controller cannot distinguish situations that produce identical observations but require different actions. A model-based reflex agent instead maintains an internal state, updating it from the previous state, the last action, and the new observation. Its model describes both changes occurring independently in the environment and the effects of its own actions. An automated driver, for example, can compare successive camera frames to detect brake lights switching on, rather than attempting to infer braking from a single image. Action selection still uses condition–action rules, but these rules operate on the maintained state rather than only on immediate sensory input. (aima.cs.berkeley.edu)

Goal-based agents explicitly represent a target condition and consider which actions could produce it. An automated taxi approaching a junction must combine its account of the road network with the passenger’s destination; the observed junction alone does not determine the turn. Search or planning can connect the current state to the goal, allowing the destination to change without replacing destination-specific reflex rules. Utility-based agents additionally rank outcomes: several routes may reach the destination, yet differ in duration, cost, safety, or comfort. A utility function represents preferences numerically, while decision theory combines those preferences with uncertainty about outcomes. The distinction is between identifying satisfactory goal states and comparing how desirable alternative outcomes are—not between possessing and lacking human-like motivation. (aima.cs.berkeley.edu)

Deliberative architectures use explicit symbolic representations of the world and reasoning procedures to select actions. Their mechanisms draw on symbolic AI and knowledge representation and reasoning: observations must be translated into useful descriptions, and inference or planning must finish while those descriptions remain relevant. Reactive architectures emphasize direct responses to environmental conditions, avoiding dependence on extensive symbolic deliberation before each action. Rodney Brooks’s subsumption architecture, published in 1986, organized robot control into interacting layers of task-accomplishing behaviors, such as obstacle avoidance. This architectural contrast is broader than the distinction between simple and model-based reflex programs: reactive control can involve multiple interacting behaviors rather than a single rule table. (cs.ox.ac.uk)

Hybrid architectures combine rapid reactive control with deliberative processes and require mechanisms for resolving conflicting action proposals. In Innes Ferguson’s 1992 TOURINGMACHINES architecture, three concurrently operating control layers shared perception and action subsystems. A reactive layer handled events demanding immediate responses; a planning layer constructed and executed plans using a topological map and partially elaborated plans; and a modeling layer represented other entities to identify interference with the agent’s goals. Control rules mediated between the layers. Thus, adding a planner to a reactive controller is insufficient by itself: the architecture must also specify which component controls execution when their proposals differ. (cs.ox.ac.uk)

Belief–desire–intention (BDI) architectures separate information about the world, represented as beliefs, from desired activities or outcomes and the intentions currently selected for pursuit. Commitment gives intentions persistence and constrains the adoption of conflicting alternatives. Michael Bratman, David Israel, and Martha Pollack’s 1988 Intelligent Resource-bounded Machine Architecture (IRMA) used a plan library, means–ends analysis, opportunity detection, and a filter for options inconsistent with existing intentions. In Michael Georgeff and Amy Lansky’s 1987 Procedural Reasoning System (PRS), an interpreter updated beliefs, activated partially elaborated plans called knowledge areas, and executed actions; active knowledge areas represented intentions. These plans could be triggered by goals or incoming data, combining deliberative and reactive operation. Beliefs, desires, and intentions here are computational representations and explanatory abstractions, not evidence of consciousness or subjective mental experience. (cs.ox.ac.uk)

Planning and sequential decision-making

In classical automated planning, an agent represents an initial state, a goal condition, and actions with explicit preconditions and effects. State-space search treats states as nodes and applicable actions as edges: forward search generates successors from the initial state, while backward search works from goal requirements toward conditions that could establish them. Preconditions determine whether an action is applicable; effects specify which facts become true or false. In the air-cargo example in Artificial Intelligence: A Modern Approach, loading requires cargo and an aircraft to be at the same airport. Loading removes the cargo’s airport-location fact and adds its presence inside the aircraft. A load–fly–unload sequence achieves delivery because each action establishes conditions needed by the next, and the final state satisfies the goal. Individually executable actions do not necessarily constitute a successful plan; their ordering and combined effects matter. (aima.cs.berkeley.edu)

For uncertain outcomes, a Markov decision process (MDP) specifies states $S$, actions $A$, transition probabilities $T(s,a,s')$, and rewards $R(s,a,s')$. The Markov property requires the current state and chosen action to contain the information needed to predict the next-state distribution. A policy selects an action for each state, rather than prescribing one fixed action sequence. In Berkeley’s racecar teaching example, the states are cool, warm, and overheated: driving fast from cool produces either cool or warm with probability $0.5$ each, whereas driving fast from warm guarantees overheating and incurs reward $-10$. A policy can therefore respond to the resulting temperature. A common planning objective is to maximize expected discounted reward,

Planning can calculate such a policy from a supplied model without learning it through interaction. (inst.eecs.berkeley.edu)

When observations do not uniquely identify the state, a partially observable Markov decision process (POMDP) adds an observation model. The agent maintains a belief state, a probability distribution over possible states, rather than assuming that its latest observation is complete. After action $a$ and observation $o$, the update combines predicted transitions with observation likelihoods:

where $\eta$ normalizes the distribution. Decisions can then depend on beliefs. Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra’s 1998 treatment explains this framework and its information-gathering behavior. A sensing action may produce little immediate physical progress yet improve subsequent choices by reducing uncertainty; its usefulness depends on the expected benefit of that information relative to its cost. (cassandra.org)

Hierarchical planning represents a task through successively more detailed refinements. A high-level instruction such as transporting an object can be decomposed into obtaining it, moving to the destination, and placing it; these subtasks must eventually resolve into executable actions. Refinements carry applicability conditions and ordering constraints, so decomposition is not merely a verbal checklist. Hierarchical search explores alternative refinements, while abstract descriptions of their possible outcomes or costs can help reject unsuitable branches before every primitive action is enumerated. The resulting plan must still satisfy the original goal and the domain’s action constraints. (aima.cs.berkeley.edu)

Execution monitoring compares observed results with the conditions on which the remaining plan depends. Unexpected outcomes can require repairing a suffix of the plan or searching again from an updated state estimate. In a 2019 study, Caelan Reed Garrett, Chris Paxton, Tomás Lozano-Pérez, Leslie Pack Kaelbling, and Dieter Fox combined observation actions with continuous robot-control actions in belief-space planning. After execution and observation, their system replanned using its new estimate, initially requiring reuse of the unexecuted plan’s structure to improve efficiency and avoid undoing progress. Such replanning changes the course of action in response to evidence; it need not change the underlying transition model or learn a new action-selection rule. (arxiv.org)

Learning and adaptation

A learning agent can be decomposed into four conceptual components. The performance element selects actions from percepts and internal information; the learning element modifies the knowledge or procedures used to select those actions. A critic evaluates behavior against a specified performance standard and supplies feedback to the learning element. The problem generator proposes exploratory actions that may produce useful experience rather than the highest immediate payoff. These are functional roles, not necessarily separate software modules. For example, an automated taxi can observe the deceleration produced by braking on a wet road, update its estimate of braking effectiveness, and use that estimate in subsequent driving decisions. The evaluation standard remains distinct from the behavior being modified: changing the criterion for success is not equivalent to improving performance. (aima.cs.berkeley.edu)

In reinforcement learning, an agent learns from rewards associated with interaction rather than receiving the correct action for every situation. A discounted return can be written as $G_t=\sum_{k=0}^{\infty}\gamma^k R_{t+k+1}$, where $R_{t+k+1}$ is a subsequent reward and $0\leq\gamma<1$ discounts more distant outcomes. A value function estimates expected return from a state, or from a state–action pair, under a particular policy. Policy improvement changes action selection to favor higher expected returns, rather than merely larger immediate rewards. Learning also creates an exploration–exploitation trade-off: exploitation selects actions currently estimated to be best, whereas exploration gathers evidence about alternatives that may ultimately be better. (incompleteideas.net)

Q-learning provides a representative value-based mechanism. Christopher Watkins and Peter Dayan’s 1992 paper established convergence results for its tabular form under specified sampling and learning-rate conditions. After observing a transition from state $s$, through action $a$, to state $s'$, with reward $r$, it updates an action-value estimate using

Here $\alpha$ is the learning rate; the bracketed quantity is a temporal-difference error. The maximization supplies a target based on the best estimated next action, even when exploratory behavior selects another action. Policy-based methods instead adjust a parameterized action-selection rule directly to increase expected return. Actor–critic methods combine a policy, the actor, with a learned value estimator, the critic, whose estimates guide policy updates. (gatsby.ucl.ac.uk)

Model-free methods learn values or policies without using an explicit environmental transition model for planning. Model-based approaches use a supplied or learned model predicting successor states and rewards. Such a model allows an agent to compare hypothetical actions or generate simulated transitions for further learning. In the Dyna architecture, real transitions serve both to update action values and to improve a model; additional updates then use experience generated by that model. Planning and learning consequently share an update mechanism while drawing on different sources of experience. Simulated experience can reduce the amount of additional interaction required, but inaccuracies in the learned model can also produce misleading updates. (incompleteideas.net)

Learning from interaction is distinct from acquiring component skills through supervised learning: a perception component, for example, can be trained on labeled observations before being incorporated into an agent. Adaptation during operation also need not change model parameters. An agent can condition later actions on stored observations, corrections, or records of previous attempts while its underlying model remains fixed. Noah Shinn and colleagues’ 2023 Reflexion framework illustrates this distinction: linguistic feedback is retained in an episodic memory and supplied as context for subsequent attempts, rather than used to update the language model’s weights. Thus, parameter training, persistent memory updates, and changes to the current task context are different mechanisms through which experience can affect behavior. (aima.cs.berkeley.edu)

Language-model-based agents

A large language model supplies generated responses and action selections; the surrounding execution system supplies instructions, tool access, observations, and control flow. At each iteration, the system presents the task and relevant history, obtains a proposed action, executes it through an available tool, and returns the result as a new observation. An action proposal is therefore distinct from an environmental change. Termination can follow a completion signal, a request for human input, or an externally imposed iteration limit; a model’s declaration of success is not itself verification that the task succeeded. (anthropic.com)

ReAct, introduced by Shunyu Yao and colleagues in October 2022 and presented at ICLR 2023, interleaves reasoning-oriented text with environment-directed actions. Its reasoning steps can identify missing information, maintain a provisional plan, and revise that plan after an observation. In the paper’s question-answering setup, actions use a Wikipedia interface to search for an entity, locate information within a page, or finish with an answer. Rather than generating an entire solution before consulting evidence, the model can alternate between gathering information and deciding what to do next. The authors also demonstrated the approach in ALFWorld and WebShop, using only one or two in-context examples to specify task-solving behavior. These examples supplied interaction patterns without requiring parameter updates during the task. (arxiv.org)

The resulting textual trace is not an independently verified account of the model’s internal computation. It records generated reasoning statements and actions, but those statements may omit influences on the answer. In a 2023 study, Miles Turpin and colleagues showed that biasing features in prompts—including rearranged multiple-choice examples—could change answers without being acknowledged in the accompanying explanations. Accordingly, readable traces support inspection of a task trajectory but do not, by themselves, establish explanation faithfulness. (arxiv.org)

An agent’s working information occupies several distinct stores. Its context window contains the material available for the current model call: instructions, selected conversation history, tool descriptions, retrieved passages, and recent observations. Persistent records reside outside that window, for example in files containing task notes, decisions, or unfinished work. They affect later behavior only when relevant content is brought back into context. Anthropic’s September 2025 account describes compaction—summarizing an accumulated interaction before continuing—and structured note-taking as mechanisms for carrying state across extended tasks. These mechanisms preserve selected information rather than providing unlimited recall, and summarization can discard details needed later. (anthropic.com)

Retrieval-augmented generation (RAG) addresses access to external information, not the entire action-control problem. Patrick Lewis and colleagues’ 2020 formulation combined a pretrained sequence-to-sequence generator with a dense retriever accessing a Wikipedia index, distinguishing knowledge held in model parameters from an external, non-parametric store. A fixed retrieve-then-answer pipeline can therefore use RAG without dynamically choosing actions. Conversely, an agent can make retrieval one option within a longer loop, deciding whether to search again, consult another tool, or produce an answer. (arxiv.org)

Control patterns differ in where decisions are made. Prompt chaining passes outputs through predefined stages, such as outline generation, checking, and drafting. Routing selects a specialized downstream process for an input category. An orchestrator–worker arrangement lets a coordinating model determine subtasks and combine worker results, rather than fixing every subtask beforehand. A dynamic agent loop repeatedly chooses its next action from observations. Anthropic’s December 2024 taxonomy distinguishes predefined workflows from model-directed control; these are composable architectural patterns, not levels of general intelligence. A single application may combine a fixed outer workflow with dynamically controlled work inside one stage. (anthropic.com)

Tools, interfaces, and execution infrastructure

Agents act through application programming interfaces (APIs), command-line programs, browsers, and graphical interfaces. These provide different action surfaces: an API exposes named operations and parameters; a command-line tool runs a program; a graphical interface accepts clicks, keystrokes, and other input events. A language model’s proposed operation is not itself an executed action. The surrounding application must dispatch the request, receive its result, and return observations to the model. Anthropic’s 2024 computer-use research illustrates the graphical route: Claude interpreted screenshots and selected operations through software tools. Screenshot-based observation also introduced a specific limitation—brief notifications could disappear between captures. Execution and observation therefore remain separate steps, even when an interface presents them as a continuous interaction. (anthropic.com)

Tool definitions form an explicit contract between probabilistic model output and executable software. They describe an operation’s name, purpose, expected arguments, and sometimes its output structure. Documentation can specify details that argument types alone cannot express: whether a file path must be absolute, which units a number uses, or how missing data is represented. Anthropic’s December 2024 account, written by Erik Schluntz and Barry Zhang, reported improving a coding agent by requiring absolute rather than relative file paths, removing ambiguity after changes of working directory. At execution time, validation checks the submitted arguments before a handler performs the operation. Returned data and explicit errors then supply feedback for the next action; successful parsing does not establish that the requested task succeeded. (anthropic.com)

The Model Context Protocol (MCP) standardizes connections between AI applications and external tools and data. Its architecture separates the host application, clients inside that host, and servers exposing capabilities. Servers can provide tools, readable resources, and reusable prompts. In the specification dated June 18, 2025, clients discover tools with tools/list and invoke them with tools/call; each tool has an inputSchema, expressed using JSON Schema, and may declare an outputSchema. This standardizes discovery and exchange rather than deciding which operation advances a task. The host and model still select tools and interpret their results. (github.com)

Error reporting preserves distinctions needed for recovery. The June 2025 MCP specification separates protocol errors, such as an unknown tool or invalid arguments, from execution failures returned with isError: true, such as an external API failure or business-rule rejection. Results may contain text, images, resource links, or structured data; declared output schemas allow structured responses to be validated. Timeouts address a different condition: a request that does not finish. MCP’s lifecycle specification calls for request timeouts, cancellation when a timeout expires, and a maximum timeout even if progress notifications continue. A stalled operation is thus distinguishable from a completed operation that reports failure. (modelcontextprotocol.io)

The execution system also controls the duration and continuity of iterative work. Anthropic’s 2024 agent pattern includes maximum iteration counts, checkpoints for human feedback, and pauses when blockers require clarification or judgment. Persistent checkpoints serve another purpose: carrying task state across separate sessions. In November 2025, Justin Young described an experimental coding harness that used a structured feature list, progress notes, and version-control commits. Each session read those artifacts, selected an unfinished feature, and left an updated record; features were marked as passing only after testing. Such infrastructure makes completion criteria operational rather than dependent on a fluent final message. When work stops at a checkpoint or blocker, the recorded state identifies what has been completed and what remains unresolved for subsequent execution or human review. (anthropic.com)

Multi-agent systems and human collaboration

A multi-agent system contains interacting agents whose decisions affect one another through messages, shared resources, or changes to an environment. Agents may have different observations, capabilities, and objectives: cooperation does not require identical knowledge, and participation does not imply a common goal. Michael Wooldridge’s account distinguishes agents’ separate spheres of influence from the organizational relationships connecting them. In language-model systems, several independent calls followed by a fixed aggregation step constitute parallel computation, but do not by themselves establish multi-agent interaction. The relevant distinction is whether components independently select actions and whether their behavior responds to other agents or their effects. (cs.ox.ac.uk)

Cooperation involves sharing tasks or results to achieve compatible objectives; competition arises when agents’ preferences conflict, for example over scarce resources. Coordination concerns aligning activities so that agents do not obstruct one another, even when they share an overall goal. Communication supplies information for coordination, but exchanging messages is not equivalent to reaching agreement: a request, a proposal, and an accepted commitment have different operational meanings. Game theory represents strategic dependence through actions, preferences, and payoffs, including situations in which an agent’s outcome depends on others’ choices. Its rationality assumptions belong to particular analytical models, rather than defining every multi-agent system; implemented participants can operate with incomplete information and bounded rationality. (cs.ox.ac.uk)

Organization can be centrally coordinated, with one agent decomposing a problem and assigning work, or decentralized, with agents arranging cooperation through local exchanges and negotiation. Cooperative distributed problem-solving commonly separates problem decomposition, subproblem solution, and synthesis of results. The contract net protocol provides a concrete task-allocation mechanism: an agent announces a task and its constraints; potential contractors decide whether to bid; the announcing agent evaluates bids and awards the task; the contractor executes it and reports completion or failure. Contractors can subcontract further work, so manager and contractor are roles within an interaction rather than necessarily permanent positions. This combines local choice with explicit allocation, instead of assuming that every task can be assigned from complete centralized knowledge. (cs.ox.ac.uk)

A centrally coordinated language-model example is Anthropic’s research system, described on June 13, 2025. A lead agent develops a research strategy, delegates distinct investigations to tool-using subagents with separate contexts, and synthesizes their findings. Delegation instructions specify objectives, output formats, relevant tools, and task boundaries; vague assignments produced duplicated searches and gaps in coverage. Communication and coordination also consume resources: Anthropic reported approximately 15 times the token use of ordinary chat interactions for its multi-agent systems, a measurement specific to its workloads rather than a universal ratio. Its synchronous execution introduced another cost: the lead agent waited for a batch of subagents, allowing one slow investigation to delay progress. Parallel work therefore increases available capacity without eliminating integration costs or dependencies. (anthropic.com)

Human collaboration adds delegation, clarification, supervision, and transfer of control to these arrangements. In Anthropic’s December 19, 2024 architectural account, an agent begins with a command or discussion, works independently once the task is sufficiently clear, and can return to the person for missing information or judgment. Checkpoints allow review during execution, while blockers can trigger a handoff rather than continued autonomous action. Operational independence concerns how the system chooses and executes intermediate steps; authority concerns which actions the person or organization has permitted. For example, independently modifying code and running tests does not settle whether the changes should be accepted: Anthropic’s coding example retains human review for alignment with broader system requirements, despite automated functional testing. (anthropic.com)

Applications and representative tasks

In robotics, embodied agents connect observations to actions that change the physical environment. A mobile robot uses cameras, range sensors, and measurements of its own movement to estimate its position, represent obstacles, and navigate toward a destination. A manipulator instead controls an arm and end effector to reach, grasp, and move objects; motion planning must account for the robot’s configuration as well as surrounding obstacles. Perception supplies estimates rather than complete knowledge, while control and feedback connect planned motion to subsequent measurements. Physical embodiment therefore involves sensors, actuators, and mechanical constraints, not merely a software representation of a robot. Software-only agents act on files, records, or interfaces without directly controlling a physical body. (aima.cs.berkeley.edu)

RT-2, presented by Brianna Zitkovich and colleagues in 2023, illustrates language-guided robotic manipulation. The system jointly fine-tuned vision–language models on robotic trajectories and web-derived vision–language tasks, representing robot actions as tokens so that observations and instructions could be mapped to executable movements. Its approximately 6,000 evaluation trials included commands involving unfamiliar objects and semantic relationships, such as placing an object onto a specified number or selecting the smallest object. Here, the application is not simply answering a question about a scene: the model participates in a robotic control system that must select and physically manipulate the relevant object. These experiments concerned specified manipulation tasks rather than unrestricted physical autonomy. (proceedings.mlr.press)

In software engineering, agents can work through an issue description by inspecting a repository, locating relevant definitions, editing code, and executing tests. SWE-bench, introduced by Carlos E. Jimenez and colleagues in October 2023, supplied 2,294 problems derived from GitHub issues and corresponding pull requests across 12 Python repositories. Each problem provides a codebase and an issue to address; solutions may require coordinated changes across several functions, classes, or files rather than an isolated generated function. The 2024 SWE-agent paper gives a concrete repository-navigation example involving SymPy issue sympy__sympy-21614: the agent searches for function.py and opens it while investigating the Derivative class. Such operations make repository inspection part of the task itself. A proposed patch remains distinct from an executed repair: the SWE-bench harness applies generated patches and runs repository tests to check their effects. (arxiv.org)

Information agents support information retrieval by locating, collating, and manipulating material from sources such as databases or networked documents. Michael Wooldridge and Nicholas Jennings’s 1995 survey described an agent searching for a technical report about agent-oriented programming and returning the report together with its author’s contact information. Interface agents emphasize assistance within a user’s application. The survey discussed Pattie Maes’s NewT news-filtering system, which received examples of articles a user would or would not read, suggested further articles, and incorporated feedback. These applications distinguish selecting source material from interpreting it: retrieving a document establishes access to evidence, whereas a generated account of its meaning is an additional processing step, not the document itself. (cs.ox.ac.uk)

Web-based agents extend assistance into sequences of browser operations. WebArena, introduced by Shuyan Zhou and colleagues in July 2023, provided 812 tasks across self-hosted environments for online shopping, discussion forums, collaborative software development, and content management. Agents could click elements, type text, switch tabs, and navigate between pages while observing page content. Shopping tasks could involve placing an order; content-management tasks required changing stored content or settings rather than merely describing the intended change. One cross-site example required finding Pittsburgh art museums on Wikipedia, locating them on a map, planning an itinerary, and updating a repository’s README with the route. These are bounded, inspectable digital tasks: completion within the supplied websites does not by itself establish competence across unfamiliar workplace systems or responsibility for an entire organizational role. (arxiv.org)

Evaluation, reliability, and open research problems

Component-level tests measure abilities such as answering questions, selecting tools, or generating valid arguments; end-to-end evaluations measure whether the complete agent changes an environment in accordance with a task. A fluent completion message is not evidence that the intended change occurred: a booking task requires an actual reservation, not merely a statement that one exists. Evaluation therefore separates the final outcome from the recorded trajectory of observations and actions. Task success can be assessed alongside action validity, latency, token consumption, and cost, while trajectory inspection helps distinguish faulty decisions from execution failures. These measurements concern the model and its surrounding execution system together. (anthropic.com)

WebArena, introduced by Shuyan Zhou and colleagues in July 2023, operationalized this distinction through 812 tasks on self-hosted websites representing e-commerce, discussion forums, collaborative software development, and content management. Its evaluators check functional outcomes through mechanisms including answer comparison, browser-state inspection, and database queries, rather than requiring an agent to reproduce one reference sequence of clicks. This allows different action sequences to satisfy the same goal. The original study reported 14.41% end-to-end success for its best GPT-4-based agent, compared with 78.24% for human participants; these figures describe that study’s configurations, not subsequent systems or a general limit on web-agent capability. (arxiv.org)

SWE-bench, introduced by Carlos E. Jimenez and colleagues in October 2023, initially comprised 2,294 issues drawn from 12 Python repositories. Each task pairs an issue description with a repository state and asks the system to produce a patch. Evaluation executes tests: relevant previously failing tests must pass, while designated previously passing tests must remain passing. This measures observable software behavior rather than the persuasiveness of a proposed explanation or textual similarity to a developer’s patch. Executable evaluation nevertheless depends on the adequacy of its tests; passing a finite suite is not equivalent to formal verification of every intended behavior. (arxiv.org)

For reproducibility, benchmark reports need to identify the task release and subset, model version, prompts, available tools, observation format, execution environment, stopping rules, and time, token, and attempt budgets. Tool design is itself an experimental variable: Anthropic’s December 2024 account of building agents reported spending more effort optimizing its SWE-bench tools than its overall prompt. Comparisons that simultaneously change the model, tools, and control loop cannot isolate the model’s contribution. Repeated trials also distinguish occasional success from consistency: $\mathrm{pass@}k$ measures success in at least one of $k$ attempts, whereas $\mathrm{pass}^{k}$ measures success in all $k$. Trials require clean initial environments so that leftover files or shared state do not distort results. (anthropic.com)

Reliability failures can arise when an unsupported assumption becomes the premise for later actions, when observations omit relevant information, or when the system fails to revise a plan after contradictory feedback. In the ReAct study, first released in October 2022, Shunyu Yao and colleagues found that interaction with external information could reduce hallucination and error propagation, but also documented failures involving uninformative search results and repetitive reasoning–action loops. Recovery consequently requires more than retrying an operation: an agent must recognize that its interpretation or plan needs revision. Long-horizon tasks test whether such corrections remain effective across extended sequences. Robustness to changed interfaces and distribution shift, and generalization to unfamiliar environments, remain research questions rather than consequences established by success on a fixed benchmark. (arxiv.org)

Security, oversight, and accountability

Indirect prompt injection occurs when an agent treats instructions embedded in external material as authoritative commands rather than untrusted task data. An attacker-controlled website, email, or file can redirect a legitimate task without the attacker controlling the user’s request. NIST describes this redirection as agent hijacking: trusted instructions and external content enter a shared model context, allowing the latter to influence action selection. The relevant distinction is authorization, not how convincingly a document claims to contain administrative instructions. (nist.gov)

Tool access converts this instruction-confusion problem into a cybersecurity risk involving external effects. NIST’s evaluation scenarios included executing malicious scripts, disclosing authentication codes, and exfiltrating cloud-drive files before deleting the originals. Filtering an agent’s final answer cannot alone prevent harm already caused through a tool call: the security boundary also includes executable operations, accessible information, and destinations to which information can be transmitted. (nist.gov)

Attack success also depends on opportunities to retry. In experiments reported in its January 17, 2025 technical blog, NIST tested five malicious tasks against agents using the October 2024 version of Claude 3.5 Sonnet. Allowing 25 attempts increased average attack success from 57% to 80%. These figures describe that experimental setup, not a general vulnerability rate for agents. (nist.gov)

Permission boundaries restrict what a compromised or mistaken agent can accomplish. Least-privilege access separates capabilities such as reading records, changing files, executing programs, and communicating externally, rather than automatically granting every capability available to the user. Sandboxing adds execution-level restrictions on resources the agent can reach. Anthropic’s response to NIST’s agent-security request for information documents sandboxing, least privilege, input and output validation, anomaly monitoring, and audit logs as complementary deployment controls. These mechanisms operate around the model: an instruction to avoid unauthorized access is not equivalent to an access-control system that rejects it. Their effectiveness therefore depends on the actual tools, credentials, and execution environment, as well as model behavior. (www-cdn.anthropic.com)

Human oversight introduces checkpoints between proposing an action and executing it. Anthropic’s April 2026 description of its products gives a concrete example: a user can permit calendar reading while requiring approval before invitations are sent. Its December 2024 agent-engineering guide also describes agents pausing for human input at checkpoints or when encountering obstacles, with stopping conditions limiting continued execution. Such arrangements distinguish delegated task scope from unrestricted discretion: permission to investigate an issue need not include permission to publish information or modify a system. Confirmation gates and restricted environments reduce exposure but do not establish that prompt injection has been eliminated; Anthropic explicitly describes its combined safeguards as providing no guarantee. (anthropic.com)

Traceability connects actions to the authority under which they occurred. Audit records support incident investigation, while explicit task boundaries and identifiable organizational owners make it possible to distinguish an authorized operation from a deviation. NIST’s AI Risk Management Framework specifies documented roles, responsibilities, and communication channels under GOVERN 2.1, and assigns responsibility for development and deployment risk decisions to executive leadership under GOVERN 2.3. In this operational account, an agent’s ability to select actions independently does not itself establish legal or moral personhood: technical autonomy describes system behavior, whereas accountability arrangements identify the people and organizations responsible for authorizing, deploying, and supervising it. (www-cdn.anthropic.com)

References

1. Intelligent Agents, Chapter 2 of Artificial Intelligence: A Modern Approachaima.cs.berkeley.edu
2. Intelligent Agents: Theory and Practicecs.ox.ac.uk
3. What is an Agent?cs.ox.ac.uk
4. A Weak Notion of Agencycs.ox.ac.uk
5. A Stronger Notion of Agencycs.ox.ac.uk
6. Building effective agentsanthropic.com
7. Introduction to AI: A Modern Approachaima.cs.berkeley.edu
8. The Foundations of Artificial Intelligenceaima.cs.berkeley.edu
9. STRIPS: A New Approach to the Application of Theorem Proving to Problem Solvingai.stanford.edu
10. A Robust Layered Control System for a Mobile Robotdspace.mit.edu
11. Rodney Brooks Publicationspeople.csail.mit.edu
12. Modeling Rational Agents within a BDI-Architecturejmvidal.cse.sc.edu
13. BDI Agents: From Theory to Practicecdn.aaai.org
14. Cooperative Problem Solving and Distributed AIcs.ox.ac.uk
15. Shoham - agent-oriented programmingcs.ox.ac.uk
16. Interface Agentscs.ox.ac.uk
17. ReAct: Synergizing Reasoning and Acting in Language Modelsarxiv.org
18. ReAct: Synergizing Reasoning and Acting in Language Modelsarxiv.org
19. Stuart Russell — Foundations: Rationality and Intelligencepeople.eecs.berkeley.edu
20. Provably Bounded-Optimal Agentsarxiv.org
21. Intelligent Agents: Chapter 2 slidesaima.cs.berkeley.edu
22. Classical Approaches: Deliberative Architecturescs.ox.ac.uk
23. Brooks - behaviour languagescs.ox.ac.uk
24. Ferguson - TOURINGMACHINEScs.ox.ac.uk
25. Bratman, Israel and Pollack - IRMAcs.ox.ac.uk
26. Georgeff and Lansky - PRScs.ox.ac.uk
27. Agents as Intentional Systemscs.ox.ac.uk
28. 11 Planningaima.cs.berkeley.edu
29. 1 Markov Decision Processes | Introduction to Artificial Intelligenceinst.eecs.berkeley.edu
30. Planning and Acting in Partially Observable Stochastic Domainscassandra.org
31. Global Algorithms for Artificial Intelligence: A Modern Approachaima.cs.berkeley.edu
32. Angelic Hierarchical Planning: Optimal and Online Algorithms (Revised)aima.cs.berkeley.edu
33. Online Replanning in Belief Space for Partially Observable Task and Motion Problemsarxiv.org
34. Q-learninggatsby.ucl.ac.uk
35. Policy Gradient Methods for Reinforcement Learning with Function Approximationpapers.neurips.cc
36. Reflexion: Language Agents with Verbal Reinforcement Learningarxiv.org
37. Building Effective AI Agentsanthropic.com
38. Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingarxiv.org
39. Effective context engineering for AI agentsanthropic.com
40. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org
41. Developing a computer use modelanthropic.com
42. MCP Server — MCP Java SDKjava.sdk.modelcontextprotocol.io
43. Core architecturegithub.com
44. First steps — MCP Python SDKgithub.com
45. Tools — Model Context Protocolmodelcontextprotocol.io
46. Lifecycle — Model Context Protocolmodelcontextprotocol.io
47. Effective harnesses for long-running agentsanthropic.com
48. An Introduction to Multiagent Systems/Mike Wooldridge/Lecture Slidescs.ox.ac.uk
49. How we built our multi-agent research systemanthropic.com
50. Robotics: Chapter 25aima.cs.berkeley.edu
51. RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlproceedings.mlr.press
52. SWE-bench: Can Language Models Resolve Real-World GitHub Issues?arxiv.org
53. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringproceedings.neurips.cc
54. Evaluation - SWE-benchswebench.com
55. Intelligent Agents: Theory and Practicecs.ox.ac.uk
56. WebArena: A Realistic Web Environment for Building Autonomous Agentsarxiv.org
57. Demystifying evals for AI agentsanthropic.com
58. WebArena: A Realistic Web Environment for Building Autonomous Agentsarxiv.org
59. WebArena: A Realistic Web Environment for Building Autonomous Agentsarxiv.org
60. Are "Solved Issues" in SWE-bench Really Solved Correctly? An Empirical Studyarxiv.org
61. ReAct: Synergizing Reasoning and Acting in Language Modelsarxiv.org
62. Technical Blog: Strengthening AI Agent Hijacking Evaluationsnist.gov
63. NIST RFI on Agentic Securitywww-cdn.anthropic.com
64. Trustworthy agents in practiceanthropic.com
65. AI RMF Core - AIRCairc.nist.gov