What Is an AI Agent? How It Differs from a Plain LLM Call

An AI Agent is an AI system that judges its own next action and carries out multiple steps in sequence without a human instructing every step. Anthropic’s engineering blog defines it as “a system where the LLM directs its own process and tool usage, maintaining control over how it accomplishes a task.” This definition is one of the most widely cited in the industry.

How is it different from “just calling an LLM”

A typical LLM call — asking ChatGPT a single question and getting a single answer — is a one-shot interaction: one input, one output. An AI Agent, by contrast, has a repeating structure:

1. Assess the current situation.
2. Decide on its own which tool to use next, or whether the task is done.
3. Run the tool and observe the result.
4. Repeat steps 1-3 until the goal is achieved.

Anthropic contrasts this with the concept of a “Workflow” — “a system where LLMs and tools are orchestrated through predefined code paths.” In other words, a human decides in advance, in code, that “do A, then B, then C if a condition is met.” The key difference with an agent is that the LLM itself decides that sequence in real time.

	Workflow	AI Agent
Execution path	Predefined in code by a human	The LLM decides for itself based on the situation
Predictability	High (only predefined steps run)	Low (the number and order of steps vary)
Good fit	Tasks where the needed steps are known in advance	Open-ended problems where the number of steps is hard to predict
Example	A data pipeline with a fixed sequence	Claude Code autonomously fixing a bug

Not every problem needs an agent

Anthropic recommends “starting as simply as possible.” In many cases, a single LLM call augmented with retrieval and examples is enough, and an agent should only be considered when it’s worth the added latency and cost. Anthropic’s stated condition for when an agent is a good fit is clear: “an open-ended problem where the number of steps needed is hard to predict and a fixed path can’t be defined in advance.”

The basic building block of an agent: the Augmented LLM

Anthropic identifies the “Augmented LLM” as the smallest unit underlying every agent system — an LLM enhanced with capabilities like retrieval, tools, and memory. Once external tools are connected via a standard like MCP, the LLM can go beyond generating text to actually reading files, calling APIs, and taking other real “actions” on other systems. Anthropic separately describes an agent as “an AI system that can take actions via tools — running code, calling external APIs, and messaging other agents.”

Common usage patterns

Anthropic’s materials introduce several representative patterns spanning both workflows and agents.

• Prompt chaining: breaking a task into steps and calling the LLM in sequence
• Routing: branching to a different processing path depending on input type
• Parallelization: processing several subtasks at once
• Orchestrator-workers: one agent splits work across several sub-workers
• Evaluator-optimizer: evaluating a result and iterating on it repeatedly
• Autonomous agent: unlike the patterns above, proceeding with no predefined path, judging for itself throughout

Limitations and caveats

• Lower predictability comes with more room for error. For hard-to-reverse actions (payments, deletions, outbound messages), it’s safer to add a human approval step.
• The term “agent” isn’t yet used with a fully unified definition across the industry, and emphasis can differ by company or source. This article uses Anthropic’s definition as its baseline.
• How to measure an agent’s level of autonomy is also still an active area of research.

Conclusion

An AI Agent isn’t “automation that follows a fixed sequence” — it’s “a system that judges for itself what to do next.” Whether that judgment comes from code a human wrote in advance or from the LLM itself is the key line separating a workflow from an agent. The higher-level concept of multiple agents coordinating toward a larger goal is covered next in Agentic AI.