You have already used a language model: you send it a prompt, it sends back text. An agent is what you get when you wrap that model in a structure that lets it do things, decide what to do next, call a tool, look at the result, and repeat until a task is finished. The model is still at the centre, but it is now making choices, not just writing sentences.
That shift, from "answer once" to "work through a task," needs a few moving parts. This guide walks through the six that show up in almost every agent, then shows how they run together in a single loop.
The six parts, at a glance
Click through each part to see what it does. The rest of the guide takes them one at a time.
The model, the brain
The language model at the core. It reads the situation, reasons about what to do next, and decides which action to take. In an agent it is choosing steps, not just producing text.
Tools, how it acts
The model can only generate text on its own. Tools let it actually do things: search the web, run code, query a database, call an API. The model decides which tool to use and with what inputs.
Memory, what it remembers
Short-term memory is the current conversation; long-term memory persists across sessions in files or a vector store. Without memory, every step starts from zero.
Planning, breaking a task down
Hard tasks need more than one step. Planning turns a fuzzy goal into an ordered list of smaller actions the agent can actually carry out.
Error handling and reflection
Actions fail: a tool errors, a result looks wrong. A good agent notices, reasons about what went wrong, and adjusts its approach instead of pushing on blindly.
Guardrails, staying in bounds
Limits that keep autonomy from becoming expensive chaos: step caps, permission checks, input and output validation. The more freedom an agent has, the more these matter.
Six parts. The model reasons, tools let it act, memory gives it context, planning sequences the work, error handling keeps it on track, and guardrails keep it safe.
The model: the brain
At the centre of every agent is a language model doing the reasoning. It is the same kind of model you met earlier, but its job has changed. Instead of writing a final answer straight away, it is asked to think about a goal, decide what needs to happen next, and choose an action.
This is the real dividing line between a chatbot and an agent. A chatbot maps your message to a reply. An agent uses the model as a decision-maker in a loop: given everything I know so far, what should I do next? The quality of that reasoning is what makes an agent feel capable or clumsy, which is why stronger models generally make stronger agents.
The mental model: the LLM is the brain. Everything else in this guide exists to give that brain hands (tools), a notebook (memory), a checklist (planning), and some fences (guardrails).
Tools: how it acts
A model on its own is, as one description puts it, a brain in a jar. It can reason about how to look something up, but it cannot actually reach out and do it. Tools are the fix. A tool is any external function the model is allowed to call: a web search, a database query, a piece of code, a calculator, an API, even another model. The model does not need to know how the tool works inside, only when to reach for it and what to pass in.
The flow is always the same: the model requests a tool with some inputs, your system runs it, and the result comes back for the model to read. Click a tool to see what that exchange looks like.
The model chooses the tool and the inputs; the observation that comes back becomes new information to reason about.
Where you have seen this: tools are function calling from Stage 3, scaled up. Back then you defined a function and let the model decide when to call it. An agent does the same thing, over and over, as part of a larger task.
Memory: what it remembers
Without memory, every turn starts from zero: the agent would forget what it just did and repeat itself. Memory comes in two flavours, and it helps to keep them separate.
Short-term memory is the current conversation, everything sitting in the model's context window right now: your request, the agent's own reasoning so far, and the results of the tools it has called. It is fast and immediate, but limited: when the context window fills up, older turns have to be summarised or dropped. Think of it as the agent's working desk.
Long-term memory persists across sessions. It lives outside the model, in files, a database, or a vector store, and the agent retrieves from it when it needs to. This is what lets an agent remember your preferences from last week or look something up it learned before. If that retrieval-by-meaning idea sounds familiar, it is exactly the embeddings and vector search from the RAG stage, pointed inward at the agent's own history.
The tie to RAG: long-term memory and RAG are the same machinery. Both embed information, store it, and fetch the relevant pieces on demand. RAG points it at your documents; agent memory points it at the agent's past.
Planning: breaking a task down
Some questions can be answered in one step. Most real tasks cannot. Planning is the agent turning a broad goal into an ordered set of smaller actions it can actually carry out, and adjusting that plan as it learns more.
You will hear a few named approaches:
- Chain of Thought: think step by step, working through the problem in order before acting.
- Tree of Thoughts: consider several possible paths, then pick the most promising one.
- Reflexion: when an attempt fails, reason about why and retry with what you learned.
The common thread: instead of blurting out the first answer, the agent slows down, lays out the steps, and gives itself a chance to catch problems before they compound. That is also why planning and reasoning are so tightly linked, planning is the model reasoning about sequence.
Error handling and reflection
Actions fail. A search returns nothing useful, an API times out, a tool hands back an error, or a result just looks wrong. What separates a robust agent from a fragile one is what happens next.
A weak agent charges ahead with a broken result. A good one observes the failure, reflects on it, and adapts: it recognises the step did not work, reasons about why, and tries a different approach, maybe a different tool, a reworded query, or a revised plan. This is reflection turned on the agent's own actions rather than its writing, and it is a big part of why the loop (coming up next) repeats instead of running once.
A subtle hard part: knowing when to stop is as important as knowing what to do next. An agent that cannot tell "good enough" from "not done" will loop forever or quit too early. Good error handling includes recognising success, not just failure.
Guardrails: staying in bounds
Give a system autonomy and you also give it room to go wrong in expensive ways: an endless loop burning tokens, a tool call that deletes the wrong thing, an answer that leaks something it should not. Guardrails are the limits you put in place so freedom does not turn into chaos.
- Input guardrails check and clean what goes into the agent (and into its tools).
- Output guardrails validate what comes out before it reaches a user or another system.
- Operational limits: a cap on steps or tokens, a sandbox for code, and human approval before anything risky.
Guardrails are not strictly part of the "anatomy," but the more autonomy you hand an agent, the more they matter. You will meet them again, in depth, in the AI safety stage.
Putting it together: the loop
None of these parts do much alone. What makes an agent an agent is that they run together in a cycle: the model reasons about what to do, acts by calling a tool, observes the result, and reasons again, using memory and planning along the way, until the task is done. This is often called the reason, act, observe loop.
Step through a real example below. The goal is a genuinely multi-step question, exactly the kind a single prompt would get wrong.
Notice step 6: the agent spotted a gap in what it had gathered and ran another search before writing. That is reflection and planning at work, and every claim in the final brief traces back to a source.
The whole stage in one sentence: the model is the brain, tools are its hands, memory is its context, planning sequences the work, error handling keeps it honest, guardrails keep it safe, and the loop is what ties all six into something that gets real work done.