The word "agent" has been stretched to mean almost anything, which makes it hard to build one on purpose. Strip away the marketing and an AI agent is a specific, simple thing: a language model in a loop, given tools it can call and a goal it works toward until a stopping condition is met. Everything that makes agents powerful — and everything that makes them expensive and fragile — follows from that loop.
This is a practitioner's view of the architecture: the parts, how they fit, and where first attempts reliably break.
The core loop
At its heart an agent repeats four steps. The model receives the current context. It decides on an action — usually calling a tool or producing a final answer. The tool runs and returns a result. That result is appended to the context, and the loop runs again. It continues until the model emits a final answer or a stopping rule cuts it off.
That is the whole engine. A chatbot answers in one pass; an agent is the same model given the ability to act, observe the result, and decide what to do next. The intelligence is not in any single call — it is in the loop.
The four components
1. The model
The model is the reasoning core that decides each step. Capability matters most on the steps that involve planning and judgment; many routine steps can run on a smaller, cheaper model. Matching model to step is one of the biggest levers on both quality and cost.
2. The tools
Tools are how the agent affects the world — search, database queries, code execution, API calls. The quality of your tool definitions determines the quality of the agent more than almost anything else. Clear names, tight scopes, and descriptive schemas let the model choose correctly; vague, overlapping tools produce confused behavior no prompt can fix.
3. The context
Context is the agent's working memory: the system prompt, the goal, and the growing transcript of actions and results. It is also where cost lives, because every step re-sends it. Managing what stays in context — and what gets summarized or dropped — is a first-class design decision, not an afterthought.
4. The stopping rule
An agent without a hard stopping rule is a way to spend money in a loop. Cap the number of steps, set a budget, and define what "done" looks like. The most common production incident with agents is not a wrong answer — it is an agent that never decided it was finished.
Where first attempts break
Three failure modes account for most of them. The first is runaway loops: no step cap, so a stuck agent burns tokens until someone notices the bill. The second is context bloat: raw tool outputs pile up until every step is enormous, because cost compounds with the length of the transcript. The third is tool confusion: too many similar tools, or tools with unclear boundaries, so the model picks the wrong one and cascades from there.
All three are architectural, not prompt problems. You fix them with step caps, context management, and disciplined tool design — not by rewording the system prompt.
Budget before you build
Agent cost does not scale with the number of steps; it scales closer to their square, because each step re-sends the accumulated context as input. A task that feels like ten quick calls can cost far more than ten times a single call. Estimate this before you commit to a design — you can model steps, tokens, and caching with the AI agent cost estimator — so the economics inform the architecture rather than surprising you in production.
The takeaway
Build the loop deliberately: one capable model, a small set of sharply defined tools, active context management, and a hard stopping rule. Get those four right and the agent is reliable and affordable. Get any one wrong and no amount of prompt-tuning will save it.