A model's behavior at any moment is a function of everything in its context window. Deciding what goes in that window, instructions, retrieval, memory, and tools, is not prompt writing. It is architecture, and it deserves the same rigor.
Celestino Salim
Senior Software Engineer
Most teams treat fine-tuning and RAG as alternatives. They are not. They are two points on a context-engineering spectrum that also includes long context with caching and distillation. Here is the decision framework I use in production.
The phrase "prompt engineering" did real damage. It framed the work as wordsmithing: find the magic phrasing, add "think step by step," sprinkle a persona on top. That framing made sense when an LLM feature was one string in, one string out.
Agents ended that era. A model's behavior at any moment is a function of everything in its context window: the instructions it loaded, the documents retrieval surfaced, the memory the system chose to carry forward, the tool definitions sitting in the prompt. Deciding what enters that window, in what form, at what time, is not a writing problem. It is a systems design problem, and I have stopped treating it as anything else.
Every context window is a finite, priced, position-sensitive resource. Tokens cost money. Attention degrades over long contexts. Content placed early behaves differently from content placed late. Filling the window is not free even when it fits.
That makes context assembly a resource allocation problem, the same shape as memory management or cache design. What gets admitted. What gets evicted. What gets summarized into a cheaper representation. What must never be dropped. When I review an agent system now, I ask for the context budget the way I used to ask for a query plan: show me what is in the window at the moment of the decision, and justify every section.
In every agent system I have built or reviewed, the context window is fed by four subsystems. Each one is usually owned by nobody and tuned by accident.
The standing rules: identity, constraints, conventions, output contracts. Instructions are code. They belong in the repo, versioned, reviewed, and layered deliberately: a global base, a per-surface layer, a per-task contract. In my own workspace a single canonical instruction file feeds every agent runtime, because two instruction files that drift apart produce two different systems that share a name.
Everything the system fetches at runtime because it could not, or should not, live in the standing instructions. RAG is the famous instance, but tool results, file reads, and API responses are all retrieval. The design questions are supply chain questions: what sources exist, how fresh they are, how relevance is decided, and what proves the right thing was fetched. Retrieval that cannot be evaluated is decoration.
What persists across sessions. Agents forget everything by default, which means memory is not a storage problem, it is a write-policy problem. The hard decisions are what is worth writing down, in what form, and when it gets loaded back. I keep agent memory in versioned files with explicit rules about what qualifies, because an append-only junk drawer of "learnings" is context rot with a nicer name.
The part most teams forget is context at all. Every tool definition, its name, description, and schema, sits in the window and shapes behavior before any call happens. A bloated tool catalog is a bloated prompt: it spends budget, dilutes attention, and invites the wrong call. Fewer tools with sharp, accurate descriptions consistently beat a full catalog. Tool descriptions deserve review for the same reason API contracts do: the consumer cannot read your intentions, only your interface.
Once you see the four subsystems, the familiar agent failures stop looking like model problems:
Every one of these has a systems fix. None of them has a lasting prompt-trick fix. Rewording will shift the failure around the way adding RAM shifts a memory leak.
The statement I keep coming back to: prompt tricks optimize a string; context engineering designs the pipeline that produces the string. It is the same shift the industry made when it stopped hand-tuning queries and started designing schemas, or stopped hand-rolling deploys and started building pipelines. The artifact stops being the thing you polish. The system that produces the artifact becomes the thing you engineer.
Practically, that means context assembly gets the full treatment: it is versioned, it is observable (you can dump the exact window that produced any decision), it is tested (evals catch regressions when a layer changes), and it has an owner. If you cannot reproduce the context that caused a failure, you are not debugging, you are guessing.
The teams that internalize this stop asking "what should the prompt say" and start asking "what should the model know right now, and which subsystem is responsible for it being there." That question has an architecture answer, and architecture is something we already know how to do well.
I teach the applied version, budgets, layering, retrieval, and memory policies for working agents, in the Context Engineering for Agents lesson of the Building AI Agents course.