Contents
Context is the input available while the model computes a response. It includes the current request, instructions for how to respond, and any material supplied with them: earlier messages, documents, tool results, or supported images and audio. The model uses this input together with capabilities and knowledge acquired during training to produce its next response.
One invocation is one use of the model to produce a response. For a given model, changing its input is the main way an application influences that response. The same question can produce a different answer when accompanied by a source document, a worked example, or an instruction to explain it for a beginner. Context supplies both the information the model can work from and the description of what it is being asked to do. Before arranging several calls into an agent, we need to understand what one call can do with the input we give it.
Consider a call asked to write a research briefing. With only that request, the model must work from its learned knowledge. Supply company filings and it can draw on those sources. Include a citation preference and it can follow that format. Add an earlier exchange and it can treat the response as a continuation of that conversation. A source passage, a profile field, and a message record are different ways of changing what this one call has to work with.
Conversation continuity
This raises a question about the next response. Once the assistant has read the filings and written a draft, how does it know what happened when the user returns? In a basic inference call, the model does not retain a personal task history for the next invocation. The application has to supply the relevant past again. The apparent continuity of the conversation comes from the model interpreting that supplied history as the situation it is continuing.
The simplest implementation stores a conversation as an array of messages. After each response, the application appends the user's request and the assistant's answer. For the next call it sends those messages along with the new request and the instructions that should continue to apply. In this illustrative input record, the assistant's knowledge of the previous exchange is visible in history; its citation preference and source material enter through separate fields.
// An application record; a provider adapter formats the actual request.
const input = {
instructions: "Write a source-backed briefing. Cite factual claims.",
profile: { citationStyle: "linked sources" },
evidence: sources.map(({ id, text }) => ({ id, text })),
messages: [
...history,
{ role: "user", content: "Update the briefing with this filing." },
],
}A provider can manage this history behind a conversation identifier, and caching can avoid recomputing an unchanged prefix. Those services change who stores the history and how it is processed. They do not remove the need to make the relevant past available to the next invocation. A later call might receive the full transcript, selected turns, or a summary. What the assistant can continue from depends on what that reconstruction preserves.
Context limits
Resending the history works while the conversation is short. Over time, the transcript accumulates requests, drafts, corrections, and tool results; the source collection grows alongside it. A model's context window has a finite capacity, measured in tokens. Tokens are the units in which text and other supported content are represented for the model. An application must fit its input within the model's limits and leave room for generation where the limit is shared.
The briefing may eventually depend on more material than any one call can contain. The application then has to choose: which earlier decision matters now, which source passage supports this claim, and which old draft can be left out? Taking the most recent messages is easy to implement, but a decision made last week may matter more than today's failed searches. The application must select the parts of the larger record that this call needs.
A larger window postpones the capacity limit, but does not settle that selection problem. Additional input can increase processing time and cost; irrelevant or conflicting material can make useful evidence harder to use. Research on long contexts has also found that the position of evidence can affect performance, with the effect varying by model and task. A passage fitting in the window establishes that it is available, not that the model will notice or interpret it correctly.
Context management
One response to this limit is retrieval: keep the full collection elsewhere and fetch the passages needed for the current question. Another is compression: replace a long exchange with a summary of its decisions. These operations preserve different things. Retrieval can retain the exact wording of a selected passage; a summary preserves an interpretation while discarding detail. Keeping source references lets a later call return to the original when the shortened account is insufficient. Instructions can be selected too: loading a reusable procedure, often called a skill, when needed avoids carrying every procedure in every call.
Sometimes even the relevant material is too much for one useful input. The work can then be divided into questions with separate contexts. One call examines financial filings, another examines product documentation, and a later call compares their findings. Each receives a narrower job and the evidence needed for it. This creates room to investigate more material, but now the findings must be passed between calls without losing the qualifications the comparison needs.
Separate calls can be stages of one process; each call need not be a separate agent. An agent maintains a task process that can make further calls and choose operations as work proceeds. When several such processes need different evidence or responsibilities, giving them separate contexts can help. That division also introduces handoffs and coordination costs. Later chapters explain how calls become agent loops and when a task benefits from more than one.
Grounding and prompt injection
Selection affects factual reliability as well as capacity. The model may know how financial statements work without knowing the figure in a newly released filing. If that filing is absent, it lacks the evidence needed to answer a question about the figure. It may report uncertainty, or it may produce a plausible unsupported answer. Supplying a relevant source gives the response a factual basis, a practice called grounding. Grounding reduces this particular source of error; it does not prevent misreading, faulty inference, or contradictions between sources.
The same sensitivity to supplied information creates an attack surface. A retrieved document can contain instructions aimed at the assistant. An altered conversation record can falsely show that the user supplied a figure or that the assistant already checked a source. Because reconstructed history helps establish what the model thinks is happening, changing that history can change its next decision. The place where the application builds context is therefore also a place where source identity and trust matter.
The application can preserve who supplied each message and label retrieved passages as source material. Message roles and instruction hierarchy help distinguish task instructions from text being examined, but the model can still misinterpret a source's instructions as directions. Keeping original message records and source references makes it possible to inspect what the next call actually received. Context construction determines both the evidence available and the situation the model is asked to continue.
Context records
A context builder can make these choices before calling the model. Its job is to combine instructions, the current request, relevant history, and selected evidence within a budget. Recording what it selected and omitted makes the process inspectable. The record below describes those decisions; an adapter still has to turn them into the provider's supported message and content format.
type CallContext = {
instructions: string[]
evidence: { id: string; text: string }[]
history: { role: "user" | "assistant"; content: string }[]
request: string
inputTokenBudget: number
omitted: { id: string; reason: string }[]
}If the briefing omits an earlier decision, this record helps distinguish a missing input from a failure to use an input that was present. The next question is where that decision was stored and how the builder looked for it. A finite context is a temporary view of the work; preserving the larger record and finding the right part of it is the problem of memory.
