A split can hide inside a normal interface
The user is looking at three selected rows. The agent has access to the dataset and no reliable way to know which three rows are selected.
Nothing in the interface necessarily looks broken. The table renders. The selection highlight works. The agent is responsive. The failure appears only when the user says, “compare these three,” and the product discovers that the phrase refers to state owned by the interface rather than state the agent can address.
This class of failure is easy to misdiagnose because the product looks unified from the user side. A chart, a result set, a selected object, a local document, and a previous tool output all occupy the same screen. That does not mean they occupy the same operational state.
The distinction is simple.
Visibility is not addressability.
An object is visible when the interface can render it. It is addressable when the agent has a stable way to identify it, recover enough of it to reason or act, and determine whether it is still the same state the user is looking at. Those are different properties, and an agentic product that leaves the mapping implicit is relying on accidental coherence.
The interface can be correct while the agent is blind to part of it.
That is the interface-side version of The Reader Changed the Documentation Contract: information can exist in a system and still fail the agent if the right operational object cannot be found, addressed, and recovered when needed.
The kitchen cannot see the table
The same failure, without the vocabulary.
You are at a restaurant with a friend. Several dishes arrive. You point and say, “another portion of these.”
Your friend can see them. The kitchen cannot. For the kitchen to act, someone has to say “table six, order forty-seven, the sea bass” — a reference the kitchen can look up in a system it actually owns.
In an agentic product, the screen is the table. The interface is sitting at it. The agent is usually in the kitchen: it knows the menu, it knows the inventory, and it has no idea what “these three” refers to, because the selection was created in the interface and never given a name the kitchen can use.
The product implies one room. The architecture may have two.
The analogy has a limit worth naming. A restaurant already runs on order numbers, so the reference exists whether or not anyone is currently looking at it. The failure class here is narrower and more awkward. In many agentic products no order number was ever written down. The user pointed, the interface understood, and nothing durable was created.
The Harmonic failure is useful because it was ordinary
LangChain’s account of Harmonic’s Scout describes a concrete version of the problem. Scout could produce a large company result set and render it for the user. A later follow-up about particular results failed because the model had never received the result set the interface was displaying.
Harmonic’s described solution was to give the result set a shared representation. Search results were written to a filesystem that the agent could inspect on demand while the frontend rendered the same data for the user.
The account states the resulting design rule in one line: “anything the user sees, the model must be able to discover.”
The filesystem is the implementation detail. The more reusable observation is that the visible artifact acquired an agent-side recovery path.
That distinction matters because copying every visible byte into model context would be a poor general rule. Large tables, document models, charts, selections, and application state can be expensive or unnecessary to serialize into every turn. The agent only needs the relevant state to be discoverable and recoverable when the product expects it to act on that state.
An identifier can do that. So can a file path, a tool-scoped handle, a document coordinate, a backend object, a client-executed capability, or another application-specific reference. The architecture can vary.
The invariant is addressability.
The Harmonic account supports the implementation it describes. It does not establish that every agentic interface should use a shared filesystem, and this article does not require one.
Visibility, addressability, and context are three separate states
A useful diagnostic starts by refusing to collapse three questions:
- Can the user see the state? The interface can render or expose it. The dish is on the table.
- Can the agent address the state? The agent can identify and recover it through some supported path. An order number exists, and the kitchen can look it up.
- Is the state currently in model context? The model already has the relevant representation in the active turn history or working context. The ticket is already on the rail in front of the line.
An object can be visible and not addressable. That is the obvious failure.
An object can be addressable and not in context. That can be healthy. A result set may live behind a stable handle and be retrieved only when needed, in the same way that nobody reads the order number aloud until someone asks about the order.
An object can be in context and stale. The agent can describe yesterday’s version while the user is looking at a local edit made thirty seconds ago. The ticket is real, legible, and no longer describes what is on the table.
And an object can be addressable for one turn but unrecoverable later. A tool returns result_4821; no later tool accepts result_4821. The product preserved a label, not continuity.
This is why the better question is not “did the prompt contain it?” The better question is: what operational object is the user referring to, and how can the agent resolve the same object now?
That is a product contract.
The identity and persistence side of the same problem appears in The Output Became a Work Object: once later behavior depends on an exact result surviving, changing, or being addressed, the state contract can no longer remain implicit in transcript prose.
The object needs a coherence contract
For any user-visible state the product reasonably invites the user to reference, query, compare, or modify, five questions are enough to expose most of the hidden assumptions:
- Identity: What stable reference distinguishes this object from similar ones?
- Discovery: How does the agent map “this,” “those three,” “the current chart,” or “the active document” to that reference?
- Recovery: If the object is outside current model context, what retrieves enough of it to reason or act?
- Freshness: How does the product establish that the recoverable representation is the version the user is currently seeing?
- Mutation: When either side changes the object, how does the other side observe the transition?
I would treat those as diagnostic questions, not as a published standard. Nobody publishes them in this form; they are assembled here from the pattern these failures share. They describe what a product has to be able to answer, not how it should be built.
What matters is that each answer exists before the interface promises shared understanding.
Without that, the state relationship is implied by proximity on a screen. Proximity is not a protocol.
Where the contract tends to break
Several mismatch classes fall out of the same contract.
Visible but unreachable. The interface renders an object that never enters any state the agent can query. A side panel can display a rich result while the agent retains only the fact that a search completed.
Reachable but unidentifiable. The agent can search the underlying data but cannot determine which items the user selected. Retrieval capacity exists; the mapping from the user’s reference does not.
Identifiable but unrecoverable. The product has a handle, but the handle cannot be dereferenced after a turn boundary, refresh, context compaction, or later tool call. The identifier is technically real and operationally useless.
Recoverable but stale. The agent can fetch the object, but not the version the user sees. This is worse than outright absence because the response can be fluent, specific, and wrong about current state.
Mutation without symmetry. The user changes something the agent can act on, or the agent changes something the user can see, and the transition is not propagated to the other side. A shared snapshot solved the initial read and left the interactive system unsolved.
These labels are not exhaustive. They are useful because they move debugging away from a vague claim that the model “forgot” something and toward the exact state boundary that failed.
The bug becomes inspectable.
Client state is where the boundary becomes obvious
LangChain’s separate article on agents and applications describes the broader client/server version of the same problem. Selection state, browser APIs, document models, editing commands, and other application-local capabilities can exist outside the server-side tool loop unless the product deliberately exposes them.
A backend agent may have excellent access to APIs and databases while remaining unable to answer a question about the active selection in the browser. Both statements can be true because they describe different state owners.
LangChain presents client-executed “headless tools” as one way to bridge that boundary. Again, the mechanism is not the rule. A client-side capability is useful because it gives the agent an addressable path to state that would otherwise remain local to the application.
The broader conclusion is narrower than a universal architecture claim: client-owned state that materially affects agent behavior needs an intentional representation or an explicit capability boundary.
The UI boundary is part of the agent harness.
Not everything visible needs an agent representation
There is an important limit here. Decorative layout, animation state, disclosure state, and other purely presentational details may have no operational relevance. Giving the agent access to every pixel would confuse observability with usefulness.
The coherence contract belongs around state the user is reasonably expected to reference or ask the agent to act on. Not every plate on the table needs an order number. The ones the user might send back do.
If the user can point at an object, select it, modify it, compare it, or ask a follow-up about it, the product should know whether the agent can resolve the same thing. If it cannot, the product can still be coherent by making that boundary explicit.
Capability is one valid answer. Declared incapability is another.
Silent shared-state assumptions are the failure.
Diagnose the state before blaming the model
A practical test begins from the interface because that is where the user’s expectation is formed.
Inventory the state the product currently exposes:
- objects the user can see and reasonably refer to;
- selections and active items;
- values the user can modify;
- generated artifacts or result sets that persist on screen;
- local application state that survives while the agent moves through later turns.
Then inspect the same set from the agent side. For each action-relevant object, ask whether the agent has a stable identity, a discovery path, a recovery path, a freshness signal, and a mutation path where one is required.
The mismatches are candidate product bugs.
This changes the debugging question in a useful way. “Why did the model forget what the user was looking at?” becomes “what representation of that visible state was the agent supposed to recover?”
“Why did it misunderstand ‘those three’?” becomes “where was the selection encoded?”
“Why did a larger context window not fix it?” becomes “was the missing object ever represented in a form the agent could retrieve?”
Those questions have owners. They can be tested.
Context capacity is downstream of representation
A larger context window can preserve more represented state. It cannot preserve state that never entered an agent-addressable system.
If the frontend rendered a result set that was never attached to a message, stored behind a handle, exposed through a tool, or synchronized to another state owner, there is nothing for a larger context window to retain. The same limitation applies to memory: it cannot recall an object the product never made recoverable, and it cannot repair a missing mapping between “these three” and the user’s actual selection.
A bigger kitchen does not help when the order was never written down.
Capacity matters after representation exists.
That ordering is easy to miss because the symptom presents as missing information. The mechanism is often missing state ownership instead.
The product boundary is the harness
The word “harness” is usually applied behind the interface: model, tools, middleware, memory, execution loop. In an interactive agentic product, that boundary is incomplete.
A tool that produces ten thousand rows is not sufficient if the UI renders those rows and later turns cannot address them. A rich client selection model is not sufficient if the agent cannot discover the current selection when asked to operate on it. A durable backend object is not sufficient if the user is looking at a newer local mutation and the agent has no freshness signal.
The product is making one promise across all of those components: when the interface invites the user to treat the agent as present in the application, the agent should have an operational path to the action-relevant state the interface asks the user to discuss with it.
The path can be a filesystem. It can be a handle. It can be a client-side tool. It can be something else entirely.
What cannot remain implicit is the mapping.
The user can see it. The agent needs an address.
// End of transmission. State needs an address — VERITAS