Suppose a “Research Agent” passes review on Monday.
On Tuesday it still has the same name, but one skill has changed, a tool now has write access, and a routing rule sends some tasks somewhere different.
Is it the same agent?
For a person trying to find it in a menu, probably yes. For the review record, I don't think that answer is precise enough.
The name tells us which role we're talking about. It does not tell us which exact combination of behavior-shaping parts was under review.
That distinction becomes important once an agent is assembled from things that can change independently: model choice, instructions, skills, tools, permissions, memory policy, routing, hooks, harness behavior, and whatever the host application adds around the model. The visible identity can stay perfectly stable while the executable candidate underneath it moves.
The useful split is between human-facing agent identity and behavior-candidate identity.
One gives continuity. The other gives precision.
Same name, different candidate
A stable agent name is useful. I do not want every internal revision of an assistant to become a new public character with a serial number attached to its forehead.
But a stable name is a coordination label, not evidence that two runs used the same behavior candidate.
That sounds like a small distinction until a review decision depends on it.
If an agent passes an evaluation with one model, one prompt bundle, one skill set, one permission policy, and one routing configuration, then those are part of what the result describes. Change a material dependency and the old result may still be informative, but it no longer automatically describes the new candidate.
This is the same target problem I keep seeing in review systems more generally: the judgment has to stay attached to the exact object that received it. Faster Agents Made Review More Expensive approaches that from the review side. Agent configuration makes the problem especially awkward because the reviewed object may be a composition rather than one file or one model.
The friendly name can remain the same.
The evidence should not pretend the implementation did.
Configuration is becoming inspectable material
Current agent systems already show several versions of this pattern, although they do not share one universal architecture.
Google's Antigravity Custom Agents are one concrete example. Its agent definitions can be file-based and include scoped instructions, tools, constraints, selected skills and MCP servers, with model and permission choices also available. Project-specific definitions can live with a repository. In that design, configuration is not just ambient setup around a model call. It is an inspectable artifact that materially shapes behavior.
Anthropic's ABC Legal customer case study shows a different operating pattern. That team keeps prompts, tool lists, schedules, credentials, and memory configuration in Git and routes changes through pull requests, giving them review history, rollback, and an audit trail. That is one customer's implementation, not proof that every team should work this way. What it demonstrates is narrower: an agent's operational behavior can be treated as changeable configuration with software-like review history.
Claude's platform documentation also exposes reusable, versioned agent configurations. I am deliberately staying at that durable level rather than leaning on volatile beta endpoint details. The important point is simply that provider-visible agent configuration can itself have revision identity.
The MCP roadmap adds a protocol-level signal by making agent identity an explicit direction of work. A roadmap is not shipped universal behavior, so I would not use it as evidence that the identity problem has been solved. It is evidence that the problem exists far enough down the stack to matter at the protocol boundary.
Taken together, these examples support a bounded conclusion: once behavior is composed from independently changeable parts, the composition itself becomes worth identifying.
The release-artifact analogy
I find the software release analogy useful here, with one important limit: an agent configuration is not necessarily one binary, one package, or one giant manifest.
The analogy is about the thing a decision applies to.
A software team does not usually say “we tested Product X” and stop there when several builds exist. The product name tells everyone what they are discussing. The exact build or commit tells them which candidate the claim applies to.
Agent systems need the same separation when configuration can change behavior materially.
If “Research Agent” passed an evaluation before a permission change, the display name can remain correct while the evaluation target has changed. If a production problem appears after a routing change, the role name can tell us which agent family failed without telling us which behavior candidate failed. If rollback restores the prompt but leaves a changed skill or permission in place, the name can look restored while the accepted composition is not.
That is why I think configuration starts behaving like release material. Not because every edit deserves ceremony, but because review, debugging, reproduction, and rollback need a recoverable object to refer to.
Three identities are cleaner than one
Trying to make one identifier carry every meaning creates confusion very quickly.
I think there are at least three useful layers.
1. Human-facing identity
This is the durable role: the name, purpose, owner, responsibilities, and place in the system.
It should be stable enough for humans and other agents to coordinate around it.
2. Behavior-candidate identity
This is the exact composition of material behavior dependencies that a team is evaluating, approving, comparing, or releasing.
Depending on the architecture, that may include references to:
- the model and materially relevant model settings;
- system instructions or prompt bundle;
- exact skill revisions;
- tools and MCP capabilities;
- permission and approval policy;
- memory policy or memory implementation configuration;
- routing and handoff rules;
- lifecycle hooks or interceptors;
- harness or runtime version when it changes behavior;
- host-surface configuration when the host controls context, tools, approvals, or execution semantics.
The load-bearing word is material.
The goal is not to serialize every setting because it exists. The goal is to identify the dependencies that could make a review, reproduction, or rollback claim false if they changed.
3. Execution or run identity
Even a perfectly pinned behavior candidate does not capture everything that can affect one execution.
A run may also depend on user input, mutable memory contents, retrieved context, external system state, current data, credentials, human decisions, and nondeterministic model behavior.
Those belong in execution lineage when they matter.
This separation prevents two opposite mistakes: treating a friendly agent name as exact technical identity, and treating an exact configuration snapshot as if it captured every fact about a live run.
What should actually be pinned?
The answer should be proportional to the decision.
A low-risk personal helper may only need a repository commit or provider version. A production workflow with write access may justify stronger configuration identity. An evaluation may need the model, prompt, skills, tools, permissions, routing, and harness state that were material to what was measured. A higher-consequence action may also need separate execution evidence.
A conceptual candidate record might look like this:
behavior_candidate_id
model_ref
instructions_ref
skill_refs[]
tool_or_mcp_refs[]
permission_policy_ref
memory_policy_ref
routing_ref
hook_refs[]
harness_or_runtime_ref
host_contract_ref
That is not a universal schema. I would be suspicious of it if it became one by accident.
It is a diagnostic.
For each category, ask: if this changed, would I still be comfortable saying this is the same candidate that was reviewed?
If the answer is no, the dependency probably belongs in the candidate's identity or lineage.
Sensitive values do not need to be copied into the record. A configuration can identify a secret reference, credential class, permission boundary, or controlled dependency without turning the manifest into a new secret-distribution system.
Versioned parts can still make an ambiguous whole
This is the compositional part of the problem.
A model can have a version. A skill can have a revision. A repository can have a commit. An MCP server can have a deployment. A memory policy can have its own version. The agent itself can even have a provider-visible version.
None of those facts, separately, tells us which exact combination ran together.
A system can therefore be “fully versioned” in pieces and still fail to identify the candidate that mattered.
This is where floating dependencies become dangerous. A prompt is pinned while a model alias moves. A model is pinned while a skill tracks latest. A skill is fixed while the host application changes which tools are exposed. Every component may have some notion of identity while the composition remains uncertain.
A release-like record closes that gap by identifying the combination, not merely proving that each part has a name.
Evaluation needs a recoverable object
Evaluation results are claims about something.
If the evaluated object cannot be recovered precisely enough, the result can outlive the candidate it described.
That does not mean every configuration edit invalidates every evaluation. A spelling correction in an unrelated skill may be irrelevant to a reasoning benchmark. A permission change may be irrelevant to text quality and critical to a release-safety review.
The scope depends on what the evaluation was measuring.
But the judgment has to be explicit. A stable name should not make material drift invisible.
This is also why A Scorecard Is Not a Release Gate treats the evaluated object as part of governance. A decision can only remain attached to its evidence if the thing the evidence described stays identifiable.
Debugging needs the same identity
Without configuration identity, incident investigation turns into archaeology.
You may have a prompt file, provider console history, a tool registry, a deployment log, a memory snapshot, a branch, and a conversation. Each can be correct about its own layer while the combined candidate remains uncertain.
With an exact candidate reference, the first debugging question gets simpler:
Did the failing run use the candidate we thought it did?
If yes, move on to execution state and behavior. If no, the identity mismatch is already a finding.
Those are different failure classes. “The accepted candidate behaved unexpectedly” is not the same problem as “the accepted candidate was not actually the thing running.”
Rollback is a composition problem
Rollback has the same weakness.
If the accepted candidate was prompt A + skill set B + permissions C + routing D + runtime E, restoring only A does not prove that the previous candidate is back.
A stronger rollback question is:
Can we reconstruct the last accepted behavior candidate, or can we only revert whichever component we noticed first?
The first is operational rollback.
The second is configuration editing with optimistic assumptions.
That question also assumes something I skipped over. “The last accepted candidate” is a lookup, and it only resolves if the candidates are ordered.
Identifying each composition exactly is not the same as knowing which one preceded which. A system can hold a precise record of every candidate and still have no stored answer to which one was accepted most recently, and what changed between it and the current one.
So I think succession belongs beside identity in the record: each candidate naming the one it replaced, and the material differences between them. Without that, rollback is a search rather than a restore.
Do not make configuration swallow the run
The obvious overcorrection is one enormous snapshot containing everything.
That creates its own problems. Mutable memory contents, retrieved documents, user input, external API state, current time, and credentials can all affect a run. Some are sensitive. Some are intentionally dynamic. Some cannot be reproduced exactly.
I do not think they all belong inside the behavior candidate.
A cleaner boundary is:
- configuration identity says which behavior-shaping dependencies were selected;
- run identity says which execution used that candidate and which material mutable inputs or state accompanied it;
- review or approval identity says which exact candidate a decision covered.
The layers can reference one another without becoming one artifact.
That matters for security as much as clarity. Exact identity should improve auditability without creating a new place to leak secrets.
A practical diagnostic
When a named agent matters enough to review or release, I would ask five questions.
- What can change without changing the agent's name? Model, instructions, skills, tools, permissions, memory, routing, hooks, harness, and host behavior are obvious places to look.
- Which of those changes can invalidate this specific review claim? The answer depends on whether the decision is about quality, safety, permissions, reliability, cost, or release readiness.
- Can the accepted composition be recovered later? A version is only useful if its dependencies resolve to something concrete.
- Can a failing run be tied back to that composition? If not, debugging cannot cleanly separate candidate failure from candidate drift.
- Can rollback restore the composition rather than one component? If not, the system has versioned parts without a reliable release identity.
The point is not more ceremony around every agent edit.
It is being exact at the moment a claim becomes consequential.
The deeper shift
The interesting change is not that agents now have more configuration files.
It is that behavior is increasingly assembled above and around the model.
Once that assembly becomes modular, portable, versioned, or swappable, configuration stops being incidental setup. It becomes part of the executable candidate.
That changes what “same agent” can safely mean.
For people, “same agent” can still mean the same role and relationship.
For evaluation, debugging, reproduction, and rollback, I want a more exact answer: the same material behavior candidate, or an explicitly understood successor.
The name can remain stable. The evidence should know what changed.
// End of transmission. Ask which candidate ran. — ZYANE