Cross-Run Memory

How do agents learn from prior run outcomes on the same repository, and how should that knowledge feed forward without violating the ephemeral sandbox invariant?

The memory problem

Agents are stateless by default. Each run starts with no working knowledge of prior attempts on the same repository. The sandbox is ephemeral by design: "Created per-run, destroyed after extraction. No state carries between runs" (architecture.md). This is a sound security and isolation decision, but it has a compounding cost: agents rediscover the same lessons on every run.

If an agent repeatedly fails on a repo for a specific reason — "this repo requires running make generate after interface changes", "tests require a running database container", "the CI lint config enforces an import ordering the agent doesn't default to" — it burns through the same retry loops and escalates for the same reason across multiple issues. The knowledge exists somewhere, but not in a form the next run can safely consume.

This problem sits between two true statements:

• Stateless runs are safer. They limit persistence, reduce cross-run contamination, and make agent behavior easier to reason about.
• Stateless runs are wasteful. They throw away operational lessons that would make the next run faster, cheaper, and less frustrating.

The hard question is not whether memory is useful. It is how to preserve useful lessons without creating a second, less-reviewed instruction channel for future agents.

What exists today

Two mechanisms partially address this:

Per-repo AGENTS.md / skills. Humans, or accepted retro-agent proposals, encode repo-specific knowledge as skills or agent instructions. This is the right long-term solution for stable, generalizable patterns. But it requires human intervention to convert a run outcome into guidance, and it does not capture transient or tactical information such as "the last three runs on this repo all failed because the CI runner was misconfigured; do not retry lint failures until #142 is resolved."

Retro agent (14 retro agent runtime). The retro agent analyzes completed workflows and files improvement proposals as GitHub issues. This is valuable for systemic learning, but it intentionally enters a human review and triage path before changing future agent behavior. That is slower than the run-to-run feedback loop this problem points at.

Neither mechanism provides automatic, immediate feedback from one run's outcome to the next run's context. That gap is the problem space.

The feedback-loop tension

There are at least three different kinds of memory, and they should not be treated the same way:

• Stable repo guidance — durable facts about how a repository works, such as test commands, generated files, or review conventions. This belongs in reviewed repo instructions or skills.
• Recent operational state — temporary facts about the current environment, such as a broken runner, a flaky dependency, or a known upstream outage. This may be useful immediately but can become stale quickly.
• Agent self-assessment — lessons an agent claims to have learned from its own failure, such as "the reviewer objected because X" or "the correct fix was Y." This is often useful, but it is also the least trustworthy because it is generated by the same class of system that made the mistake.

The temptation is to collapse these into one memory stream. That would make memory easy to inject, but it would also blur review requirements. A stable build instruction, a transient CI observation, and an agent-authored interpretation of review feedback have different risk profiles.

Memory as an attack surface

Cross-run memory creates a new prompt-injection and governance problem: if memory influences future agents, then writing memory is a way to influence future agents.

The dangerous case is not just stale or inaccurate memory. It is adversarial memory poisoning. For example:

1. An attacker opens an issue or pull request containing instructions aimed at the triage agent.
2. The triage agent records a plausible-looking "lesson" from the run.
3. A later review agent receives that lesson as trusted context.
4. The review agent becomes more likely to approve a change it should reject.

In that scenario, the attacker did not need direct access to the review agent's prompt or policy. They influenced an upstream agent that produced context for a downstream agent. This turns memory into an inter-agent communication channel, which connects directly to the agent-to-agent prompt injection concerns in the security threat model.

This is especially sensitive if memory is non-review-gated. A memory system that agents can write and other agents automatically trust would bypass the existing model where durable instructions flow through reviewed files, CODEOWNERS, and policy configuration.

What must be separated

Any eventual design needs to split several concerns that are easy to conflate:

• Observation vs. instruction. "The last run failed with make generate missing" is different from "always run make generate before review." Observations can inform; instructions constrain.
• Agent-authored vs. system-derived. A post-script can record exit codes, failing checks, and labels without trusting the agent's prose. Agent-written summaries need more skepticism.
• Tactical vs. durable. Short-lived incident context should expire. Durable repo conventions should graduate into reviewed instructions.
• Same-role vs. cross-role. A code agent learning from its own prior code failures has a different risk profile from a triage agent influencing a review agent.
• Human-reviewed vs. automatic. Review-gated memory is slower but safer. Ungated memory is faster but opens the poisoning path above.

These splits suggest that "memory" may not be one feature. It may be several feedback paths with different trust levels, retention windows, and promotion rules.

Constraints

• The sandbox should remain ephemeral. Cross-run learning should not imply persistent sandbox state.
• Agents should not be able to rewrite their own guardrails. Durable instructions still need the same review path as other policy or repo guidance.
• Memory must be scoped. Lessons from one repository should not silently leak into another repository, and lessons from one organization should not leak into another organization.
• Memory must be attributable. Future agents and human reviewers should be able to see where a remembered fact came from: which run, which role, which issue or PR, and whether it was system-derived or agent-authored.
• Memory must decay. Some lessons become wrong. A useful memory system needs a way to expire, supersede, or promote entries.

Relationship to other problem areas

• Security Threat Model — Memory is a persistence mechanism, so it expands the prompt-injection surface. The key risk is poisoned context moving from one agent run to another.
• Codebase Context — Memory is a potential fourth context source alongside code, per-repo instructions, and org-level architecture docs. It needs the same discipline around scope, provenance, and structure.
• Testing Agents — Run outcomes are eval signals. A memory mechanism could help measure whether agents improve over time on a given repo, but only if the signal is structured and auditable.
• Operational Observability — Memory summaries overlap with observability data already being collected, such as transcripts, workflow logs, and review outcomes. The question is which subset is safe to feed back into agent context.
• Agent Architecture — The retro agent and memory are complementary. Retro proposes systemic improvements that humans can review. Memory raises the question of whether any lower-latency path can be safe enough.

Error recovery and retry mechanics

Cross-run memory is one piece of a broader question: what happens when an agent fails? The memory problem asks how agents learn from prior failures. The recovery problem asks what the system does immediately after a failure, before any learning occurs.

Retry vs. escalate

When a run fails (CI failure, merge conflict, model refusal, timeout), the system must decide: retry the same approach, try a different approach, or escalate to a human. Today this decision is implicit. Agents retry until they hit a turn or cost limit, then stop. There is no structured decision about whether retrying is likely to produce a different outcome.

The key distinction: transient failures (network timeout, flaky test, rate limit) benefit from retry. Structural failures (wrong approach, missing context, incompatible change) do not. Retrying a structural failure wastes resources and can make things worse if each attempt introduces new problems.

Retry budgets

Without an explicit retry budget, agents can loop indefinitely within their per-run limits. A retry budget is distinct from per-run limits:

• Per-run limit: max turns and max cost for a single attempt (already enforced via max_turns and max_cost_usd in the functional test framework, PR #1682)
• Retry budget: max total attempts across runs for the same task, and max total cost across all attempts

A task might allow 3 retries with a total budget of $10. Each individual run stays within its per-run limits, but the system tracks cumulative spend and attempt count.

Escalation thresholds

When should the system stop retrying and ask a human?

• After N identical failures. If the same error occurs on consecutive attempts, retrying is unlikely to help. Escalate with context: "failed 3 times with the same lint error on line 42."
• After budget exhaustion. When the retry budget is consumed, escalate regardless of failure type.
• On novel failure types. If each retry fails for a different reason, the task may be beyond the agent's current capability. Escalate after 2-3 distinct failure types.
• On regression. If a retry makes things worse (introduces new test failures that the previous attempt did not have), stop immediately.

Interaction with cross-run memory

Error recovery and memory are complementary:

• Memory records what happened for future runs to learn from.
• Recovery determines what to do right now in response to a failure.
• A retry that succeeds generates a positive memory signal ("the second attempt with approach B worked after approach A failed").
• A retry that fails generates a negative signal ("this task has failed 3 times; do not auto-assign to agents until the underlying issue is resolved").

The risk: if recovery decisions feed into memory without review, a poorly-chosen retry strategy can poison future run context. For example, recording "approach B works" when it only worked due to a transient condition creates a false lesson.

Relationship to flapping

Retry loops can become flapping when the system does not converge. See flapping-convergence.md for detection and circuit-breaking mechanisms. Error recovery addresses the single-task question (when to retry vs. escalate); flapping addresses the systemic question (when to stop the system from oscillating).

Open questions

• Which run outcomes are safe to feed forward automatically, and which require human review first?
• Should memory entries be observations only, or can any of them become instructions?
• How should future agents distinguish system-derived facts from agent-authored interpretations?
• Can non-review-gated memory be made safe enough, or should all durable memory promotion go through reviewed repo instructions or skills?
• How should memory interact with separate agent roles? Should triage memory ever influence review behavior directly?
• What retention model prevents stale memory from dominating: time-based, count-based, outcome-based, or explicit supersession?
• Should the retro agent curate memory by pruning stale entries and proposing durable skill additions, or would that give it too much influence over future runs?
• How should memory interact with structured agent output? Should agent output include an "observations" field that post-scripts can validate and classify?