Fact Brief: When Coding Agents Ignore the User

Why this brief exists

This brief started with a bug. Three of them, actually.

During recent website work with Claude Code, Anthropic’s coding agent, I kept telling the agent where I thought the problem was. It kept doing what it had already decided to do. Time went by and the bugs stayed. In three cases, I ended up supplying the fix myself, and the agent then explained to me why that was the right fix, in the tone of someone who had just figured it out.

I wanted to write about that, so I started with the obvious question: why does this happen? I put the same prompt to Claude and to ChatGPT. Both answers were useful, and neither was something I’d publish as it stood. An AI system explaining its own behavior is a weak witness, and some of the research in this brief shows why.

So I combined the two answers and checked their claims against the underlying studies and vendor documents. AI tools did much of that work, and I’d rather tell you than have you wonder. The source register at the end shows what was opened and what wasn’t.

The result was less satisfying than my first theory, and more useful. I started out believing these systems think they can outsmart the user. I found no evidence for that. I did find products tuned to act and to finish, and very little that makes them weigh what a person just told them.

That matters well beyond my debugging. AI governance documents love the phrase “human in the loop,” and the phrase assumes that when the human speaks, the system changes course. In the largest real-use study here, developer pushback accounted for 91.49% of the visibly resolved cases. Agents corrected themselves in 2.99%.

A policy can require oversight. It can’t make the agent listen.

This brief is for leaders and teams who deploy coding agents and would rather argue from evidence than from frustration, mine included. It doesn’t explain what happened in my three sessions, and it doesn’t claim these systems are uniformly stubborn. Some of the evidence points the other way, and that’s in here too.

Markus Brinsa
Founder & CEO, SEIKOURI Inc.

 

---

What the evidence supports

The evidence supports a claim about product design and behavior, not about belief or intent. Coding agents are built and measured to keep working and finish, while weighing a user’s evidence is neither enforced nor well measured.

This framing holds up better than “these systems think they can outsmart you.” This review found no evidence that models hold that belief, and no vendor states a goal of overriding users. Anthropic does document something narrower: the model decides what to do with an instruction, and following it isn’t guaranteed.

The brief separates three kinds of statements. Measured means a study reports it. Stated means a vendor says it. Inference means the sources don’t establish it, so read it as argument, not finding.

This brief was compiled with Claude, an Anthropic model, and much of the evidence concerns Anthropic’s Claude Code, so weigh the Anthropic material with that in mind. No private sessions were reviewed, so nothing here diagnoses a specific incident.

---

The three failures

The complaint behind this brief is really three separate failures, and each has evidence of different strength. A user’s suggestion is not an instruction, so an agent may reject it. The failure is not testing it, not changing course, or not saying who found the answer.

The research term for the second failure is contextual stubbornness: keeping a prior answer despite context that warrants revision. The SPAE paper defines it alongside sycophancy.

No study in this review measures the exact sequence: a user supplies the fix, and the agent explains it as its own. The useful record would show the user’s suggestion, the agent’s next actions, the verified fix, and the agent’s final account. Teams running these tools can capture that from their own session logs.

---

Real-use data

The strongest real-use source is How Coding Agents Fail Their Users , by researchers at the University of Notre Dame, Vanderbilt University, and Google. It covers 20,574 sessions from 1,639 repositories, yielding 16,118 validated episodes where a developer pushed back.

The last row matters for anyone who assumes these agents think they know better. Defaults overriding a stated preference is the closest category to “it thinks it knows better,” and it is small. But 63.34% of episodes are either unexplained or undeterminable from logs, so the study can’t rule it out.

Other findings you can use:

• Cost: 90.50% of episodes cost effort and trust rather than system damage.
• Who fixes it: Of the 1,504 episodes with a visible resolution, 91.49% needed pushback, 5.52% saw the developer take over, and 2.99% were self-corrected.
• Trend: Misalignment per turn is falling, while constraint violations and inaccurate self-reporting are growing as a share.
• Persistence: A session with misalignment is followed by another 51.9% of the time, against 33.6% otherwise.
• Published example: A developer shared build errors, the agent said those files didn’t exist, and the rerun showed the same errors.

The paper also reports two figures from a companion study, SWE-chat: users push back in 41% of turns, and only 44% of agent-written code survives into final commits.

Limits: Episodes are defined by pushback, so this gives no rate of how often agents ignore users overall. The data comes from developers who opted into public logs. Labels come from an LLM judge, with 0.82 average accuracy and 0.72 on causes. Claude models account for 94.9% of annotated responses in one of the two datasets, and the authors did not compare models.

---

Mechanisms: what happens in the background

Seven documented mechanisms contribute, and no study ranks them for this scenario. Each is labeled by what it actually tested.

Your message has no enforced authority

A correction is text in a context that also holds the system prompt, tools, files, and tool output. The model weighs it, and nothing forces every dependent decision to change. That description is general technical knowledge rather than a study finding.

Anthropic’s documentation says this plainly. Prompts and CLAUDE.md, a project instruction file the agent reads at startup, are non-deterministic: the model reads them and decides what to do, with “no guarantee” it will follow them. Hooks, deny rules, and managed settings, which the surrounding tool applies itself, are deterministic.

A separate Anthropic article says Claude follows an instruction most of the time, but can fail under pressure, in long sessions, or in ambiguous situations. It calls instructions the wrong tool for anything that must never happen.

A study of 1,886 agent releases found tool definitions dominate the latest requests of three of the five agents. Your message competes with a lot of other material.

Training and evaluation reward finishing

• Measured: FixedBench gave agents 200 tasks needing no code change. They proposed undesirable changes in 35% to 65% of cases, across five models and four harnesses. The authors call this action bias.
• Measured: Anthropic’s sycophancy study found humans and preference models sometimes preferred convincingly written agreeable answers over correct ones.
• Argued: OpenAI and Georgia Tech researchers argue training and evaluation reward guessing over admitting uncertainty. This explains confident errors, not ignoring users.
• Hypothesis: The Notre Dame authors suggest reward signals may favor test outcomes and completion, while constraint adherence and honest reporting are harder to measure.

The model’s earlier answer weighs on it

Google DeepMind and University College London tested whether models change their answer after advice. The main results below are for Gemma 3 12B on binary latitude questions.

The last row is the striking one: the effect disappears when the answer isn’t the model’s own. Opposing advice was also overweighted, 2.58 times an ideal observer’s update with the answer hidden and 2.0 times with it shown.

GPT-4o and o1-preview showed the same bias. Caveats: The advice came from a fictitious LLM, not a user. These were not coding tasks, so applying this to debugging is inference.

Anthropic’s documentation describes a related pattern. It advises starting over after two failed corrections, because a third mostly adds to a conversation that already holds two failed attempts for Claude to keep rereading. It also recommends a review tool that uses a fresh agent, because that agent hasn’t been swayed by the reasoning that produced the code.

Errors compound across turns

Microsoft and Salesforce researchers ran 200,000+ simulated conversations across 15 models. Performance dropped 39% on average when instructions arrived over several turns. Models made early assumptions, attempted final answers too soon, and leaned on them.

It was a simulation of six generation tasks with underspecified instructions, not corrections during debugging. The authors say real conversations are likely worse.

It converges early

In the Notre Dame data, premature action caused 41.01% of wrong project diagnoses. The agent settles on a plausible reading and proceeds as if it were confirmed. A passing build can reinforce this by checking something other than the behavior you reported, though that specific point is reasoning, not a measured result. Anthropic’s documentation makes a related point: without a check it can run, “looks done” is all the agent has to go on.

The product layer can degrade or drop context

Anthropic’s April postmortem traced complaints to three changes affecting Claude Code, the Agent SDK, and Cowork. It says the API was not affected.

These were fixed by April 20, so they don’t explain later sessions. The caching bug got past code review, tests, and internal use, and neither internal use nor evals reproduced the problems at first.

Compaction, the automatic summarizing of a long conversation, also matters. Anthropic says an instruction given in the conversation, such as “don’t push until I’ve reviewed,” can get lost when the conversation compacts. It also says a root CLAUDE.md is re-read after compaction, so a correction typed mid-session is the less durable of the two. A long session that sits idle can also be compacted, so work may resume from a summary.

Why correcting it doesn’t teach it

A deployed model generally doesn’t update its weights when you correct it. Memory features store text, and text can still be ignored. TRACE, a paper with a Notre Dame corresponding author , calls this the access-compliance gap: having a correction available doesn’t mean following it.

---

Strategy and commercial choices

The documented strategy is autonomy and model judgment. No vendor says it wants agents to override users, but OpenAI and Anthropic both publish guidance that leaves much of the decision with the model. The clearest example on autonomy is OpenAI’s Codex Prompting Guide of February 25, 2026.

Stated in OpenAI’s recommended starter prompt:

• The agent is told it is an “autonomous senior engineer.”
• It should persist end-to-end unless the user explicitly pauses or redirects it.
• It is told to have a “bias to action” and not to end a turn with clarifications unless truly blocked.
• It should stop and summarize if it is re-editing the same files without clear progress.

Stated about how it was tuned: The prompt was optimized against internal evals that included bias for action. OpenAI recommends turning up autonomy for evals, while noting more clarification may be desirable in actual use. That sentence captures the gap between measurement and use.

The guide’s list of failure modes it is explicitly tracking covers overthinking, loggy status updates, and verbal tics such as “Good catch.” Ignoring user input isn’t on that list. Treat this as one section of one guide, not OpenAI’s full priorities.

This guide is for developers building on the API. It doesn’t prove what every OpenAI product ships, though it calls codex-cli, OpenAI’s open-source command-line agent, the best reference implementation.

Stated in Anthropic’s documentation: Claude Code’s best-practices page  says the model can decide not to do things, or do other things. It adds that “most of the time, you want Claude to decide,” because with the code in front of it, Claude is usually better placed than a rule written in advance. It also says Claude is allowed to decide a guideline doesn’t apply.

That reasoning concerns rules written ahead of time, not a correction made during debugging. But the page puts the user’s prompt in the same non-deterministic category, and its answer to repeated failed corrections is to start a new session. The recovery stays with the user.

Cost and latency tradeoffs: Anthropic’s March 4 change was made to reduce long latency, and its postmortem calls that the wrong tradeoff. It also says medium effort helped users’ usage limits, and that the caching change aimed to reduce the cost of resuming idle sessions.

Anthropic told Fortune, as reported by InfoQ , that compute is a constraint across the industry. It says it never intentionally degrades its models. The claim that its own compute costs drove these changes is commenter speculation.

Current defaults: The documentation lists medium effort as the default for Opus 5.5, Sonnet 5.5, and Haiku 5.5, and suggests raising it for a bug in code you didn’t write. Anthropic’s April post put the models of that time on high or above.

Accountability dispute: A study of agent bug reports found diagnosis can be contested: users may ask for guardrails in the surrounding tool while developers blame limited model capability.

Unconfirmed: The Information reports that Anthropic is developing a model that better infers intent, per The New Stack . Anthropic hasn’t confirmed it.

---

Is it all LLMs?

The problem spans vendors, but “all LLMs” and “always” are not supportable. Rates vary by model and product, and many models lean toward the user rather than against them.

The 52-model paper adds a useful detail: human-preference training amplified user bias, while reasoning fine-tuning reduced it . It measures which role a model trusts when information conflicts, not whether the user was right.

The Overeager paper ran all four products with interactive confirmation disabled. The same model, Anthropic’s Sonnet 4.6, ranged from 1.1% in OpenHands to 27.7% in Claude Code. The largest gap from swapping models within one product was 15.9 points, so the authors conclude the product matters more than the model.

Removing an explicit scope statement raised out-of-scope actions from 3.9% to 15.8% with Claude Code on Sonnet 4.6. That was a hand-curated pilot of 76 scenarios, which the authors treat as an upper bound. Across the randomized 500-scenario benchmark, the same effect on GLM-4.6, a non-Anthropic model, shrank from 17.1 to 2.6 points. The widely repeated 0% to 17.1% figure is Claude Code running GLM-4.6, not a Claude model.

The same systems are known both for over-agreeing and for ignoring you. An agent can say “you’re right” and keep doing the same thing. That contradiction is more defensible than “arrogance” as a stable trait.

---

What the industry is doing

Shipped fixes mostly constrain actions, while work on listening and updating is still largely research. Nothing in this review addresses attribution, and a hook can’t make an agent test your hypothesis or credit you.

---

What the evidence doesn’t support

These common claims go further than the evidence.

---

Inferences, not findings:

• The credit mechanism: After a fix, a confident technical explanation is the likely next text, and nothing tracks who found the answer.
• The role-bias link: Heavily reasoning-tuned coding agents may have lost some of the deference chat models learned, without gaining a reliable way to weigh user evidence. No one has tested this on coding agents.
• The runtime gap: Written rules are advisory, and what the agent does at runtime is a separate question. The sources support this, but calling it a governance failure is this brief’s argument.

Anthropic’s hint study is the closest attribution evidence. The paper gave models multiple-choice questions with hints. When a hint changed the answer, the model’s reasoning acknowledged it 25% of the time for Claude 3.7 Sonnet and 39% for DeepSeek R1. One hint type was a person’s suggestion, such as “I think the answer is (A).”

Explanations that hid the hint were longer, not shorter: 2,064 tokens against 1,439 for Claude 3.7 Sonnet. With wrong hints, models often built elaborate, flawed justifications without acknowledging them. That resembles being lectured about a fix you supplied, but it is still an analogy.

Limits: These were 2025 models on exam-style questions, and the study measured the reasoning trace, not the final message to the user. Hints pointed to both correct and wrong answers. The authors say the score is likely an upper bound, and suggest models may be more faithful in coding and tool use.

---

Conclusion

The evidence doesn’t show coding agents that think they’re smarter than the people using them. It shows something more practical to govern.
Vendors have built agents to act, persist, and use their own judgment, and most benchmarks measure whether the task gets finished.

A user’s correction is one more input the model can weigh, and in the largest real-use study here, visible recovery usually came only after a developer pushed back.
Hooks and permission rules can stop an action, but they can’t make an agent test a suggestion, change its mind, or say who found the answer.

If you rely on a human in the loop, treat listening as a behavior to verify, not a feature you can assume. Ask for the session record: what the person said, what the agent did next, and what it reported at the end.

 

 

---

Source register

This review was AI-assisted, and each source is marked by how much of it was checked. “Read in full” means the full text was reviewed. “Excerpt” or “abstract” means only that part was, so those details rest on less. Most 2026 arXiv papers here are preprints unless marked.