A support agent receives the message “Please refund my last order.” The customer has three orders from the past month. The agent picks the most recent one, issues CHF 300 back to the card, and closes the ticket. Two days later the customer writes again: the refund was meant for the order before. The money is gone, the second order is still disputed, and an employee now spends an hour untangling both.
The agent worked as built. It filled a gap with an assumption, which is what language models do by default. Every multi-step workflow with an AI agent needs a few designed points where the agent stops and asks, and those points cost far less than the cleanup after a wrong guess. This article covers why agents assume instead of asking, where the evidence says to place the stops, how to price a question against an error, and how to avoid the opposite failure of asking too much. If you want to design these points for a workflow of your own, our AI development service starts with exactly this kind of decision map.
Scope note: this piece is about the question and approval points inside one workflow. For the wider operating model, see Managing AI Agents in Production, and for policy, ownership and audit duties see AI Agent Governance.
Why Agents Assume Instead of Asking
A person who receives a vague request tends to notice the vagueness. A model tends to produce an answer. The research on multi-turn conversations shows what that costs.
Laban and colleagues tested this in “LLMs Get Lost In Multi-Turn Conversation”. They split a complete instruction into small pieces and revealed them to the model over several turns, with another model playing the user. The study covered 15 models from eight families and more than 200,000 simulated conversations. According to the paper, average performance fell from about 90% in single-turn runs to about 65% in the multi-turn setting. The authors report the average drop as 39% in the abstract and 35% in a figure caption, so the honest summary is a fall of roughly 35 to 39%, depending on where in the paper you read.
The breakdown is more useful than the headline. The best-case performance of the models dropped only modestly, by about 16% on average. The gap between their best and worst runs grew by about 112%. In plain terms, the models could still do the task, but they became far less consistent. The authors describe a pattern in which models make assumptions in early turns, attempt final solutions too soon, and then fail to recover from a wrong turn.
Read the scope before quoting it. The 90% to 65% figures are absolute scores, while the 35% and 39% figures come from the paper’s own average-degradation measure, so the two do not reconcile by simple arithmetic. The six generation tasks covered code, SQL, API actions, math, data-to-text and summarisation, the conversations were simulated and automated, the text was English, and the models are 2025 vintage. The number does not mean that language models are 39% worse in general. What it supports is narrower and still relevant to operations: when information arrives in pieces, an agent that commits early is less reliable than one that checks.
Single-Run Success Hides the Problem
Operations leads usually see a demo, and a demo is one run. The tau-bench paper makes the gap between one run and many runs measurable.
Yao and colleagues built tau-bench to test an agent that talks to a simulated user while calling tools under policy rules, such as those in a retail or airline service desk. Success is judged by whether the final database state matches the goal. The paper defines a metric called pass^k, which it describes as the chance that all k independent trials of a task succeed, averaged across tasks. A pass^1 score is a single run. A pass^8 score asks for eight successes in a row on the same task.
According to the paper, the best model it tested, gpt-4o, reached a pass^1 of 61.2% on the retail tasks and 35.2% on the airline tasks. On retail, its pass^8 was below 25%. The same agent that looks acceptable once becomes unreliable when you need it to behave the same way every time, which is what a business process needs.
Two limits apply. The models are from 2024 and current systems will differ. The lesson that holds is structural: a single success rate overstates how safe it is to let an agent act alone on a consequential step. That is the case for a checkpoint where the action cannot be reversed.
Where to Put the Stops in a Workflow
Vendor guidance and the research point to a short list of places. They are practitioner consensus rather than measured thresholds, and the sources differ in how firmly they say it.
Before an irreversible or external action
OpenAI’s practical guide to building agents names high-risk actions as one of two primary triggers for human intervention. It describes them as sensitive, irreversible or high-stakes actions, with canceling orders, authorizing large refunds and making payments as examples. Microsoft’s AI Red Team goes further and recommends tiered approval requirements that scale with action reversibility and blast radius. Sending an email to a customer, writing to an ERP, moving money and deleting a record all belong here.
On a missing required field
If a step needs an order number, a delivery address or an approval limit and the input does not contain one, the agent should ask for it rather than infer it. The evidence here is thin but consistent. In the CRMArena-Pro benchmark by Salesforce AI Research (Huang et al.), the authors sampled 20 failed runs of one model and found that in 9 of them the agent had not collected all the information it needed. The benchmark is vendor-built and the sample is small, so it supports a design instinct rather than a statistic.
On ambiguity between entities
The opening example sits in this category: which order, which customer, which invoice, which contract. This is practitioner judgement. No primary study tests identity checks specifically, so treat it as a design rule drawn from the general ambiguity research. It earns its place because the cost of a wrong match is usually the cost of the action that follows, and the agent often has two plausible candidates in front of it.
After repeated failure
OpenAI’s second trigger is exceeding failure thresholds. Its guide suggests setting limits on retries or actions, and escalating to a person when an agent cannot understand customer intent after multiple attempts. Anthropic’s Building effective agents makes a related point and notes that it is common to include stopping conditions such as a maximum number of iterations. It also notes that agents can pause for human feedback at checkpoints or when they hit blockers. The wording is permissive. It describes an option, and it does not impose a rule.
| Trigger | Typical example | Source of the placement rule |
|---|---|---|
| Irreversible or external action | Refund, payment, outgoing email, ERP write | OpenAI guide; Microsoft AI Red Team (tiered by reversibility) |
| Missing required field | No order number on a return request | CRMArena-Pro sample (9 of 20 failures, one model) |
| Entity ambiguity | Two open orders for one customer | Practitioner judgement, no primary study |
| Repeated failure | Third failed attempt to classify intent | OpenAI guide (failure thresholds) |
| Low confidence | Agent reports it is unsure | A design pattern only, no validated cut-off exists |
The last row deserves emphasis. Many guides say “ask when confidence is low”, but no published study gives a measured confidence threshold that works in production. A model’s self-reported confidence is not a calibrated probability, so build triggers on facts about the case (a field is empty, two records match, the action is irreversible) before you build them on the agent’s feelings.
What a Question Costs Compared with an Error
Every question has a price, mostly the time of the person who answers it. Every wrong action has a price too, often higher. A simple rule compares them.
Ask when the probability that the assumption is wrong, multiplied by the cost of the wrong action, exceeds the cost of asking. Rearranged, the agent should ask whenever that probability is above a break-even value, P*, where P* equals the cost of asking divided by the cost of the wrong action.
The figures below are illustrative, chosen to show the mechanism. They are not benchmarks, and you should replace them with your own.
- Cost of one question. An operator earning the equivalent of CHF 60 an hour spends two minutes on it, so the question costs about CHF 2 (illustrative).
- Hard-to-reverse refund of CHF 300. P* is 2 divided by 300, about 0.7%. If even one assumption in a hundred is wrong, asking is cheaper (illustrative).
- Mis-tagged support ticket costing CHF 5 to fix. P* is 2 divided by 5, which is 40%. Unless the agent is wrong four times in ten, it is cheaper to let it tag, log the result and let people correct the odd one (illustrative).
Scale it to a month to see the pattern. Take 500 refund requests where the agent has a 3% chance of picking the wrong order. Letting it act alone leads to about 15 wrong refunds, which is CHF 4,500 of cleanup. Asking on every request costs 500 times CHF 2, or CHF 1,000. Now take 2,000 tickets where a 10% mis-tag rate produces about 200 errors at CHF 5, so CHF 1,000. Asking on all of them costs CHF 4,000. The same agent justifies a question in one case and wastes money in the other (all figures illustrative).
The framing matches how the research community now thinks about it. A 2026 benchmark called RegretBench, described in an arXiv abstract, scores clarification policies on interaction cost, unhelpful questions and regret, though the abstract gives no formula. No source supplies a validated cost model, so the arithmetic above is a thinking tool and not a finding.
Two things push the real threshold up for frequent low-stakes tasks. One is the customer’s wait while the question sits in someone’s queue. The other is the effect the next section describes. Reversibility should set the threshold, and the agent’s own sense of certainty should not.
The Opposite Failure: Asking Too Much
Adding a question everywhere feels safe. The evidence says it is not.
Zheng and colleagues ran a controlled experiment with 180 participants across six everyday decision scenarios (arXiv 2611.01087). Clarifying questions raised perceived understanding and satisfaction. They also led participants to spend less time reviewing responses and to accept more responses unchanged, including erroneous ones. The setting was consumer decisions and not agent actions in a business, and the paper reports no change in general trust, so apply the result with care. Still, it shows that a question can make people feel involved while they check less.
The security side is sharper. The Microsoft AI Red Team post cited above states that bypass of human-in-the-loop controls was the most consistently exploited failure mode in its work, achieved through consent fatigue. It recommends monitoring approval frequency for signs of that fatigue. Anthropic reported in its post on Claude Code auto mode that, in practice, users accept 93% of manual permission prompts. That is a vendor statement with no published method, so read it as a signal and not as an industry fact.
The consequence for design is direct:
- Keep full approval for the small set of actions where an error is costly and hard to undo.
- Let agents act, with logging, on reversible and cheap steps.
- Track the approval rate. If a checkpoint is approved unchanged nearly every time, either the trigger is too broad or the reviewer is no longer reading, and both need attention.
- Show the approver what matters. A prompt that says “Approve action?” invites a click. A prompt that shows the order, the amount and the one detail the agent was unsure about invites a decision.
A coding-agent study by Edwards and Schuster (arXiv 2603.26233) shows how easily asking multiplies. Their multi-agent clarification setup asked about 3 questions per task on one model and nearly 9 on another, and it cost more per task in API fees than a single-agent version. It also improved the results. Asking has real value here, and a scaffold can still ask a great deal, so budget questions the way you budget any other cost.
A Design Checklist for Clarification Points
For each stop in a workflow, write down four things before building. A table keeps it honest, and it is the artifact an operations lead can review without reading code.
| Design question | What to specify | Example for a refund workflow |
|---|---|---|
| Trigger | The fact that causes the stop | Refund above CHF 100, or two open orders match the request |
| Who answers | A named role with authority and a backup | Team lead on shift, with the deputy as fallback |
| What the question contains | The case, the options, the agent’s proposed action and the one thing it is unsure about | ”Orders 4411 and 4418 both match. Proposed: refund 4418, CHF 300. Confirm or choose.” |
| Log | What is recorded for review | Question, answer, time to answer, final action, and whether the person changed the proposal |
Add three operating rules around the table.
- Set a time limit and a default. If nobody answers within the agreed time, the case should escalate or wait safely. It should never proceed on the agent’s guess.
- Review the log monthly. Count how often the answer differed from the agent’s proposal. A checkpoint that never changes anything is a candidate for removal. A skipped step that keeps producing errors is a candidate for a new checkpoint.
- Test the stops before launch. Run real historical cases through the workflow and check that the triggers fire where you expect. Our guide on building evals from real tickets describes how to assemble that case set.
Mapping the process first makes all of this easier, because the trigger list falls out of the exception column. The workflow mapping guide shows how to build that map in half a day.
What the Evidence Does Not Tell You
Be clear about the limits before you tune anything.
- No validated threshold exists. Nobody has published a field study that links question counts, error costs and confidence cut-offs in production. The break-even rule is a decision aid and the inputs are yours.
- The benchmarks are simulated. The user in each study is a language model or a scripted counterpart, and several tasks are generation exercises or built around a vendor’s own CRM. Real customers are messier.
- The models are older. Laban et al. and tau-bench used 2024 and 2025 models. Current models may ask more, ask less, or recover better from a wrong turn. Measure your own system.
- Human review is not a guarantee. The evidence on consent fatigue means a checkpoint protects you only if the person at it has the time and the information to decide.
Legal duties can add requirements. Article 14 of the EU AI Act sets human oversight duties, including the ability to override or stop a system, for high-risk AI systems. Whether a given workflow counts as high-risk is a legal question for your case, and it falls outside what this article can answer.
The practical conclusion is modest and testable. Pick the three or four actions in your workflow that you could not easily reverse, put a well-formed question in front of each, leave the cheap steps alone, and read the log after a month. The numbers from your own cases will tell you whether to move a stop, remove one, or add another, and they will be worth more than any benchmark quoted here.
Frequently asked questions
What does human in the loop mean for an AI agent?
It means the agent pauses at defined points and a person answers, approves or corrects before the workflow continues. The points are designed in advance, for example before a refund is paid, when a required field is missing, or after the agent has failed several times. Everything between those points runs without a person. The aim is to place few, well-chosen stops where an error would be expensive.
When should an AI agent ask a question instead of assuming?
Ask when a wrong assumption would be costly or hard to undo, and assume when a mistake is cheap to fix. OpenAI names two triggers in its practical guide for agents: exceeding failure thresholds and high-risk actions such as large refunds or payments. A useful extra check is a missing required field. No published study validates a numeric confidence threshold, so set limits from your own error costs.
How do I price a clarifying question against a wrong action?
Divide the cost of asking by the cost of the wrong action. The result is the probability of error above which asking pays. With illustrative figures of CHF 2 for a two-minute question and CHF 300 for a hard-to-reverse refund, the break-even is about 0.7%. For a CHF 5 mis-tag it is 40%. Replace the example figures with your own operator time and error costs.
Can too many approval prompts make an agent less safe?
Yes. Microsoft's AI Red Team reports human-in-the-loop bypass as the most consistently exploited failure mode in its agentic systems work, achieved partly through consent fatigue. A controlled experiment with 180 participants found that people who received clarifying questions reviewed responses less and accepted more erroneous ones. Tier approvals by consequence and track how often people approve without changes.
Do the benchmark results apply to my business workflow?
Only loosely. The studies cited here use simulated users, mostly generation or customer-service tasks, English text and models from 2024 and 2025. They show a pattern, which is that assuming early and acting without checks lowers reliability. They do not give a threshold you can copy. Treat them as a reason to test your own workflow with logged cases before choosing where the checkpoints go.