AI Agents Act Before You Ask. Who Owns the Outcome?
As autonomous AI agents move from chat windows into inboxes, records, and payments, the confirm dialog is proving to be a weak safeguard. Designers and researchers argue that reach, reversibility, and accountability — not click counts — should determine when software acts alone.
AI agents are no longer waiting to be prompted. They triage inboxes, update records, and send messages while their users sit in other meetings. That proactivity is the entire point of the technology, and it is also where the trouble begins. When an agent acts before anyone asks, and reaches past what anyone authorized, the standard remedy — a confirm dialog with a human click — does not restore accountability. It moves it.
That is the central argument emerging from design and human-factors research on autonomous systems. The click lands on whoever had the least information and the least power over the outcome. The team that decided a particular step needed no stronger gate stays out of the story entirely. The reviewer who approved step 41 of a 60-step run becomes the one holding the consequences.
The problem is not new. Eric Horvitz modeled when software should act alone in a 1999 paper on mixed-initiative interfaces. Sixteen years earlier, Lisanne Bainbridge warned in «Ironies of Automation» that automating most of a job leaves people holding its hardest part: the watching. What has changed is volume. Gartner predicts that at least 15 percent of day-to-day work decisions will be made autonomously through agents by 2028.
Against that backdrop, the debate over whether an agent should act before the user asks is largely a decoy. Initiative is not what harms people; reach is. The more useful test is what an agent can touch, and whether a person can put it back. That question also settles liability. Whoever sets an agent's reach owns what happens inside it, and no confirm click should shift that responsibility to whoever happened to click.
Confirm buttons record that a person was present. They say nothing about whether that person read what they approved. Kathleen Mosier and Linda Skitka named this automation bias in the 1990s, and it takes two forms. Errors of commission follow the machine's wrong suggestion. Errors of omission miss what the machine never flagged. In Skitka's 1998 NASA report, people using a decision aid were 65 percent accurate when the aid was wrong, compared with 97 percent for people working without one. Omission is the quieter failure, because nothing on the screen ever asks to be doubted.
Interruption, by contrast, is a judgment rather than a ritual. Anthropic's study of its own coding agent found that newer users ran full auto-approve in roughly 20 percent of sessions, rising past 40 percent for experienced users. Those experienced users also interrupted more, in about 9 percent of turns against 5 percent for newcomers. The company's own conclusion is that forcing a human to approve every step adds friction without necessarily adding safety. The per-click version resembles security theater: visible, reassuring, and blind to how far the agent can reach afterward.
The stakes of getting this wrong are not hypothetical. Madeleine Clare Elish described the pattern in 2019 as a «moral crumple zone»: responsibility for a failure gets pinned on the human operator closest to the event, even when that person had little control over what the automated system did. A car's crumple zone protects the driver. The moral version protects the system, and the operator absorbs the impact.
The most telling example remains the March 2018 crash in Tempe, Arizona, where an autonomous Uber test vehicle struck and killed Elaine Herzberg. A safety driver, Rafaela Vasquez, sat behind the wheel — in the loop by every definition the program used. She was watching television on her phone, which illustrates the point rather than excusing it. The federal safety board found she had looked away for more than a third of the trip and that an inadequate safety culture at Uber contributed. Uber faced no criminal charges. Vasquez pleaded guilty to endangerment and received three years of supervised probation. Being in the loop, it turns out, mostly meant being nearest the crash.
Horvitz's 1999 rule still frames the design question: act only when the expected value beats doing nothing. His math leaves one variable implicit — whether the action can be undone. Drafting a reply and sending it carry the same chance of being right and very different costs of being wrong. Proactivity should scale with reversibility, not with confidence. An agent that prepares, drafts, flags, and stages work extends the user's reach inside a space they can still edit and undo. One that sends, deletes, pays, or commits is spending the user's authority.
The accuracy an action requires should shape the interaction as well. A probabilistic output is a prediction about what is likely right; a deterministic one is a certainty. A summary can be a prediction and still be useful, because a reader applies judgment. A payment cannot. People still belong in the loop, but at the point where judgment holds up: watching a run they can stop, rather than pre-signing a list they cannot read.
6
