The agent can be busy while the person makes little progress. It can produce ten comparisons that need repair, send repeated questions or finish a task whose purpose has changed. It is easy to count actions. It takes more care to judge whether the work helped.
A recurring review asks whether delegation is improving the whole job. A week is a convenient interval for many ordinary routines. It is not an experimentally established optimum. A high-consequence task may need review before every action. A stable, low-consequence task may need less frequent inspection. Changes in the tool, permissions or task can justify an earlier check.
The review should be small enough to survive a busy week. Begin with the tasks that ran, the outcomes accepted, the human work they required and any trouble. If the record is becoming another unpaid job, simplify it without removing the facts that affect decisions.
Count the work that moved to you
Speed claims often describe one part of a process. Writing a first draft faster can be valuable. It can also move effort into checking and revision. Both belong in the account.
For a recurring task, record active human time spent defining the job, reviewing its work, correcting it and handling failures. Track elapsed time separately. An agent may run while the person does something else; ten minutes of waiting and ten minutes of close inspection impose different demands. Work done in parallel may finish sooner. It can also make the person’s own time harder to count.
Include the outcome standard. A quick result that misses the required quality is not the same service at a lower cost. If the person now checks more thoroughly than before, the comparison may include a quality gain as well as an effort change. Say so instead of forcing everything into a single percentage.
The research on coding assistance shows why context matters. In a 2024 randomized study at Google, 96 engineers were assigned a standardized task with or without certain AI tools. Among those who completed it, the unadjusted average times were 96 and 114 minutes. After accounting for differences among engineers, the best estimate was about 21% less time. But the uncertainty was wide enough that the study could not rule out no benefit at the usual confidence level. One task with internal tools does not establish a general gain for autonomous agents. It does provide evidence that assistance can save time in a particular setting. Enterprise trial
METR studied a different setting in early 2025: 16 experienced developers completing 246 tasks in open-source projects they knew well. With AI allowed, tasks took an estimated 19% longer. The study’s 95% confidence interval ran from 2% to 39% more time. Developers nevertheless believed they had been faster. The trial does not isolate review overhead as the cause of the slowdown, and its historical tools and specialized sample matter. METR’s 2025 study
That finding should not be frozen in time. METR’s February 2026 update reports later work involving 57 developers and more than 800 tasks. The later estimates suggested faster work with AI, but were also consistent with no improvement. The authors say this is an unreliable signal. Who joined, which tasks they submitted and how time was measured had changed. Some people also ran agents in parallel. The update does not establish a precise new speedup. It shows why a comparison must be renewed as the work changes. 2026 update
These studies need not cancel each other out. They ask related questions in different conditions. A familiar, complex repository is not the same as a standardized task with unfamiliar code. Tools, workers and measures differ. The sensible response is to measure the job you are delegating rather than borrow the most flattering or alarming headline.
Keep the denominator
Suppose an agent attempted twelve workshop-related tasks in a month. Nine were accepted, two needed a person to finish them and one was abandoned. Reporting “nine tasks completed” is accurate but incomplete. The three other tasks may contain most of the cost.
Record attempted tasks as well as accepted outcomes. Keep blocked and unresolved cases visible. A system that avoids every hard case may look reliable among the cases it finishes. That could still be useful if the boundary is deliberate, but it is not the same as handling the full job.
For each recurring task type, a short record can include:
- What counted as an acceptable result, and how it was checked
- How many tasks were attempted, accepted, corrected or left unresolved
- Active human time and elapsed completion time
- Direct charges, including unsuccessful attempts where they were billed
- Consequential errors, near misses and the outcome of recovery
- Any change in tools, permissions, instructions or the task itself
This is a proposed review record, not a validated scorecard. Use it to support a decision. Do not collect a field merely because a dashboard can display it.
The historical airline archive illustrates why the denominator matters. The primary calculation retained all 200 saved records, including five with no detailed reward diagnostic. It counted success within each of the 50 tasks. An all-four success means four stored successes for that task, not one successful attempt found after several tries.
The analysis also reproduced the archive’s two- and three-attempt figures, by counting all possible pairs and groups of three within each task. The method note explains that calculation. These remain summaries of saved scores. They do not predict how a current agent will behave. Public methods and results
There is a second question: what did the task itself require? Grouping tasks by their reference solutions showed that many steady passes came from tasks with no required write or answer text. That is a reason to inspect the test, not a reason to declare every such task invalid. A correct refusal or handoff can be a valuable outcome when the task requires it. The stored score alone does not tell us whether that is what happened.
In ordinary work, define useful completion and useful stopping before counting them. An agent that correctly refuses to spend beyond its cap has respected a boundary. An agent that never does any work is not therefore a good assistant.
Use the review to change a decision
The result of a review can be to keep the task as it is. It can also be to narrow it, improve its evidence, add a checkpoint or stop delegating it. Giving the agent more power is only one way to make progress.
If the venue comparison is accurate but its prices take too long to verify, improve the source record before permitting purchases. If a workflow produces repeated unresolved sends, investigate its message-state handling before increasing the number of recipients. If a permission is no longer needed, remove it through the appropriate account controls.
Do not let a good average erase a serious exception. A low overall error rate may conceal a rare action with large consequences. Review those cases individually. At the same time, do not let one vivid but easily repaired mistake prove that every task is unsuitable. Ask how serious the error was, how often it happens and what it takes to fix.
A before-and-after comparison can help a household or small team decide what to do. It does not, by itself, show that the agent caused the change. The person may choose easier tasks for the agent, gain experience over time or change the quality standard. Record those changes. For a stronger study, compare similar tasks under a planned design and keep the cases that go badly. No one needs to pretend a private work log is a peer-reviewed trial for it to be useful.
Review access as well as results. A task that no longer runs may still have permission to read files or send messages. A helpful new feature may introduce another tool or data destination. The question is whether the current authority still matches the current job.
And review the burden on people who receive the agent’s work. Saving your time by sending vague requests to ten venues can move the cost to ten staff members. A system should not be judged only by the effort it removes from its owner’s screen. A clear, necessary inquiry is a better unit of progress than a large count of outgoing messages.
A smaller job can be a better bargain
The ambition of agent software is often described as doing more without us. I would judge it by what people gain: time they can use, less tedious work, a clearer route through a task and fewer avoidable failures.
That can mean full automation of a narrow routine. It can mean research followed by a human decision. It can mean an agent preparing a change while a person makes the final commitment. A system that reliably does the useful half of a job may be worth more than one that claims the whole job and leaves a confused person to finish it.
There is room for optimism here. Clear permissions, better evidence and honest stopping conditions can make delegation easier to trust. The research does not promise one perfect control scheme. It gives us reasons to test particular arrangements, compare them fairly and improve them when they fail.
The work of managing should get lighter as the task becomes clearer. If it only grows, something needs to change. The person should not be trapped in a permanent second shift of checking, repairing and explaining the machine.
Begin with a job you can describe. Give it authority you can bound. Ask for evidence you can inspect. Keep a way to stop and recover. Then decide, from the whole result, whether this use of an agent has made life better.