An assistant that asks permission for everything has found a very safe way to remain your problem.
You authorize it to organize a set of files. It asks whether it may open the first file, whether it may read the next, and whether the folder you named is really the folder you meant. By the time it has finished demonstrating its caution, you could have done the work yourself.
At the other extreme, a service takes your general wish to be organized and begins deleting things it considers unnecessary. It has saved you several approval messages and created a much more interesting afternoon.
The useful arrangement lies in deciding where your judgment is needed. Some steps can proceed within a clearly defined task. Others change the commitment, expose information or make a mistake hard to repair. Treating them all alike either wastes attention or leaves too much unexamined.
A good safeguard protects something specific. Its presence should make the work safer without making the user perform the whole task again.
Let ordinary work remain ordinary
Consider a folder of receipts you want sorted by month. You might authorize the assistant to read the files, identify dates and prepare a proposed arrangement. That could be enough for a first attempt. Once you have seen how it handles the material, you might permit it to move files within that folder while preserving the originals. You could continue to reserve deletion or sharing for a separate decision.
The scope can be narrow without being useless. The assistant has room to do the work, and the person has retained authority over consequences that were not part of the request. This is a more useful way to think about autonomy than treating it as one switch. A service may have considerable freedom inside a limited task and little freedom outside it. It can choose how to compare documents without being allowed to send them to someone else. It can research a purchase without being allowed to pay. The important question is where its discretion begins and ends.
A task also changes as it develops. The assistant may discover that some receipts belong to a different year or that a file appears to be a contract rather than a receipt. These findings need not stop all work. It can set the unclear items aside and continue with the parts it understands. A focused question about the exception is more useful than a stream of questions about the obvious. The provider has to make this pattern possible. The system should be able to preserve uncertainty without treating it as an instruction to improvise. It should also be able to stop one part of the work without abandoning everything else.
A person who asks for help often has limited attention available. The assistant should spend that attention carefully. Interrupting for an immaterial choice can be as unhelpful as failing to interrupt for a material one, though the consequences differ. We need to judge an approval request by the decision it gives back to the person.
Suppose the assistant is arranging a routine delivery. The seller you approved is out of stock. Another seller has the same item at the same total, with the same delivery promise. Whether the assistant may switch depends on the authority you gave and the actual differences between the offers. A new seller could change the return process, expose information to a new party or alter the reliability of the transaction.
The fact that the price stayed the same does not settle the scope. A service should be able to say what changed. Then, if the existing permission does not cover it, it should ask about that change. The user does not need to repeat the whole task merely because one part requires a new decision.
Put the limit where it can hold
There is a difference between asking software to respect a boundary and restricting what it can do beyond that boundary. Both can matter. The instruction helps the assistant understand the task. The enforced limit reduces what can happen if it misunderstands, encounters hostile material or simply makes a mistake.
The old political problem of delegated power offers a useful connection. Federalist No. 51 argued that dependence on the people needed “auxiliary precautions”: arrangements that would help keep power within bounds.[1] The modern setting is different, but the design question is recognizable. How do you give a system enough authority to do useful work while placing some restraints beyond its own discretion?
In software, that can mean keeping a permission check outside the part of the system the assistant can alter. OpenAI’s September 2026 description of dots says its action-review controls sit outside the working environments dots can change. The company describes checks against user instructions and safety requirements before certain actions proceed.[2]
Such separation serves a practical purpose. If the assistant has misunderstood the instruction, the same misunderstanding should not automatically give it the power to remove the check. The check still needs to be reliable, and it can make mistakes of its own. A separately enforced review is not proof that every decision it makes is correct. The constitutional analogy also has a limit. Several components controlled by one company are not independent branches of government. Technical separation can constrain the assistant while leaving substantial authority with the provider that sets the rules. Both facts belong in the account.
Payment controls offer a more concrete example. Meta’s public account of Muse describes single-use payment credentials limited to a merchant, amount and period of validity.[3] A credential constrained in this way has a narrower range of uses than a general payment method. The boundary can reduce the consequences of a mistake without deciding whether the purchase itself was wise.
That last distinction is worth retaining. A payment can be authorized, correctly limited and still fail to serve the user’s purpose. Technical controls protect selected conditions. They do not absolve the provider from the work of understanding the task or presenting the choice honestly.
An effective spending limit also has to match the way the person thinks about the budget. A per-purchase limit can leave a series of purchases that exceeds the intended total. A daily limit can be wrong for a project that spans a week. A rule for the base price can fail if fees are added later. The assistant should preserve the unit of the permission: this purchase, this trip, this month, this total. If the available payment control does not express the needed boundary, the service must account for the gap elsewhere rather than pretend the two limits are the same.
Time matters too. Permission for a task this week should not silently become permission to repeat it next year. A recurring task may be useful, but it creates a different relationship from a one-time errand. The user should be able to recognize that relationship and end it. A button that says allow should not conceal the word forever.
The page is allowed to inform the assistant
An assistant working on the web encounters material written by people other than the user. Some of it is useful. Some is wrong. Some may be written specifically to redirect the assistant.
Researchers call one form of this problem prompt injection: outside content tries to become an instruction that changes what the agent does. The material might appear in a webpage, an email or a document that the assistant reads while carrying out a legitimate task.
Imagine asking the assistant to compare a product. A page that should supply information instead contains a demand to send a private file elsewhere. The page has no authority to make that demand. The user asked for a comparison, and reading the page did not grant its author the right to enlarge the task.
The distinction sounds obvious when stated in ordinary language. It remains a serious design challenge when a system uses language both to receive instructions and to interpret the material it encounters. The assistant must preserve the difference between information relevant to the task and authority to direct the task.
AgentDojo, a research environment introduced in 2024, was built to test agents using tools over untrusted data. Its published version includes ordinary tasks alongside security cases, allowing researchers to examine both task performance and resistance to attacks.[4] Those are separate questions. A system that refuses everything may avoid some harms while being of little use; a system that completes tasks readily may still fail under adversarial conditions.
The consumer does not need a catalogue of attack techniques to understand the consequence. Access, communication and action create routes through which a mistaken interpretation can matter. A service should constrain those routes rather than rely entirely on the assistant recognizing every suspicious sentence.
Suppose the task requires reading a document and producing a private summary. It does not require sending that document to an arbitrary outside address. A boundary around transmission can reduce the harm even if some malicious text is misread. The design should make unnecessary actions unavailable or subject to a meaningful check.
This is one reason broad permission can be costly. The more unrelated actions a service is allowed to perform, the more ways there may be for an error to escape the original task. Keeping access and authority close to what the work needs can reduce that exposure.
At the same time, “least access” cannot mean no access. The assistant needs enough information and capability to help. The practical aim is to remove powers that do not contribute to the authorized work, not to build a perfectly safe system that cannot do anything.
Security must be evaluated alongside usefulness. NIST’s generative-AI risk profile emphasizes testing in relevant conditions and warns against broad conclusions drawn from narrow or anecdotal assessments.[5] That discipline should apply equally to confident claims of safety and dramatic claims of failure.
A demonstration can reveal a problem. It cannot, on its own, establish how often the problem occurs across ordinary use. A provider’s successful test can show that a protection worked in that test. It cannot guarantee that no different case will defeat it.
The right response is continued evaluation, clear boundaries and a way to limit damage when an error gets through. A personal assistant should not require the user to win every encounter with hostile content.
Make the stop understandable
A safeguard can fail by allowing the wrong action. It can also prevent a reasonable task from being completed. The second failure should not be ignored simply because the first can be more dangerous. An assistant that repeatedly blocks ordinary work may lead users to abandon it, grant broader permissions than they wanted, or search for ways around controls they no longer trust.
Good design should give a person a useful explanation of a stop without disclosing details that would undermine the safeguard. Was the recipient unclear? Did the action exceed the amount approved? Is the step outside the service’s supported scope? The user should be able to distinguish a missing fact from a firm limit.
If the issue is a missing fact, the assistant can ask for it. If a narrower action would solve the problem, it can propose that action. If the task requires a step the person must perform themselves, the handoff should preserve the work already done and explain what remains.
The user should not have to start from zero because the final step belongs to them. There is a temptation to treat a block as an obstacle the assistant should overcome at all costs. That reverses the purpose of the safeguard. A capable service should look for permitted ways to fulfill the task, not treat persistence as authority to ignore the boundary.
The same restraint applies when the user is unavailable. Silence can mean that the person is asleep, busy or no longer interested. It should not be interpreted as fresh permission for a consequential action the service had stopped to ask about.
If a task can continue safely within its existing scope, it can do so. Otherwise, the unresolved decision should remain unresolved. The assistant may lose an opportunity. It should not solve that inconvenience by taking a power it was never given.
This is not a reason to make every standing permission expire after a few minutes. People can deliberately authorize repeated, bounded actions. The important point is that the bounds should come from an actual instruction or supported setting, not from the service’s impatience with waiting.
A clear ongoing permission can save many interruptions. It can identify the task, the conditions, the period and the point at which the assistant must return. A delivery reorder might be limited by item, frequency and total cost. A research task might allow continued reading while reserving external messages for review.
The service needs to maintain those limits as circumstances change. A new recipient, a new kind of commitment or a meaningful increase in cost may require a new decision even when the broad goal remains the same.
Give authority an ending
A useful delegation includes a way to finish. The task may end when a document is delivered, an order is confirmed or a deadline passes. A recurring arrangement may continue until the user changes it. In each case, the service should make the state legible. The person should know whether something is complete, still active or waiting for them.
This sounds like administration. It is also a form of control. An assistant that keeps acting after the purpose has ended can become intrusive even when each individual action seems helpful. A price watcher may continue sending alerts after the item has been purchased. A reminder may survive the event it concerned. A standing instruction may fit a job or household situation that has changed.
The problem is not solved by making the assistant more energetic. It needs to know when the work has reached its stopping point.
The user should also be able to withdraw authority without negotiating with a personality. A service may need to explain the effects of stopping a task or disconnecting a source. It should not make an ordinary decision to stop feel like a betrayal of a relationship. That matters when a product speaks warmly and remembers personal details. The social ease of the interface should help the person use the service, not make it emotionally difficult to set a boundary.
Stopping future work and reversing past work remain different. Canceling a monitoring task should stop new checks. It does not cancel an order already placed. Revoking permission to send messages does not retrieve messages already delivered. A good interface should make these differences clear at the moment the user acts.
There can also be work already underway. A person who says stop needs an accurate account of what has happened, what can still be halted and what requires a separate remedy. The assistant should not offer a false sense of reversal merely because it has stopped producing new actions.
We will examine the records and remedies in the next chapter. For now, the design principle is simple enough: authority should have a visible scope and a visible end.
The best controls make ordinary help easier to trust. They let the assistant carry the routine parts while bringing consequential changes back to the person. They preserve room for action without requiring faith that every future interpretation will be correct.
You should be able to leave a task with an assistant and turn your attention elsewhere. That is what the service is for. The brakes should still be there when you return, and they should work without asking the engine’s permission.
Sources
[1] The Federalist No. 51 (1788), original English text via the American Presidency Project. The connection concerns authorization and supporting safeguards. It does not equate software components under one provider with constitutionally independent institutions.
[2] OpenAI, “How we build safety, security, and privacy into dots,” September 29, 2026, “A separate check before dots act.” Provider-described design; separation of enforcement is not a claim of flawless review decisions.
[3] Meta, “How We Built Safety Into Muse,” September 8, 2026, “Purchases.” Used for the described merchant-, amount- and time-bound single-use credentials.
[4] Edoardo Debenedetti and colleagues, “AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents,” arXiv:2406.13352v3, November 24, 2024. The benchmark illustrates evaluation design; its historical results are not used to rank September 2026 products or estimate a current real-world attack rate.
[5] NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, AI 600-1, July 26, 2024, especially MEASURE 2.3 and 2.5. A voluntary risk-management resource, not product certification.