Skip to reading
Forecaster
Complete research draft8 chapters published
Contents Chapter 3

Who AI Really Serves When It Starts Running Your Life

The Choices You Never See

Before you approve a recommendation, other choices have already shaped what reaches you. Follow the missing options.

By Kevin L. Michel15 minute read

A list can be accurate from top to bottom and still leave you badly served.

The hotels exist. The prices are correct. Each room is available on the dates you requested. You choose one, and nothing about the purchase goes wrong. A week later, someone asks why you stayed so far from the place you were visiting. There was a suitable hotel around the corner.

It never appeared in your list.

You could not reject the recommendation on that ground. You did not know there was a comparison to make. If the assistant kept a record of the booking, the record may show that you approved the hotel. It may not show why the other hotel was absent.

This is a quiet form of influence. It can arise from limited access, poor information, an unsuitable search or a mistaken account of what you wanted. Deliberate favoritism is another possibility, but it is far from the only one. The first job is to establish where the option disappeared.

We tend to notice decisions at the point where we are asked to make them. The choice of what to put in front of us happens earlier.

How much of the world is in the list?

No assistant can show every option for every request. An unfiltered catalogue would return much of the work to the customer. The value of a recommendation is partly that someone, or something, has reduced the field.

The reduction can be helpful. If you need a room within walking distance of a venue, hundreds of distant hotels can disappear without a loss. If you have a firm budget, removing rooms you cannot afford saves attention. A good assistant uses your purpose to make the search smaller in the right places.

Trouble begins when the smaller field is mistaken for the whole one. Consider the difference between “These are three suitable options I found” and “These are the best options available.” The second statement carries a much larger burden. It implies enough knowledge of the available field to rank the selected options above the alternatives. A service may have grounds for that claim in a narrow catalogue. Across an open and changing market, the claim is harder to support.

We do not need to demand a search of the entire earth. We need a useful account of the search that was actually possible. The assistant might have consulted a booking service that lists only participating properties. It might have been able to read one site’s public pages but unable to check its live inventory. It might have encountered a listing with incomplete prices. Each condition changes what “available” means. An option can exist in the world and remain unavailable to this search, at this time, through this route.

That distinction becomes important when a person treats a polished answer as a finished investigation. A confident summary can make the boundary disappear. The reader sees three hotels rather than the systems that made those three hotels visible.

OpenAI’s shopping help page offers a useful example of a provider acknowledging limits. It says product selection draws on context and product information, that not every product will appear, and that prices can lag changes at the seller. It also cautions that generated labels are not guarantees and that reviews are not verified by OpenAI.[1]

Those limits are ordinary features of a difficult information problem. They should shape the language of the result. A label such as best price needs to mean something the service can support. Is it the lowest price among the sellers checked? The lowest price for the exact model, including delivery? The lowest one seen recently? These are different claims, and a customer may act differently depending on which is true.

The practical remedy is often modest. Tell the person the important boundary. If the search covered one marketplace, say so. If the price needs confirmation, keep that uncertainty attached to it. If a required feature could not be verified, do not silently promote a guess into a match.

The customer should not need to interrogate an answer until its limits fall out. There is also a danger in overloading the result with qualifications. A paragraph of generic warnings can make every uncertainty look equally important. The assistant’s job should include deciding which missing fact could change this person’s choice. In the hotel example, distance may matter more than the exact wording of a room category. In another trip, the ability to cancel could matter most. Information becomes useful when it is connected to a decision.

The standard hidden inside “best”

Even with perfect access, the assistant would still need a basis for choosing. A hotel can be excellent for someone who wants nightlife and poor for someone who needs sleep. A device can offer more features while being harder to repair. A low upfront price can come with costs that arrive later. The word best contains a set of priorities, whether anyone has named them or not.

An assistant has to work with the priorities it is given and the ones it infers. There will always be some inference. You cannot list every preference before asking a question, and a service that refuses to proceed without doing so will not be much help. But an inference that materially affects the result should remain open to correction.

Suppose you ask for a quiet hotel. The assistant finds one far from the city center. You meant a quiet room close to the place you need to be. It has treated distance from activity as a way to satisfy your request, while overlooking the reason for the trip. The hotel may be quiet. The recommendation may still be poor. Nothing about that failure requires a commercial incentive. It requires an unsuitable interpretation of a word.

Now suppose the assistant remembers that you once liked a remote hotel. That memory may help, or it may carry an old preference into a new situation. A successful previous recommendation can become a source of future error if the context changes and the assistant does not notice. Personalization should help a person express what matters now. It should not trap them inside a record of what mattered before.

This raises a subtle issue about asking the assistant to know you. The more it infers, the less you may have to explain. Yet some of the saved effort comes from decisions being made before you see them. The service may decide that you value speed more than price, that you prefer familiar brands, or that an old budget remains suitable. These decisions can be sensible and still deserve an easy way to revise them. A useful recommendation might expose one important assumption: “I treated walking distance as more important than room size.” That sentence invites a correction without demanding a complete account of the user’s preferences.

Compare it with “I selected the best hotel for you.” The second sounds more personal while giving you less to work with.

There are preferences a service should be careful about inventing. A person who has asked for inexpensive options has not necessarily asked to see only the cheapest. A person who has bought a premium product has not granted permission to treat every future budget as flexible. The assistant needs enough restraint to leave room for the user’s changing judgment.

The option to delay also belongs in the field. So does the option to keep what you have. A search designed only to compare sellers begins after a prior decision to buy. If that decision is still open, the assistant should not close it merely because its shopping tool is ready.

This is where an apparently small change in the task can change the whole arrangement. “Which laptop should I buy?” invites comparison among laptops. “How can I make this work easier within my budget?” may lead to a repair, a setting change, a borrowed device or a purchase. The broader question permits more kinds of help.

A person should not need to know the perfect wording to receive that help. The service can ask whether the purchase itself is settled.

When a label changes the choice

Commercial labels add another layer to selection. If an item is sponsored, a person may want to know. What happens when the assistant reads the label before the person ever sees the item?

Davood Wadi and Yu Ma tested a version of this question in a September 2026 preprint. They assigned a model to act for either a traveler or a booking platform, then varied a sponsorship label in constructed hotel choices. In 2,000 calls to Gemini 3.1 Pro, the target hotel’s selection share fell from 53.0% to 2.8% when sponsored under the traveler role. Under the platform role, it fell from 54.2% to 25.0%. Both roles reduced selection of the sponsored option; the reduction was smaller under the platform role.[2]

These were model calls in an experiment, not purchases by human travelers. The study tests assigned-role sensitivity. It does not show what instructions commercial assistants receive, establish that a provider secretly favors sponsors, or measure a consumer’s loss.

The result raises a question about where disclosure does its work. A label can reach the assistant and affect selection without reaching the person who will rely on the selection. The final recommendation may look ordinary even though the earlier process changed.

What should we want the assistant to do with such a label? Automatic rejection would be a crude rule. A sponsored product can be good. Paying to advertise does not make a hotel noisy or its beds uncomfortable. The label tells us something about a commercial relationship; it does not settle the quality of the offer. The assistant still needs to evaluate the evidence relevant to the customer’s purpose.

Automatic acceptance would be equally crude. The fact that someone paid for visibility is not evidence that the option suits the user. A good process has to keep those ideas apart.

This is why a fair comparison cannot be reduced to counting how often sponsored items are selected. We would need to know the available alternatives, their relevant qualities, the user’s priorities and the information the system had. A change in selection might be appropriate in one setting and concerning in another.

A single recommendation rarely supplies all of that evidence. The user sees an outcome. The provider can examine patterns across many outcomes and test how changes in instructions, labels or data affect them. Independent researchers can design comparisons that hold selected conditions steady. Each has a different view of the process.

The service should not make the individual shopper responsible for discovering a pattern that only a large set of observations could reveal. At the same time, an outside critic should not infer the pattern from an anecdote that happens to fit a suspicion. If an assistant recommends an expensive product, we should ask whether the user’s stated needs justified it. If it recommends a partner’s product, we should investigate the relationship and the comparison. The outcome is a reason to inquire, not permission to invent the missing evidence.

A useful disclosure survives the journey from search to recommendation. If paid placement was material to what the person is being shown, the person should not have to reconstruct that fact from somewhere else. Whether current systems reliably achieve this is an empirical question, one that deserves better testing than a collection of screenshots chosen to win an argument.

The website has a say

So far, we have imagined the assistant looking across an available field. Sometimes it reaches a door and cannot enter.

This changes the meaning of personal assistance. You may be able to use a website yourself while the service acting for you encounters a restriction. Your account is one part of the relationship. The website’s rules and technical controls are another.

The dispute between Amazon and Perplexity makes the issue concrete. In a November 2025 statement, Amazon said third-party purchasing agents should operate openly and respect businesses’ decisions about participation. Perplexity’s response argued that users should be able to choose an assistant to act through their own accounts. Each company presented its position as protecting the customer.[3]

Their disagreement cannot be resolved by repeating the word user more loudly.

A website may have legitimate reasons to control automated activity. It has to protect accounts, operate a service and consider the people affected by transactions. A user may also have good reasons to choose a helper outside the website’s own system. That helper might make comparisons easier or reduce work the user cannot readily perform alone.

The question is how those interests should be accommodated, and who gets to decide when they collide. In August 2026, the Ninth Circuit vacated a preliminary injunction against Perplexity and sent the case back for further proceedings. On the record before it, the court found Amazon unlikely to establish that Perplexity itself had “accessed” Amazon’s computers under the relevant federal and California statutes. The opinion limited its holding to the issue and technology presented, and expressly preserved Amazon’s ability to regulate user access through private terms.[4]

It would be wrong to turn this into a universal right to send any agent anywhere. It would also be wrong to describe the earlier injunction as if the appeal had never happened. The legal account must follow the actual decision.

For our purposes, the case reveals a practical dependency. Choosing an assistant does not guarantee that every service you use will accept its way of working. A person can authorize an errand while lacking the power to open all the routes the errand requires.

That dependency deserves attention before consumers build their lives around a particular form of assistance. If an assistant can research a trip but cannot complete the booking, the user needs to know where the handoff occurs. If a website changes its access rules, an apparently reliable workflow may need a new route. A service should not promise seamless action while concealing an important boundary until the deadline arrives. Nor should it treat a blocked route as permission to evade the restriction. A capable assistant needs lawful and authorized ways to proceed, including telling the user when the remaining step belongs to them.

The question extends beyond one lawsuit. In a June 2026 theoretical paper, Yukun Zhang and Kemu Xu call attention to delegation rights: the ability to authorize a bounded, revocable proxy to act through an account. Their model considers the costs of platform control as well as the risks of giving users broad control, and explores conditional access based on safeguards. These are proposed arrangements under stated assumptions, not measured market outcomes or an existing universal entitlement.[5]

The useful distinction is between having a service available to you and being able to use it through the helper you choose. A person who can perform a task manually may gain little from that right if doing so is exhausting, inaccessible or prohibitively time-consuming. Assistance can make an existing option practical. Restrictions can therefore affect more than convenience.

Yet opening every route to every agent would create its own costs. A consumer’s authority over an account is not unlimited authority over other people’s information or the service’s infrastructure. A workable arrangement has to preserve legitimate boundaries while preventing those boundaries from becoming an unexamined answer to every request for access. That is an institutional problem. Better prompts alone cannot solve it.

A smaller list with an honest boundary

What, then, should the person asking for help receive? A complete history of the search would often be too much. A list without an account of its limits can be too little. The useful middle depends on the decision.

For a routine purchase, the assistant might need to identify the exact item, give the total cost, and flag a material uncertainty about delivery. For an expensive purchase, it may need to show a credible alternative and explain the tradeoff that led to its preference. For a trip, it should preserve the constraints that made the trip worth taking in the first place.

The explanation should be open to a test. “This is the best option for you” gives little direction. “This costs more, but it is the only option I could verify within walking distance” tells you where to look if you disagree. You can change the distance requirement, ask it to search another source, or decide that the premium is too high.

The provider should also distinguish between a reason supported by the record and a story generated after the choice. A smooth explanation is not, by itself, evidence of the process that produced the recommendation. Useful records can show which sources were consulted, which constraints were applied and which material facts were unresolved. They do not need to pretend to reveal every internal operation of the model.

Some decisions warrant an outside comparison. Most errands do not warrant an independent investigation. The reader needs to judge the value of an extra check against the cost of a mistake. A second source can be worthwhile for an expensive, hard-to-reverse choice even when it would be wasteful for a routine refill. The larger responsibility remains with providers. They choose how to describe the reach of their services. They have the ability to test patterns, improve defaults and make commercial relationships visible. A person who accepts a recommendation should not thereby inherit every problem that the provider was better placed to prevent.

This is the standard I would put to a personal assistant: reduce my work without quietly reducing my ability to question the result.

It is a demanding standard because the reduction in work often comes from narrowing what the person sees. The service has to make that narrowing useful, preserve the facts that could change the decision, and leave room to reopen the search.

Return to the hotel around the corner. The assistant might have missed it for a perfectly ordinary reason. Perhaps it was outside the service it could check. Perhaps its listing was incomplete. If the boundary had been clear, you could have decided whether to search further.

You might still have booked the first hotel. You might have paid a little more to stop looking. A person is allowed to value their time.

What matters is that the choice about ending the search should belong to the person whose evening, money and journey will follow from it.

Sources

[1] OpenAI, “Shopping with ChatGPT Search,” accessed September 30, 2026. Sections on selection, labels, reviews and pricing. The document describes limitations rather than an independently measured accuracy rate.

[2] Davood Wadi and Yu Ma, “Whom Do AI Agents Work For? Role Assignment Induces Sponsorship Bias in LLM Recommenders,” arXiv:2609.17989v1, September 16, 2026. Preprint; Study 1 and Table 3. There were 500 calls per condition. Its single-turn constructed choices do not establish deployed-product behavior or consumer losses.

[3] Amazon, “Statement about Perplexity,” November 2025, and Perplexity, “Bullying is Not Innovation,” November 2025. These are the parties’ public positions, not independent findings about the other’s motives.

[4] Amazon.com Services, LLC v. Perplexity AI, Inc., Ninth Circuit No. 26-1444, August 4, 2026. See the opinion’s scope limitation on page 17 and footnote 5 on page 21. The discussion concerns this dated appellate ruling on preliminary relief, not a final resolution of all claims or a general license to bypass site restrictions.

[5] Yukun Zhang and Kemu Xu, “Delegation Rights: Property, Agency, and Investment Incentives in the Age of AI Agents,” arXiv:2606.31935v1, June 30, 2026. Theoretical preprint. Its numerical examples are illustrative simulations and its proposed safeguards require separate practical evaluation.