The useful question is rarely whether two screens look different. It is what kind of difference we have found and which explanations remain possible.
This chapter separates two tasks. A shopper needs enough information to make a reasonable decision without wasting the afternoon. A researcher needs a design strong enough to support a claim about how a system behaves. We should not demand a research project from every shopper. We should not accept a shopper’s anecdote as a completed research project either.
What earlier audits actually found
Aniko Hannák and colleagues examined commerce and travel sites in 2014 using participant searches and controlled accounts. They used duplicate controls to help distinguish ordinary variation from personalization. Their nine-site finding combined steering and price differences, including membership deals and experimental ranking changes. It was not a finding that nine sites knew every buyer’s maximum price. The study documents measurable personalization in a historical sample; it does not establish its current prevalence or a universal loss to consumers. Original paper
The European Commission’s 2018 study, using mystery shopping conducted in 2017, examined 160 websites across eight countries and four sectors. Its prominent 61 percent figure concerned personalized rankings or offers under tested scenarios. More than 94 percent of matched products had identical prices. The authors did not find consistent, systematic personalized pricing across the examined markets, while recognizing limits to detection. A study can therefore find substantial personalization of the shopping experience without finding widespread same-product price changes. Those are different outcomes. Commission study
These results should restrain both enthusiasm and alarm. They are reasons to be precise, not reasons to stop investigating. A historical audit can show what happened under its tests. It cannot settle the behavior of every system years later.
The control that catches an easy mistake
Imagine testing whether a logged-in customer receives a different price from an anonymous visitor. You run the two requests and find a difference. A tempting conclusion follows immediately: login caused it.
Now add a second anonymous request at almost the same time. If the two anonymous requests also differ, some variation is occurring without the treatment you meant to test. Inventory, random experimentation, infrastructure or another uncontrolled feature may be involved. That extra control does not identify the cause. It shows why the first comparison was insufficient.
A strong design records the complete offer, repeats the comparison and decides in advance what difference counts as meaningful. It also keeps the treatment clear. If you change the device, location, account and referral route together, you have tested a bundle. You cannot then attribute the result to whichever feature makes the best headline.
Matching should also happen before looking at the result. If investigators exclude inconvenient comparisons only after seeing their prices, they can shape the answer. A clear protocol specifies eligible products, dates, sellers and terms, then records missing or failed matches. Exclusions may be necessary. They should be explainable.
When identical prices do not answer the whole question
A null result is useful when the test could have detected the effect of interest. If the test is weak, “we found no difference” can mean several things: no effect, too little evidence, the wrong comparison, or a mechanism the design did not expose.
There is a further complication. Benjamin Shiller’s 2025 theoretical analysis studies a mechanism in which a profile-informed price can remain temporarily available to other shoppers too. Under that mechanism, synchronized visitors can see the same price even though customer information helped set it. This is a modeled possibility, not proof that a particular retailer operates that way. It shows why the mechanism a test can detect must be stated. Shiller’s paper
Likewise, an audit of posted prices may miss differences in coupons, shipping, bundles or the products placed first. The answer is not to declare every unmeasured difference suspicious. It is to define the outcome before drawing the conclusion. Evidence of equal posted prices supports that narrow finding. It does not automatically support equality of the entire shopping experience.
A May 2026 preprint by Huang and Hooker discusses equivalence tests and statistical power for group-fairness audits of insurance pricing. Its methods offer a related lesson, rather than a retail-shopping finding. An equivalence test asks whether the evidence rules out differences larger than a chosen meaningful threshold. That is a different question from failing to reject exact equality. The preprint is useful methodological work to examine; it is not peer-reviewed proof of what retailers currently do. Preprint, version 1
A practical comparison a person can use
For an ordinary purchase, begin with the final amount and the terms you care about. Record the item or service, seller, date, currency, quantity, delivery and cancellation conditions. Note membership and coupon eligibility. If the price changes, preserve both records before assuming why.
Then ask whether another genuinely comparable offer is available. A lower number attached to worse terms may be a worse deal. A price that requires buying more than you need may save money per unit while costing more overall. These are arithmetic and preference questions you can often resolve without knowing the seller’s algorithm.
If you compare logged-in and logged-out offers, remember that you have changed membership or account context. Any difference is worth understanding, but that test alone cannot isolate hidden profiling. Private browsing is also not a guarantee of a better price or an anonymous identity. The recommendation here is to document and compare, not to promise a technological trick that always beats the system.
For a meaningful complaint, keep the observation separate from the explanation. “These matching offers differed at this time; can you explain the applicable condition?” is an answerable question. An accusation that the seller identified your desperation makes a much larger claim. It needs evidence of a mechanism and intent that the price comparison alone does not supply.
In my book The New Prince of Hard Times, I wrote, “Precision is how the citizen gets traction.” The same principle applies here. A specific record gives a buyer, journalist or regulator something to pursue. A sweeping allegation can bury the useful fact inside a claim nobody can verify. “Rules for Staying Human”
There is a limit to how much work we should ask a person to do. An exhausted customer should not need multiple devices, spare accounts and a statistics course to discover the terms of a routine purchase. Better individual comparisons help today. Better public standards should reduce the need for such effort tomorrow.
For researchers, the next step is a sharper record of what each test can rule out. For businesses, it is an offer that remains understandable when questioned. For shoppers, it is a manageable decision based on comparable terms. Those goals support one another, but they are not identical jobs.