The sentence reaches a new customer without its photograph.
This customer has asked an assistant to compare local electricians. The answer includes a short account of Tess’s business. One line describes a concern about her billing and supplies a link. The customer has never seen the original image. What arrives now looks like the result of a search already completed on his behalf.
He opens the link. It leads to a discussion page which reproduced the image. Below the post is a correction. The generated answer has given him the allegation and left out the correction.
At this point, blaming the machine in general would be less helpful than identifying the two sentences. What did the answer say? What does the page actually support? Those questions allow the customer to locate the failure without first knowing which technical component caused it.
The answer is itself a new record. If the customer copies it into a message to the building committee, that record will have another audience. It may no longer carry the link. The committee may treat the customer’s note as the product of his research. A false attribution can acquire an additional layer of apparent checking even when the last person has only read a summary.
I would assign responsibility at that last step quite firmly. If you use an automated answer to make or recommend a consequential decision about a person, you owe the person a check of the supporting evidence. The convenience of the answer belongs to you. So does the duty to examine it before passing its cost to someone else.
NIST’s July 2024 Generative AI Profile identifies confidently presented false outputs and fabricated supporting citations as risks. It recommends reviewing output sources and citations and assigning incident-communication procedures. This is voluntary guidance, not a measured error rate or a guarantee about any product. Its value here is narrow: a citation supplied by a system still needs verification. S01
Find the sentence that changed
Let us follow three versions of the customer’s answer. They call for different correction requests.
In the first, the output invents an interview. It states that Tess admitted a practice in a trade magazine, but the linked magazine page contains no interview with her and no such admission. The complaint should identify the invented attribution and the mismatch between the sentence and the citation. Calling the whole magazine unreliable would miss the point. In this version, the page never made the claim being assigned to it.
In the second, the generated answer accurately summarizes the discussion page as it stood before correction. Its report of that page’s allegation is faithful. Yet the allegation itself is unsupported. The problem includes the underlying publisher’s error. We cannot solve it merely by asking for a better summary of the same bad account. We need the source corrected and the output to reflect the evidentiary status of what it repeats.
In the third, the page clearly presents a claim as unverified, while the answer presents it as an established event. A small change in wording has changed the burden of proof. Someone alleged becomes she did. The output has gone beyond the evidence even if every name and date came from the page. A summary must preserve qualifications that determine whether its central statement is justified.
These are proposed categories for reading an answer, not a test that exposes a system’s internal cause. The same visible mistake could have more than one technical explanation. A user can document that an answer omits a correction without claiming to know whether the source was retrieved before an update, an excerpt was incomplete, or a later generation step lost the qualification.
There is a fourth problem worth separating because it can contaminate the rest of the inquiry. Two people can share a name. A correct account of one person’s business does not become evidence about another person’s business because an answer combines them. Before investigating the alleged conduct, compare the identifying details needed for the task. A different town, occupation or organization may reveal that the answer concerns someone else. Use only the identifying information needed to resolve that match; do not collect private details merely to make the file feel thorough.
The practical question is what should change. An invented interview calls for removal of an unsupported sentence. An outdated summary calls for checking the corrected source. A merged identity calls for separating the records. The person affected should not have to guess a provider’s technical vocabulary to make those differences understood.
Two records, two dates
Tess receives the customer’s screenshot. She asks him to keep the full answer and the question that produced it. He sends a copy with unrelated parts of the conversation removed. It shows the exact words, the displayed source link and the time he received the answer.
That preserves one event: the output he encountered. The source page is a second event to document. Tess records its address, the relevant passage and when she inspected it. If the page displays an update date, she notes that as the page’s stated date. She does not silently convert it into the date when every search service or summarizing system must have received the change.
A short excerpt beside a search result deserves the same separate treatment. Suppose it displays the old allegation while the opened page carries a correction. Preserve both, including the search wording and inspection time. Ask for review of the displayed excerpt as well as the page when the service provides an appropriate route. Do not assume that changing one has changed the other. A search excerpt is the text the searcher encountered, even when a fuller answer was available one click away.
This distinction can prevent an argument about the wrong record. The provider might inspect today’s page and find a proper correction. The customer might still possess yesterday’s misleading answer. Both records can exist. Showing the correct page today does not prove that the earlier answer was accurate, while the old answer does not prove that the mistake continues in every current response.
A report can be short and still preserve these boundaries. It can identify the sentence at issue, explain what the cited source does and does not say, and describe the consequential use already known. An attached capture should contain enough context to establish the mismatch. It need not expose the customer’s private conversation or turn Tess’s complaint into a public archive of every allegation.
If a service displays a model name, output identifier or shareable record, that information may help locate what happened. Record what is actually available. Do not infer a version from the date or fill an empty field with a plausible guess. A report with an honest blank is more reliable than a complete-looking report with invented detail.
Tess also keeps two questions apart: can someone reproduce the output, and did the customer receive it? A saved record can support the second even when a fresh request produces different wording. A different later answer supplies evidence about that later request. By itself, it cannot cancel the earlier record.
The reverse matters too. One repeated error does not establish how often a service produces it. A search designed to investigate Tess’s incident should not be described as a product-wide audit. There is dignity in a modest, accurate claim: this answer appeared in this exchange and should not have been used as stated.
Ask each recipient for the part they can repair
The discussion-page publisher can correct its post and make the correction clear to readers. A service provider can receive a report about an output and investigate whatever part of its system it controls. The customer can withdraw his committee note. These requests overlap, but none substitutes for all the others.
Suppose the provider acknowledges Tess’s report. That is progress in handling the complaint. It is not evidence that the committee received a correction. Suppose the committee removes the note from its working document. That improves its record, but says nothing about the public source page. A repair ledger should name the object being changed so that one success is not mistaken for total repair.
This is especially important when a request crosses organizations. Each may reply about its own responsibility in terms that sound final. A publisher says the page has been updated. A provider says feedback has been received. A committee says its recommendation is under review. Tess should be able to record these as three different states without having to accuse anyone of dishonesty or pretend the work is finished.
A useful correction request would ask for an answer that can be checked: which output or page was reviewed, what factual change was made, and whether any known downstream record was notified. Some recipients may not be able to provide all of that information. They should say what they can confirm and what they cannot. Silence about a hidden process should remain unknown in the record.
The book’s proposed standard for organizations is stronger than requiring the affected person to submit the same explanation indefinitely. Once an organization has accepted a well-supported correction, it should retain enough of that decision to handle a recurrence intelligently, with access and retention limited to the purpose. A later staff member should be able to find the earlier resolution without asking Tess to reconstruct the incident from the beginning.
That proposal also needs a protection against misuse. A complaint system should not accept the subject’s preferred account merely because it arrived first or arrived often. It should preserve the distinction between a disputed statement and an established error, allow relevant evidence to be examined, and correct its own decision if that evidence changes. Otherwise, a repair system could become another route for suppressing justified criticism.
The search can acquire its own bad method
A helpful friend offers to check whether the false claim still appears. He starts asking several assistants increasingly leading questions about Tess’s supposed admission. Eventually one returns the words he was looking for. He sends it to her as proof that the internet still believes the story.
What has he proved?
He has documented an output under a particular question, assuming he saved the question and answer honestly. He has not established that an ordinary customer would encounter the same response, that many people have seen it, or that the system holds a stable belief about Tess. His own question may have supplied the disputed premise. The output still deserves scrutiny, but the test conditions belong beside the result.
For an incident check, I would begin with the actual query that produced the known problem, if it is available and appropriate to repeat. I would decide what outcome matters before trying again. Does the answer still assign the invented quotation to Tess? Does it cite a page that contradicts it? Is a public summary still missing the correction? These are concrete questions with recordable answers.
I would avoid generating a collection of ever more elaborate accusations in order to measure whether they can be elicited. That exercise can create fresh harmful text while drifting away from the original task. Specialist testing may have a separate purpose and safeguards. A person seeking correction of one false statement need not undertake it.
Even a well-bounded check should have an owner and an end condition. The customer may agree to examine his committee’s records once the correction is delivered. Tess may ask a trusted helper to check the known public copy after its publisher confirms an update. A further review could be triggered by a new report from someone who actually encountered the claim. The scope should follow the risk being addressed, rather than expand every time a new wording is imaginable.
Put the human decision back on the record
The committee’s draft recommendation contains a line saying Tess presents a reputational concern. Nobody remembers writing those words. They were copied from the customer’s summary, which was copied from the automated answer. The phrase no longer identifies the disputed quotation or the page behind it.
Deleting only the quoted sentence would leave this judgment untouched. The committee must ask whether the broader concern has any independent basis. If it does, that basis should be stated and assessed fairly. If the phrase rests only on the unsupported attribution, it should be withdrawn with it. Otherwise a factual error survives by becoming an impression.
The customer opens the draft and finds every place his summary was used. He sends the committee a correction that names the unsupported part. He links to the publisher’s withdrawal and asks that the comparison be reconsidered without that allegation. The committee records the change rather than quietly pretending the earlier version never existed.
Tess still has to submit an estimate. Another electrician may offer a better price or have the right availability. A fair outcome does not require Tess to win the job. It requires the committee to judge the actual choice before it, using evidence it can defend.
The corrected comparison is shorter. It lists the work, the proposed dates and the estimates. Beside Tess’s entry is a link to her own description of what she can do. The invented interview is gone. The customer now has the task he meant to delegate in the first place: choosing someone to fix the building.