We Quoted Reviews That Do Not Exist

Our methodology page explains the three levels of evidence a product can qualify under. Each level ended with an example, and the examples quoted star ratings and review totals in quotation marks, as if they had been lifted from somewhere.

They had not been lifted from anywhere. Nobody wrote those reviews. No product on this site carries those figures. They were written as illustrations of what a strong evidence trail looks like, and they had been sitting on a trust page long enough that they read as findings.

Two lines above them, the same page says we do not make claims we cannot back up.

Why they are deleted and not corrected

Every other repair on this page swapped a wrong figure for the right one. These two could not be repaired, because there is nothing to repair them to. A rating with no product behind it has no true version. So they are gone, and the levels now describe what qualifies without pretending to quote anyone.

The thresholds around them stayed exactly as they were. Those are rules we set, not measurements of anything, and stating a rule is not the same as quoting evidence.

The page also disagreed with itself

The same review found the page publishing one figure for how many things we record about a product, while the summary that appears under our link in search results published a different figure for the same fact. Both were typed by hand, on a page that has been able to count that number automatically for weeks.

Nothing on this page is typed by hand any more. The headline dropped its figure completely, the machine-readable summary we publish for AI systems dropped its version of it, and everything countable now resolves as the page loads. The search-results summary still carries the older figure; it lives in a different place and is corrected next.

Describing something that no longer runs

Two paragraphs counted the checkpoints in our verification process. That process has since been rebuilt, so the count described machinery that is not there. The paragraphs now describe what each stage has to prove instead, which stays true when the stages change and a number does not.

The same page had a second one

Further down, a case study showed what our deep research finds that a standard product listing misses. It quoted a candle maker’s back story: a previous employer, a job title, a line about how the business started, a shop in the same town as the workshop.

Checked against that product’s own record, most of it was not there. The founder’s name, the town and the price were real. The rest appeared nowhere in the product record or its research note, one detail contradicted what we do hold, and the story named one founder where our record holds two people who started the company together.

That one could not be repaired either, for a different reason. The candle predates the process we run today, so it has no locked source file at all. There was nothing to check a rewrite against.

So the example was replaced rather than edited, using a product that did go through the current process. It is a fountain pen whose nib is stamped Fine while owner after owner reports it writing broader, one of them with measurements, one who had the nib reground. Our record files that as an open disagreement and publishes every voice with its source attached, including the owner who argues the opposite. Every line of it traces to a numbered claim and a quote we can show you.

What this cost us to find

Our scanner reads our own pages looking for claims that have quietly gone stale, and it found every counting problem. It found neither of the invented ones, because a made-up figure is not stale. It never had a source to drift away from.

That gap is the lesson we are keeping. A tool can tell you when a number stopped matching its source. Nothing we had could tell us a sentence never had one. Both were caught the same way in the end, by a person reading a line and asking where it came from.

So we built the check that was missing. It now takes every sentence we put in quotation marks anywhere on the page and proves it exists, character for character, in the research file it claims to come from. Run against the page as it stands, nothing is unaccounted for. Run against the version from this morning, it finds all four.

The rebuilt page is at our methodology.