We Told Machines Our Research Log Was Smaller Than It Is

Every page on this site carries a short description written for machines rather than for readers. Search engines and AI assistants read it to decide what a page is about. You do not normally see it.

The one on our research log ended with a claim about its own size, and that claim was typed by hand a long time ago and never touched again.

What it said, and what was true

It described the page as raw notes from more than a hundred audits.

There were seven hundred and thirty five notes. Five hundred and eighty five of them are product verdicts, and the number had been wrong for months in the direction that undersells the work rather than overstates it, which is the only reason nobody noticed.

It was not a lie and it was not a guess. It was a true sentence that stopped being true while nobody was reading it. That is what every hand-typed number on this site eventually does, which is why we are steadily replacing them.

The second thing on the same page

Alongside that description, every page hands machines a small structured summary of itself. The one for our research log described the page as a collection and then never said what the collection contained. A machine was told a library existed and was handed no catalogue.

The published notes about our own system also described this page’s structured summary incorrectly, naming the format used by an individual note rather than the one used by the listing, and described our product verdict feed as holding only rejections when it has always held both verdicts.

What changed

The description now reads its numbers from the same place the page does, so it cannot drift again. If those numbers are ever unavailable, the sentence renders without any figure at all rather than with a stale one.

The structured summary now lists what is actually on the page, built from the very same code that draws the page, so the two cannot come apart. The three descriptions in our published system notes are corrected.

How we found it

We were rebuilding the page for other reasons and checked its published claims as part of that work. It was not caught by a monitor, because we had no monitor that reads a sentence and asks whether it is still true. Building one is on our list.