Why measure
The rule that keeps only what measures better, which boards are published and why the live-page suites are not.
The project has one rule about changes to extraction: a claim is a measurement.
CONTRIBUTING.md states it — a change to extraction, reading order, routing or discovery
carries before-and-after numbers from the runners in benchmark/, and a neutral or
negative result is recorded as rejected so the next person does not repeat it. The
engineering journal (MEMORY.md) is mostly a list of obvious-looking improvements that
were tried and measured as neutral or harmful.
The rule exists because intuition about extraction is unreliable in a specific way. The
whole-page session of 14 September (docs/SESSION-18-WHOLE-PAGE.md) records the case: a
change that rendered links inside table cells was correct on the pages it was written for,
passed "WCXB =, Zyte =" for six pull requests, and had cost WebMainBench's table score
0.390 → 0.338 the whole time, because those two boards score plain text and cannot see a
table. Nothing caught it until the board that could see it was re-run.
Two products, two kinds of measure
The engine returns two things per page, and they need different instruments.
Filtered content (content_markdown) is what the public boards score: WCXB, Zyte's
article-extraction benchmark, WCEB and WebMainBench each ship cached HTML and a
human-annotated main content. They are comparable across systems — the other rows are
published figures or runs of the corpus's own scorer — and they are blind to everything
that needs a network or a second page: geometric reading order (no browser, so every page
falls back to DOM order, and the runners print that distribution rather than assert it) and
cross-page chrome detection (one page per site). See
Public boards.
The whole page (markdown) has no public board, and none is published here. The
repository carries three development suites that score it against Chromium's own visible
text on live pages (benchmark/fidelity, benchmark/live, benchmark/random_web); they
are diagnostics for a pull request, not numbers for a reader, because a figure scored
against a live page cannot be produced again once the page changes. Firecrawl's
scrape-evals was dropped from the boards for the same reason: 90 of its thousand URLs
were gone within a year.
| suite | scores | against | comparable across systems? | published |
|---|---|---|---|---|
| WCXB, Zyte, WCEB, WebMainBench, CleanEval, DAnIEL | filtered content, plain text (WebMainBench: Markdown) | human annotation on cached HTML | yes | yes |
| fidelity, live, random web | whole page / filtered content | Chromium innerText, live | no — a diff between two runs | no |
How to read any number here
Every figure on these pages names the commit it was measured at and the file it came from.
The public-board numbers are from apps/web/lib/benchmarks.ts (measured from fresh
clones on 21 September 2026, 928cf84; docs/research/2026-09-21-benchmarks-from-scratch.md
and 2026-09-21-router-canonical-and-multilingual.md)
and CHANGELOG.md for later commits. A number without a source does
not belong in these docs.
Neo4j sync
Push a site's knowledge graph into Neo4j (or Memgraph, or FalkorDB) over bolt in UNWIND/MERGE batches; the node and relationship shape; Aura's limits; why the database is a target and never the source of truth.
Public boards
WCXB (both splits), WCEB, Zyte article extraction, WebMainBench, CleanEval and DAnIEL — the numbers from a from-scratch run, who was run here beside the engine, and what each board cannot see.