Every public benchmark, including the ones we lose
Six boards, six definitions of a correct extraction, one official number per board, and only corpora that ship their own HTML, so every number here can be produced again by anyone with the clone. Where a rival could be installed it was run here, through the corpus’s own scorer, over the same bytes. The row for this engine is what the package ships. It is first on the one board built for pages that are not articles, second on the widest independent English board, second on the one that grades tables and code, eleventh of thirty-five on the oldest, and fourth on the multilingual one. All of it is below, the losses included.
Rows for this engine measured 21 September 2026 (evening) on main@928cf84, with the runners in benchmark/, from fresh clones. Every other row says whether it was run here or is its authors’ published figure.
WCEB
3,985 pages · 8 corporaSome rows are published figures
Webis Web Content Extraction Benchmark: eight older annotated corpora combined by a university group with no extractor of its own (SIGIR 2023)
How much of the annotated main content survives, across eight independently built corpora.
metric: ROUGE-LSum F1
webgraph — production path, the content field; run here 21 Sep 2026 (evening); was 0.832
The widest corpus here and the only English one whose authors have no system in the comparison. Second, 0.007 behind trafilatura. Two of the eight corpora (Dragnet and CETD, 2,079 pages) count the comment thread under an article as content; the engine returns the thread as a separate field, and with that field joined on it scores 0.885, ahead of trafilatura — reported here in words rather than as a second column, because it is the same run. In the morning run this row was 0.832, sixth: the page-type router typed these older news pages as listings, and the evening fix — route by the declared canonical, train on five languages — is what moved it.
How the rows were produced. Every row is scored on the same cached HTML that ships with the corpus, with the corpus's ROUGE-LSum restated and checked against the authors' own per-page CSVs (worst disagreement 8e-06). Baselines are the authors' published per-page scores over those same files, comment threads excluded, which is how this engine's row is scored too.
WCXB
1,497 pages · 7 page types · official leaderboard splitEvery row run here
Web Content Extraction Benchmark: 1,497 human-annotated pages over seven page types — the split the official leaderboard at webcontentextraction.org ranks on
Whether an extractor holds up away from articles, on forums, products, listings and documentation.
metric: word-level F1
webgraph — routed out of fold; run here 21 Sep 2026 (evening); 0.837 without routing; was 0.862 before the router learned five languagesrs-trafilatura — published; author's own, tuned on this splitrs-trafilatura 0.1.1 — run here, same scorer, same bytesMinerU-HTML — published 2026; not re-run heretrafilatura 2.2.0 — run hereresiliparse 1.0.9 — run hereReaderLM-v2 — published; not re-run herereadability-lxml 0.9 — run here; 0.675 publishedjusText 3.0.2 — run here
The page-type router trained on this split, so the engine's row uses its out-of-fold predictions (a router that never saw the page). The corpus ships 139 of these pages in the held-out test split as well, byte-identical, and its own scorer counts them here. First of the systems run here by 0.014; against the author's published figure the margin is 0.002, a tie. On the corpus's second, 511-page held-out split the engine scores 0.878 to rs-trafilatura's 0.891, second.
How the rows were produced. Every row marked “run here” is that extractor with its defaults over the same cached HTML, written in the corpus's results format and scored by the corpus's own evaluate.py in one command per file. rs-trafilatura is the benchmark author's extractor, built from its PyPI sdist. Published rows are kept for the systems that could not be installed (neural models) and for the author's own figure, which the Python build does not reproduce on this split (0.847 vs 0.859).
WebMainBench
545 pages · calibrated subsetSome rows are published figures
OpenDataLab's Markdown benchmark: 545 pages with element-by-element ground truth, tables, code and equations graded apart
Whether tables, code and equations survive as structure, not just as words. The only board that grades them separately.
metric: edit distance, 5 columns
webgraph — column mean, production path; run here 21 Sep 2026 (evening); per-page overall 0.727; was 0.643
Built by the team that also built the first and second place systems, whose published rows were produced with an LLM on. Its ground truth for tables is the source HTML, which favours an extractor that returns markup unchanged.
How the rows were produced. Cached HTML shipped with the corpus, scored offline with the corpus's own calculator (the June 2026 revision needs python-dotenv, without which it silently scores nothing). The engine's figure is a plain mean of the five columns, which is how the published rows are computed; trafilatura and resiliparse were also run here through the same calculator as a check (0.357 and 0.217 by the same mean).
DAnIEL
1,689 pages · Greek, English, Polish, Russian, ChineseEvery row run here
The DAnIEL corpus: news articles in Greek, English, Polish, Russian and Chinese, from the SIGIR 2025 study of extractors across languages
Whether an extractor built on English pages still works in four other scripts. The only non-English board; independent for extraction, no longer for routing.
metric: ROUGE-LSum F1, macro over languages
readability (paper) — the study's published rowreadability-lxml 0.9 — run hereboilerpipe (paper) — the study's published rowwebgraph — production path; run here 21 Sep 2026 (evening); was 0.582trafilatura (paper) — the study's published rowtrafilatura 2.2.0 — run herejusText 3.0.2 — run here; no Chinese stoplistresiliparse 1.0.9 — run here
Fourth of nine rows (readability appears twice: the study's figure and the run here), behind readability and boilerpipe; 0.582 in the morning run, when the router called 987 of these articles listings. The corpus records no URLs, so 978 pages are routed under a fabricated host root, which reads as a homepage; the 711 that declare a canonical score 0.842. Whitespace tokens make the Chinese column a sentence-match for every system, which is how the study reports it.
How the rows were produced. Every row marked “run here” is that extractor over the same 1,689 cached files, scored with the SIGIR 2025 study's own metric restated (ROUGE-LSum over whitespace tokens). The restatement reproduces the study: readability 0.838 here against 0.840 published, trafilatura 0.762 against 0.773. Since 21 September the page-type router trains on the 711 of these pages that declare a canonical URL; the board routes those out of fold, and the other 978 live.
Zyte article-extraction
181 news articlesEvery row run here
Zyte's article-extraction benchmark: 181 news articles with a hand-checked body, 34 published systems
How cleanly a news article body comes out, against a field of article-specific extractors.
metric: 4-gram shingle F1
webgraph — 11th of 35, production path; run here 21 Sep 2026
Every leader here is built for news articles and nothing else. This engine is general-purpose, which is the trade, not an excuse. The production path is 11th; the rules boundary without routing scores 0.946, 10th. Shown: the top of a 35-entry board.
How the rows were produced. Every competing row was produced by running the corpus's own scorer over the outputs those systems committed to the repository, in the same command that scored this engine.
CleanEval
681 gold pages · 2007 shared taskSome rows are published figures
CleanEval, the 2007 shared task on cleaning web pages: paid annotators, the organisers' scorer, nine teams
The original shared task for this problem, scored by the organisers' own program, eighteen years on.
metric: cleaneval.prl Ave (text and markup / text only), %
webgraph — 24-page sample, ± 1.7; production path; run here 21 Sep 2026 (morning engine; Evert word-F 87.3 → 87.8 in the evening)
The one board no interested party arranged: paid annotators, an independent scorer, nine teams. Over all 681 pages with Evert's fast scorer the production path scores 87.8 word-F (the rules boundary 88.3). Participant names are omitted because the only copy of the paper's table available collapses its columns.
How the rows were produced. The 2008 paper's own Perl scorer (cleaneval.prl) over a seeded 24-page sample of the 681 gold pages, at about seventy seconds of Perl per page pair; the 2007 participants' rows are the paper's Table 5 over the full corpus. A range check, not a ranking: the engine's figure carries its standard error.
WCXB by page type
The overall number is 53% articles, which every system reads well. The corpus was built to show the other six, and that is where the systems separate: within a few points on articles, twenty apart on collections. Best cell in each row in bold, whoever holds it.
Page type
n
webgraph
rs-trafilatura 0.1.1
rs-trafilatura (published)
MinerU-HTML (published)
trafilatura 2.2.0
article
793
0.945
0.934
0.932
0.928
0.928
documentation
91
0.921
0.938
0.932
0.838
0.914
service
165
0.845
0.825
0.844
0.824
0.759
forum
113
0.787
0.760
0.808
0.794
0.689
collection
117
0.692
0.672
0.716
0.506
0.564
listing
99
0.718
0.669
0.707
0.710
0.559
product
119
0.634
0.636
0.641
0.619
0.618
Where the distance sits
An overall score averages five columns and hides that four of them are close. The honest picture is per column.
this engine MinerU-HTML, the leader
Equations started the day at 0.307 and half the gap was currency: an unescaped dollar amount reads as a maths delimiter, so two prices in one paragraph scored as a formula wrapping the prose between them. The rest was MathML, which the engine deleted before extraction began. Tables started at 0.349; a merged-cell table cannot be written in Markdown pipe syntax without losing the merges, so those now keep their own markup, and tables that were really navigation widgets are no longer emitted as tables at all. This round moved prose 0.767 to 0.774 and equations 0.517 to 0.605, and cost tables 0.425 to 0.404 — not yet diagnosed, and recorded rather than hidden.
Looked at and not run
A page that lists only the boards it scores on is an advertisement. These are the ones a reader would expect to see here, and why they are not.
Firecrawl scrape-evals
1,000 live URLs
It fetches at scoring time, and ninety of its thousand URLs already return 404 ten months after the dataset was cut; a number produced on it describes the web on the day it ran and cannot be reproduced by anyone later. It was run here once (0.446 best-window F1 from one address, no proxy, September 2026) and is kept out of the boards for that reason. Firecrawl itself has withdrawn the benchmark.
CrawlBench (Firecrawl)
LLM structured extraction
It scores JSON pulled from live pages against a schema by a language model, not the Markdown of a page, and the dataset is not published; running it needs an LLM key and their harness. This engine's schema extraction could be scored on it, and has not been.
Hydrafetch extraction benchmark
same corpus as WCXB
Not a separate corpus: it is the WCXB dev split scored with WCXB's own metric, published with Hydrafetch's own extractor added. The row for this engine is the same WCXB run, so nothing was run twice.
Dragnet and Boilerpipe corpora
inside WCEB
Both are among the eight corpora WCEB combines (Dragnet as its own set; the Boilerpipe-era L3S-GN1 and Google-Trends sets alongside it), so they are scored above under WCEB rather than a second time on their own.
trafilatura's own evaluation
750 pages, the maintainer's
Run and published by trafilatura's maintainer over a corpus of their choosing; WCEB covers the same ground with a group that has no extractor in the field.
How to read these
Rows marked as this engine are runs of the benchmark runners in this repository, on one machine, against a fresh clone of each corpus, and they are the production path — the page-type router, its per-type policy, the content boundary and the title, exactly as the API and CLI run it. A rival marked “run here” was installed with its defaults and scored by the corpus’s own program over the same files; where that was not possible the row is the authors’ published figure and says so.
Bars start at zero and rankings are never reordered. Where this engine places eighth it is drawn eighth.
One thing none of these boards can measure: every one hands each extractor the same saved HTML file, so fetching is factored out of the result entirely, which is deliberate — a board that fetches live pages cannot be reproduced once its URLs rot, and no such figure is published here. And every metric here is a bag of words or an edit distance over one, which cannot distinguish a correctly read two-column page from one read straight across, because both contain the same words. The engine’s reading-order work is invisible to all of it, in both directions.