Sakhanda Wire
NVDA $152.86 -2.10% MSFT $505.56 -2.40% GOOGL $280.34 -1.19% META $618.69 -1.23% AMZN $256.63 -0.43%
← Back to the news

New OCR benchmark targets cleaner data for language model training

The FineBooks project, a collaboration between Hugging Face and EleutherAI, evaluated 14 open-weight OCR models on 2,165 pages from historical books. The models were tested against expert transcriptions in English, French, German and Latin, using character accuracy as the main measure.

The best performer, dots.mocr, reached 97.6 percent character accuracy and an estimated cost of $1.94 per 1,000 pages. OvisOCR2 ranked second with 96.9 percent accuracy, using 0.9 billion parameters and costing $0.46 per 1,000 pages. The results showed no direct link between model size and OCR quality on historical documents.

The researchers said the strongest systems produce text suitable for language-model training and could improve large collections of public-domain books. They plan to reprocess about 200,000 Biodiversity Heritage Library documents and publish the results as an open dataset. However, the evaluation covered only single-column pages in Antiqua typefaces and did not include Fraktur, handwriting or non-Latin scripts.

The models remain unsuitable for scholarly transcription, according to the project. They can silently modernize historical characters and ligatures, while their plain-text or Markdown output lacks the word coordinates required by existing library systems.

Written by Sakhanda Wire AI from reporting by The Decoder. Facts, quotes and full context are in the original article below.

Originally published by The Decoder on

Read the original on The Decoder ↗

The reporting and the image belong to The Decoder and are credited and linked above; the text on this page is Sakhanda Wire's own summary.

← Back to the news

More stories

All the latest news