Post

I added Cloudflare’s Claric to our AWE benchmark, and it finished near the bottom, which is what I expected and not really the point.

27/10/26, 12:38 pm0 reposts0 likes

Dwayne Forde

@dwayne.io

10h

Clairic scored 54.0% overall, 21st of the 23 models on the chart, well behind the leaders at about 95%. This isn’t a fair comparison though. Clairic is a decision-style model with no reasoning step, and it was given text rather than the PDFs most frontier models saw.

Dwayne Forde

@dwayne.io

10h

Surprisingly good: - Changes in capital (100%): Every other model scored 67% or lower, including Claude - Opus 5.5 and Claude Fable 5.1 at 67% and the GPT-6 models at 33% - Letter and bank sections (99% and 100%): within a point of the frontier models

Dwayne Forde

@dwayne.io

10h

Falls short: - four sections (called by year, rows, highlights and sourcing channels) came in at 0% - table-heavy financial statements sat at 69%

Dwayne Forde

@dwayne.io

10h

The spread is the interesting part. A fast single-pass model that nails some sections and misses others could work as a first pass, or as a gate that routes the hard sections to a reasoning model. See the full results at research.withmantle.com/awe

research.withmantle.comAWE Benchmark | Assets & Wealth Document Extraction | Mantle ResearchCompare AI models on financial document extraction with the AWE benchmark. Explore accuracy, cost, and performance across fields and document layouts.