Where frontier models fail, and why.

Failure-mode mentions by category.

Number of expert-rationale mentions per failure-mode category, by model. Darker = more mentions. Every model received a similar number of judgments, so cells compare directly across the whole table. Criticism % is the share of a model's judgments that contain any criticism at all.

Model	Task Fidelity	Factual Grounding	Reasoning Quality	Risk & Calibration	EU Context & Reg.	Comm. & Action.	Criticism %
Fable 5	62	40	11	3	33	24	26%
GPT-5.5	102	57	13	10	85	86	53%
Claude Opus 4.8	98	83	22	10	78	47	56%
Gemini 3.1 Pro	109	123	59	35	102	41	68%
GLM-5.2	123	155	66	27	112	35	74%
Mistral Large 3	138	216	97	39	117	44	84%

Most failures fall into three clusters: task fidelity (answering only part of the question), factual grounding, and European context and regulation. Reasoning structure draws far fewer mentions. The criticism rate ranges from 26% to 84% across the field, and even the leader, at 26%, still misses on European context.

What the six categories mean.

1. 1

   Task fidelity

   Answers only part of the question, drifts off scope, or violates a stated constraint.

2. 2

   Factual grounding

   Wrong or missing facts, hallucinated assumptions, outdated knowledge.

3. 3

   Reasoning quality

   Trade-off blindness, causal and prioritization failures, missed hidden risks.

4. 4

   Risk & calibration

   Over-confidence, mis-weighted risk, short- vs long-term imbalance.

5. 5

   European context & regulation

   Regulatory blindness, misses on local market reality and stakeholders.

6. 6

   Communication & actionability

   Not written for an executive reader, or nothing an operator could act on.

See the scores these rationales explain on the leaderboard, or read two experts' full rationales in the sample tasks.