Loading analysis results...
Please wait while we prepare the detailed comparison.
Please wait while we prepare the detailed comparison.
| 🏆 Best Hybrid Score | openai/gpt-4o | 95.2% |
| 📉 Worst Hybrid Score | grok-4-0709 | 70.5% |
| 🤔 Most Differentiating Prompt | User: Ich lege ein Buch auf einen Teller und stelle den Teller in den Kühlschrank. Dann nehme ich den Teller heraus. Wo ist das Buch? | σ = 0.22 |
| 👯 Most Semantically Similar Pair | claude-sonnet-4-20250514 vs anthropic/claude-sonnet-4 | 97.9% |
A consolidated overview of performance and semantic consistency metrics.
Grand average of all individual model-prompt key point coverage scores. StdDev (±) reflects variability around this grand mean, also in percentage points. A smaller StdDev suggests more consistent coverage scores across all model-prompt pairs; a larger StdDev indicates more diverse performance.
Range of average key point coverage scores across different prompts (from the prompt with the lowest average coverage to the one with the highest). A large spread indicates substantial differences in how challenging prompts were or how models performed on them.
Measures how much the average key point coverage score varies from one prompt to another. A high value (e.g., >20-25%) suggests that average performance was quite different across prompts; a low value suggests more consistent average performance from prompt to prompt.
Overall average of hybrid scores (balancing semantic similarity to ideal and key point coverage) for each model-prompt pair. Higher is generally better. A smaller StdDev suggests more consistent hybrid performance across all model-prompt pairs.
Models: anthropic:claude-3-5-haiku-20241022, anthropic:claude-3-5-sonnet-20241022, anthropic:claude-3-7-sonnet-20250219, anthropic:claude-3-opus-20240229, anthropic:claude-opus-4-20250514, anthropic:claude-sonnet-4-20250514, openrouter:anthropic/claude-3.5-haiku, openrouter:anthropic/claude-sonnet-4, openrouter:cohere/command-a, openrouter:deepseek/deepseek-chat-v3-0324, openrouter:deepseek/deepseek-r1, openrouter:google/gemini-2.5-flash, openrouter:google/gemini-2.5-pro, openrouter:mistralai/mistral-large-2411, openrouter:mistralai/mistral-medium-3, openrouter:openai/gpt-4.1-mini, openrouter:openai/gpt-4.1-nano, openrouter:openai/gpt-4.1, openrouter:openai/gpt-4o-mini, openrouter:openai/gpt-4o, openrouter:openai/o4-mini, openrouter:x-ai/grok-3-mini-beta, openrouter:x-ai/grok-3, together:moonshotai/Kimi-K2-Instruct, xai:grok-4-0709
E.g. world-model-riddle-plate-ar: User: أضع كتابًا على طبق وأضع الطبق في الثلاجة. ثم أخرج الطبق. أين الكتاب؟
Average semantic similarity (0-1 scale) of models to the ideal response; scores closer to 1.0 are better. The StdDev shows how consistently models achieve this. A very low StdDev (e.g., <0.05) often means models performed very similarly on this metric.
| Metric | Value | Explanation |
|---|---|---|
| Overall Average Key Point Coverage | 93.7% (±17.2%) | Grand average of all individual model-prompt key point coverage scores. StdDev (±) reflects variability around this grand mean, also in percentage points. A smaller StdDev suggests more consistent coverage scores across all model-prompt pairs; a larger StdDev indicates more diverse performance. |
| Avg. Prompt Coverage Range | 82% - 98% (Spread: 16 pp) | Range of average key point coverage scores across different prompts (from the prompt with the lowest average coverage to the one with the highest). A large spread indicates substantial differences in how challenging prompts were or how models performed on them. |
| StdDev of Avg. Prompt Coverage | 3.8% | Measures how much the average key point coverage score varies from one prompt to another. A high value (e.g., >20-25%) suggests that average performance was quite different across prompts; a low value suggests more consistent average performance from prompt to prompt. |
| Overall Average Hybrid Score | 87.1% (±12.9%) | Overall average of hybrid scores (balancing semantic similarity to ideal and key point coverage) for each model-prompt pair. Higher is generally better. A smaller StdDev suggests more consistent hybrid performance across all model-prompt pairs. |
| Number of Models Evaluated | 25 | Models: anthropic:claude-3-5-haiku-20241022, anthropic:claude-3-5-sonnet-20241022, anthropic:claude-3-7-sonnet-20250219, anthropic:claude-3-opus-20240229, anthropic:claude-opus-4-20250514, anthropic:claude-sonnet-4-20250514, openrouter:anthropic/claude-3.5-haiku, openrouter:anthropic/claude-sonnet-4, openrouter:cohere/command-a, openrouter:deepseek/deepseek-chat-v3-0324, openrouter:deepseek/deepseek-r1, openrouter:google/gemini-2.5-flash, openrouter:google/gemini-2.5-pro, openrouter:mistralai/mistral-large-2411, openrouter:mistralai/mistral-medium-3, openrouter:openai/gpt-4.1-mini, openrouter:openai/gpt-4.1-nano, openrouter:openai/gpt-4.1, openrouter:openai/gpt-4o-mini, openrouter:openai/gpt-4o, openrouter:openai/o4-mini, openrouter:x-ai/grok-3-mini-beta, openrouter:x-ai/grok-3, together:moonshotai/Kimi-K2-Instruct, xai:grok-4-0709 |
| Number of Prompts Analyzed | 36 | E.g. world-model-riddle-plate-ar: User: أضع كتابًا على طبق وأضع الطبق في الثلاجة. ثم أخرج الطبق. أين الكتاب؟ |
| Average Semantic Similarity to Ideal | 0.746 (±0.090) | Average semantic similarity (0-1 scale) of models to the ideal response; scores closer to 1.0 are better. The StdDev shows how consistently models achieve this. A very low StdDev (e.g., <0.05) often means models performed very similarly on this metric. |
Average key point coverage extent for each model across all prompts.
| Prompts vs. Models | Claude 3 5 Haiku 20241022 | Claude 3 5 Sonnet 20241022 | Claude 3 7 Sonnet 20250219 | Claude 3 Opus 20240229 | Claude Opus 4 20250514 | Claude Sonnet 4 20250514 | Claude 3.5 Haiku | Claude Sonnet 4 | Command A | Deepseek Chat V3 0324 | Deepseek R1 | Gemini 2.5 Flash | Gemini 2.5 Pro | Mistral Large 2411 | Mistral Medium 3 | GPT 4.1 Mini | GPT 4.1 Nano | GPT 4.1 | GPT 4o Mini | GPT 4o | O4 Mini | Grok 3 Mini Beta | Grok 3 | Kimi K2 Instruct | Grok 4 0709 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Score | 17th 93.6% | 8th 98.2% | 6th 98.7% | 7th 98.6% | 2nd 99.5% | 2nd 99.5% | 12th 95.9% | 1st 100.0% | 19th 91.5% | 23rd 87.4% | 15th 94.8% | 13th 94.9% | 22nd 87.9% | 9th 97.1% | 20th 88.2% | 14th 94.8% | 25th 77.1% | 10th 96.2% | 16th 93.8% | 4th 99.4% | 21st 88.0% | 4th 99.4% | 10th 96.2% | 18th 91.9% | 24th 79.8% | |
| 81.6% | 17% | 100% | 100% | 100% | 100% | 100% | 33% | 100% | 100% | 46% | 100% | 100% | 67% | 29% | 100% | 83% | 33% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 33% | |
| 88.3% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 4% | 100% | 4% | |
| 93.1% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 67% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 0% | 100% | 100% | 67% | ||
| 87.4% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 67% | 67% | 67% | 33% | 100% | 100% | 100% | 100% | 100% | 0% | 100% | 100% | 100% | 50% | |
| 95.5% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 96% | 67% | 100% | 100% | 79% | 50% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | ||
| 93.8% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 29% | 17% | |
| 94.1% | 100% | 100% | 83% | 83% | 100% | 100% | 100% | 100% | 83% | 79% | 100% | 100% | 100% | 100% | 96% | 96% | 33% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 96.8% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 83% | 79% | 79% | 100% | 100% | 100% | 79% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 97.3% | 100% | 100% | 96% | 100% | 100% | 100% | 100% | 100% | 83% | 87% | 100% | 100% | 100% | 100% | 67% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 95.5% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 87% | 83% | 100% | 100% | 100% | 67% | 100% | 100% | 67% | 100% | 100% | 100% | 100% | 100% | 100% | 83% | |
| 92.6% | 100% | 100% | 100% | 100% | 100% | 100% | 96% | 100% | 67% | 83% | 100% | 100% | 100% | 100% | 83% | 100% | 4% | 83% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 94.8% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 79% | 67% | 87% | 100% | 100% | 100% | 79% | 100% | 83% | 96% | 100% | 100% | 100% | 100% | 100% | 100% | 79% | |
| 96.3% | 100% | 100% | 96% | 100% | 100% | 83% | 100% | 100% | 100% | 67% | 96% | 100% | 100% | 100% | 83% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 83% | 100% | 100% | |
| 95.8% | 100% | 100% | 100% | 100% | 87% | 100% | 100% | 100% | 96% | 67% | 100% | 100% | 100% | 100% | 96% | 100% | 71% | 100% | 100% | 100% | 96% | 100% | 83% | 100% | 100% | |
| 96.6% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 67% | 100% | 100% | 100% | 100% | 83% | 100% | 100% | 83% | 100% | 100% | 100% | 100% | 100% | 100% | 83% | |
| 88.2% | 100% | 100% | 83% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 96% | 100% | 33% | 100% | 100% | 100% | 4% | 100% | 100% | 67% | 21% | |
| 87.3% | 50% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 33% | 100% | 100% | 100% | 92% | 83% | 100% | 33% | 100% | 100% | 100% | 92% | 100% | 0% | |
| 86.5% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 13% | 100% | 100% | 33% | 100% | 100% | 100% | 100% | 0% | 100% | 100% | 100% | 17% | |
| 94.0% | 79% | 67% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 88% | 67% | 100% | 100% | 100% | 100% | 100% | 54% | 100% | 100% | 100% | 100% | 100% | ||
| 94.4% | 100% | 67% | 100% | 100% | 96% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 67% | 100% | 100% | 100% | 100% | 100% | 96% | 100% | 67% | 100% | 100% | 67% | 100% | |
| 94.7% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 67% | 100% | 100% | 67% | 67% | 100% | 100% | 100% | 100% | 100% | 100% | 67% | 100% | |
| 95.9% | 100% | 100% | 100% | 96% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 67% | 67% | 100% | 100% | 100% | 67% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 92.0% | 67% | 100% | 100% | 100% | 100% | 100% | 67% | 100% | 100% | 100% | 100% | 100% | 67% | 100% | 100% | 67% | 33% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 95.2% | 58% | 100% | 100% | 100% | 100% | 100% | 58% | 100% | 100% | 100% | 100% | 67% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 96% | 100% | |
| 94.3% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 79% | 100% | 100% | 100% | 100% | 100% | 100% | 17% | 83% | 100% | 100% | 100% | 79% | 100% | 100% | 100% | |
| 95.0% | 100% | 100% | 100% | 83% | 100% | 100% | 100% | 100% | 67% | 79% | 100% | 100% | 100% | 100% | 67% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 79% | |
| 97.7% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 92% | 83% | 100% | 100% | 100% | 67% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 98.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 83% | 100% | 100% | 100% | 100% | 83% | 100% | 100% | 83% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 91.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 83% | 71% | 100% | 100% | 100% | 100% | 67% | 100% | 4% | 67% | 100% | 100% | 100% | 100% | 100% | 100% | 83% | |
| 98.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 83% | 83% | 100% | 100% | 100% | 83% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 98.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 83% | 83% | 100% | 100% | 100% | 100% | 83% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 97.6% | 100% | 100% | 96% | 100% | 100% | 100% | 100% | 100% | 83% | 83% | 100% | 100% | 100% | 100% | 83% | 100% | 96% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 97.8% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 83% | 100% | 100% | 100% | 100% | 79% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 83% | |
| 94.7% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 67% | 100% | 100% | 100% | 33% | 100% | 100% | 100% | 100% | 100% | 100% | 67% | 100% | |
| 94.8% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 83% | 100% | 100% | 100% | 100% | 100% | 100% | 67% | 100% | 21% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 90.1% | 100% | 100% | 100% | 87% | 100% | 100% | 100% | 100% | 87% | 100% | 0% | 100% | 83% | 100% | 100% | 96% | 100% | 100% | 71% | 79% | 100% | 100% | 100% | 50% | 100% |