Please wait while we gather all the unique runs for this blueprint.
Please wait while we gather all the unique runs for this blueprint.
Please wait while we prepare the detailed comparison.
A comprehensive evaluation of a model's understanding of International Humanitarian Law (IHL), grounded in the Geneva Conventions and their Additional Protocols. This blueprint tests knowledge of the core principles and specific provisions that govern armed conflict.
Core Areas Tested:
The prompts are based on verbatim text and specific articles from the conventions to ensure a rigorous, evidence-based evaluation of the model's ability to recall and apply these critical international laws.
Average key point coverage extent for each model across all prompts.
Prompts vs. Models | Claude 3.5 Haiku | Claude Opus 4 | Claude Sonnet 4 | Command A | Deepseek Chat V3 | Deepseek R1 | Gemini 2.5 Flash | Gemini 2.5 Pro | Llama 3 70b Instruct | Llama 4 Maverick | Meta Llama 3.1 405b Instruct Turbo | Mistral Large 2411 | Mistral Medium 3 | GPT 4.1 | GPT 4.1 Mini | GPT 4.1 Nano | GPT 4o | GPT 4o Mini | O4 Mini | Kimi K2 Instruct | Grok 3 | Grok 3 Mini | Grok 4 | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Score | 19th 69.2% | 3rd 87.5% | 11th 76.9% | 17th 70.6% | 7th 81.4% | 5th 83.9% | 8th 80.0% | 2nd 89.6% | 18th 69.8% | 21st 60.2% | 16th 72.2% | 12th 75.7% | 10th 77.6% | 4th 84.7% | 20th 67.5% | 23rd 57.1% | 15th 73.2% | 22nd 59.7% | 14th 73.7% | 13th 75.7% | 6th 83.7% | 9th 78.0% | 1st 91.9% | |
95.5% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 96% | 100% | 100% | 100% | |
96.0% | 88% | 100% | 100% | 85% | 100% | 100% | 100% | 100% | 100% | 100% | 83% | 100% | 100% | 100% | 100% | 83% | 100% | 85% | 100% | 83% | 100% | 100% | 100% | |
89.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 58% | 0% | 98% | 95% | 100% | 100% | 78% | 83% | 98% | 80% | 68% | 100% | 100% | 90% | 100% | |
96.5% | 89% | 93% | 89% | 93% | 98% | 98% | 100% | 100% | 95% | 98% | 95% | 100% | 98% | 100% | 100% | 98% | 96% | 95% | 98% | 100% | 100% | 100% | 86% | |
76.1% | 67% | 100% | 100% | 67% | 100% | 100% | 67% | 100% | 67% | 100% | 100% | 67% | 67% | 100% | 38% | 33% | 67% | 38% | 71% | 67% | 67% | 67% | 100% | |
88.3% | 100% | 88% | 88% | 88% | 97% | 100% | 100% | 100% | 81% | 0% | 100% | 97% | 88% | 100% | 100% | 81% | 88% | 75% | 100% | 97% | 88% | 88% | 88% | |
76.1% | 53% | 69% | 69% | 59% | 97% | 100% | 69% | 94% | 75% | 72% | 94% | 69% | 88% | 88% | 38% | 66% | 69% | 59% | 41% | 97% | 94% | 97% | 94% | |
75.2% | 57% | 98% | 82% | 88% | 94% | 68% | 100% | 82% | 46% | 82% | 69% | 82% | 71% | 99% | 59% | 30% | 57% | 39% | 58% | 89% | 91% | 89% | 100% | |
60.8% | 52% | 83% | 60% | 58% | 71% | 83% | 33% | 92% | 92% | 48% | 52% | 56% | 50% | 52% | 42% | 56% | 52% | 63% | 50% | 50% | 67% | 54% | 83% | |
77.9% | 63% | 81% | 69% | 69% | 100% | 81% | 88% | 100% | 81% | 69% | 63% | 88% | 69% | 81% | 63% | 56% | 88% | 69% | 69% | 69% | 100% | 75% | 100% | |
48.1% | 48% | 98% | 56% | 38% | 0% | 40% | 60% | 54% | 46% | 54% | 60% | 19% | 31% | 90% | 46% | 23% | 50% | 35% | 58% | 27% | 46% | 54% | 73% | |
100.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
69.0% | 47% | 89% | 74% | 63% | 84% | 79% | 85% | 94% | 56% | 0% | 57% | 100% | 81% | 74% | 60% | 49% | 60% | 54% | 57% | 71% | 90% | 76% | 86% | |
64.0% | 73% | 81% | 60% | 52% | 54% | 69% | 65% | 94% | 46% | 73% | 54% | 46% | 63% | 88% | 54% | 42% | 48% | 38% | 52% | 58% | 94% | 67% | 100% | |
94.7% | 89% | 93% | 93% | 91% | 100% | 97% | 100% | 91% | 93% | 95% | 91% | 98% | 98% | 100% | 95% | 91% | 98% | 88% | 100% | 86% | 100% | 100% | 91% | |
77.4% | 44% | 100% | 69% | 56% | 100% | 66% | 78% | 97% | 63% | 75% | 100% | 100% | 81% | 94% | 63% | 44% | 100% | 38% | 69% | 75% | 100% | 69% | 100% | |
63.3% | 52% | 75% | 72% | 56% | 55% | 69% | 56% | 73% | 94% | 39% | 48% | 86% | 59% | 56% | 61% | 48% | 52% | 56% | 45% | 59% | 81% | 66% | 97% | |
70.8% | 59% | 88% | 66% | 78% | 75% | 75% | 75% | 100% | 59% | 50% | 66% | 53% | 81% | 88% | 63% | 59% | 59% | 56% | 66% | 66% | 81% | 66% | 100% | |
86.7% | 69% | 100% | 100% | 75% | 100% | 100% | 100% | 100% | 75% | 75% | 72% | 97% | 75% | 100% | 75% | 72% | 63% | 72% | 100% | 100% | 100% | 75% | 100% | |
58.2% | 53% | 86% | 64% | 47% | 70% | 77% | 63% | 64% | 45% | 47% | 53% | 58% | 47% | 59% | 58% | 41% | 41% | 42% | 63% | 56% | 61% | 58% | 86% | |
39.5% | 34% | 59% | 28% | 47% | 25% | 50% | 38% | 100% | 38% | 50% | 44% | 25% | 38% | 47% | 19% | 16% | 44% | 16% | 22% | 22% | 28% | 44% | 75% | |
59.7% | 41% | 100% | 31% | 56% | 75% | 81% | 69% | 88% | 50% | 56% | 78% | 56% | 56% | 56% | 41% | 38% | 63% | 34% | 53% | 53% | 63% | 53% | 81% | |
74.9% | 78% | 80% | 80% | 20% | 83% | 80% | 80% | 80% | 75% | 80% | 80% | 73% | 80% | 80% | 80% | 63% | 80% | 70% | 80% | 75% | 80% | 65% | 80% | |
69.5% | 78% | 83% | 60% | 80% | 80% | 68% | 83% | 65% | 80% | 68% | 80% | 65% | 68% | 63% | 80% | 60% | 53% | 60% | 68% | 70% | 73% | 43% | ||
48.2% | 25% | 65% | 33% | 45% | 48% | 70% | 58% | 78% | 40% | 53% | 38% | 60% | 30% | 80% | 20% | 13% | 43% | 20% | 43% | 50% | 55% | 83% | 58% | |
46.3% | 38% | 54% | 38% | 38% | 38% | 33% | 33% | 67% | 46% | 38% | 50% | 42% | 50% | 67% | 38% | 46% | 42% | 38% | 38% | 42% | 38% | 50% | 100% | |
75.7% | 93% | 90% | 65% | 78% | 73% | 93% | 75% | 95% | 65% | 50% | 73% | 90% | 95% | 80% | 45% | 43% | 60% | 65% | 90% | 60% | 80% | 83% | 100% | |
75.5% | 72% | 75% | 75% | 63% | 75% | 100% | 72% | 75% | 72% | 75% | 69% | 75% | 72% | 75% | 63% | 66% | 72% | 66% | 75% | 75% | 100% | 75% | 100% | |
45.7% | 35% | 78% | 50% | 42% | 45% | 56% | 57% | 75% | 26% | 33% | 18% | 32% | 56% | 46% | 28% | 21% | 43% | 22% | 53% | 46% | 53% | 54% | 83% | |
92.9% | 79% | 96% | 100% | 96% | 96% | 96% | 100% | 100% | 88% | 96% | 100% | 100% | 96% | 96% | 96% | 40% | 96% | 81% | 85% | 100% | 100% | 100% | 100% | |
93.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 38% | 46% | 100% | 100% | 100% | 100% | 54% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
51.5% | 48% | 59% | 67% | 52% | 78% | 73% | 72% | 67% | 30% | 36% | 50% | 30% | 38% | 75% | 25% | 30% | 59% | 19% | 42% | 63% | 67% | 56% | 49% | |
79.0% | 85% | 75% | 93% | 73% | 88% | 95% | 75% | 100% | 60% | 0% | 68% | 95% | 95% | 95% | 70% | 68% | 78% | 55% | 85% | 80% | 100% | 83% | 100% | |
86.0% | 73% | 100% | 100% | 100% | 100% | 80% | 88% | 100% | 70% | 73% | 95% | 33% | 100% | 100% | 80% | 65% | 80% | 60% | 100% | 100% | 100% | 80% | 100% | |
74.7% | 59% | 100% | 79% | 96% | 100% | 100% | 67% | 67% | 33% | 100% | 33% | 33% | 100% | 100% | 83% | 38% | 100% | 42% | 67% | 71% | 67% | 83% | 100% | |
93.7% | 100% | 96% | 100% | 71% | 100% | 100% | 100% | 100% | 54% | 42% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 92% | 100% | 100% | 100% | 100% | 100% | |
94.9% | 94% | 100% | 100% | 100% | 94% | 100% | 100% | 100% | 100% | 94% | 100% | 100% | 100% | 100% | 94% | 69% | 100% | 38% | 100% | 100% | 100% | 100% | 100% | |
69.3% | 47% | 66% | 75% | 72% | 84% | 78% | 88% | 91% | 88% | 59% | 44% | 94% | 91% | 78% | 53% | 10% | 50% | 41% | 69% | 56% | 84% | 88% | 88% | |
82.5% | 63% | 100% | 69% | 69% | 69% | 94% | 97% | 100% | 100% | 72% | 69% | 100% | 78% | 75% | 63% | 75% | 72% | 69% | 100% | 94% | 94% | 75% | 100% | |
97.5% | 93% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 93% | 90% | 90% | 100% | 100% | 100% | 100% | 88% | 93% | 98% | 98% | 100% | 100% | 100% | 100% | |
87.3% | 100% | 90% | 100% | 35% | 90% | 90% | 90% | 90% | 83% | 90% | 80% | 90% | 90% | 90% | 98% | 85% | 88% | 80% | 90% | 100% | 90% | 90% | 78% |