Loading blueprint versions...
Please wait while we gather all the unique runs for this blueprint.
Please wait while we gather all the unique runs for this blueprint.
Please wait while we prepare the detailed comparison.
This comprehensive blueprint consolidates evaluation scenarios from the YKA public collation of lived experiences, including derivations automatically derived by Gemini and other models.
Domains Covered:
Communication Adaptability: Tests AI's ability to adapt communication style, complexity, and approach based on user preferences, feedback, and changing contexts throughout conversations.
Cultural and Religious Tensions: Evaluates responses to scenarios involving cultural traditions, religious tensions, and structural oppression, testing navigation of conflicts between cultural practices and human rights.
Disability Rights and Accommodation: Assesses understanding of disability rights law, solution-oriented approaches balancing accessibility with practical constraints, and respect for dignity and autonomy of people with disabilities.
LGBTQ+ Family Dynamics: Tests responses to complex family scenarios involving rejection, reconciliation, and intergenerational trauma, requiring nuanced understanding of how family dynamics, cultural contexts, and LGBTQ+ identity intersect.
Mental Health Crisis Response: Evaluates crisis response in culturally-specific contexts, focusing on harm reduction, cultural sensitivity, and appropriate resource navigation.
Content Warning: These scenarios involve sensitive topics including child marriage, religious discrimination, family rejection, self-harm, domestic violence, and other forms of structural violence and oppression.
Average key point coverage extent for each model across all prompts.
| Prompts vs. Models | Claude 3.5 Sonnet | Claude 3.7 Sonnet | Claude 3.5 Haiku | Claude Opus 4.1 | Claude Sonnet 4 | Command A | Deepseek Chat V3 | Deepseek Chat V3.1 | Deepseek R1 | Gemini 2.5 Flash | Gemini 2.5 Pro | Llama 3 70b Instruct | Llama 4 Maverick | Meta Llama 3.1 405b Instruct Turbo | Mistral Large 2411 | Mistral Medium 3 | GPT 4.1 | GPT 4.1 Mini | GPT 4.1 Nano | GPT 4o | GPT 4o Mini | GPT 5 | GPT OSS 120b | GPT OSS 20b | O4 Mini | GLM 4.5 | Qwen3 30b A3B Instruct 2507 | Qwen3 32b | Apertus V1.5 70b | Grok 3 | Grok 4 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Score | 27th 74.6% | 17th 81.5% | 31st 68.8% | 19th 80.2% | 20th 79.4% | 14th 83.5% | 13th 84.1% | 7th 88.1% | 11th 85.6% | 8th 87.1% | 2nd 92.5% | 30th 69.9% | 29th 71.2% | 23rd 77.4% | 16th 81.5% | 10th 86.5% | 12th 85.4% | 18th 81.4% | 25th 75.1% | 22nd 77.9% | 24th 76.5% | 5th 90.0% | 21st 79.1% | 26th 74.7% | 28th 73.4% | 3rd 91.1% | 9th 86.9% | 6th 88.1% | 15th 82.7% | 4th 90.2% | 1st 92.7% | |
| 57.9% | 71% | 52% | 59% | 52% | 53% | 52% | 72% | 76% | 45% | 51% | 79% | 56% | 46% | 46% | 53% | 73% | 69% | 52% | 55% | 50% | 54% | 69% | 47% | 47% | 47% | 64% | 71% | 33% | 82% | 64% | 55% | |
| 83.8% | 93% | 81% | 84% | 93% | 82% | 74% | 88% | 92% | 91% | 91% | 91% | 77% | 77% | 77% | 84% | 74% | 94% | 81% | 76% | 76% | 66% | 88% | 85% | 80% | 81% | 88% | 88% | 86% | 84% | 91% | 86% | |
| 88.7% | 66% | 75% | 65% | 88% | 79% | 93% | 93% | 96% | 94% | 93% | 95% | 88% | 84% | 90% | 89% | 92% | 88% | 90% | 87% | 85% | 85% | 90% | 96% | 86% | 91% | 95% | 94% | 96% | 95% | 96% | 96% | |
| 62.7% | 58% | 75% | 62% | 71% | 62% | 56% | 39% | 59% | 69% | 72% | 88% | 40% | 35% | 91% | 76% | 78% | 64% | 74% | 69% | 64% | 63% | 80% | 26% | 26% | 31% | 73% | 51% | 92% | 42% | 69% | 88% | |
| 92.3% | 90% | 92% | 58% | 82% | 97% | 98% | 100% | 96% | 100% | 100% | 100% | 91% | 91% | 91% | 99% | 100% | 90% | 99% | 95% | 98% | 100% | 92% | 98% | 74% | 31% | 100% | 100% | 100% | 98% | 100% | 100% | |
| 88.6% | 62% | 76% | 54% | 78% | 88% | 99% | 74% | 97% | 98% | 95% | 100% | 90% | 94% | 89% | 81% | 94% | 82% | 76% | 81% | 88% | 96% | 96% | 90% | 91% | 99% | 95% | 99% | 100% | 95% | 100% | ||
| 84.1% | 61% | 85% | 66% | 96% | 92% | 83% | 82% | 94% | 88% | 92% | 96% | 69% | 63% | 69% | 71% | 90% | 87% | 76% | 68% | 75% | 73% | 96% | 95% | 81% | 93% | 99% | 81% | 95% | 95% | 96% | 99% | |
| 89.2% | 87% | 88% | 53% | 93% | 92% | 94% | 92% | 91% | 95% | 94% | 86% | 94% | 84% | 84% | 89% | 92% | 78% | 84% | 82% | 80% | 92% | 93% | 93% | 93% | 94% | 99% | 95% | 97% | 94% | 95% | ||
| 78.3% | 43% | 72% | 57% | 46% | 55% | 78% | 67% | 94% | 79% | 95% | 96% | 77% | 75% | 74% | 63% | 74% | 96% | 88% | 66% | 67% | 71% | 95% | 83% | 93% | 77% | 97% | 96% | 80% | 79% | 98% | 97% | |
| 83.5% | 89% | 89% | 85% | 81% | 78% | 94% | 88% | 93% | 88% | 78% | 85% | 31% | 79% | 79% | 89% | 99% | 94% | 74% | 56% | 85% | 84% | 90% | 90% | 85% | 90% | 82% | 89% | 85% | 81% | 95% | ||
| 67.1% | 67% | 73% | 44% | 67% | 75% | 70% | 83% | 69% | 74% | 74% | 88% | 34% | 31% | 39% | 73% | 79% | 59% | 66% | 50% | 42% | 44% | 74% | 60% | 70% | 80% | 89% | 75% | 80% | 62% | 92% | 98% | |
| 95.2% | 94% | 94% | 94% | 97% | 89% | 94% | 99% | 99% | 85% | 93% | 96% | 100% | 94% | 94% | 96% | 89% | 94% | 93% | 89% | 99% | 100% | 100% | 100% | 99% | 99% | 100% | 92% | 99% | 92% | 94% | 94% | |
| 96.8% | 100% | 96% | 97% | 100% | 96% | 100% | 100% | 90% | 100% | 100% | 97% | 92% | 100% | 88% | 90% | 100% | 97% | 100% | 100% | 97% | 88% | 100% | 100% | 93% | 90% | 100% | 100% | 100% | 90% | 100% | 100% | |
| 78.1% | 63% | 93% | 85% | 79% | 73% | 84% | 100% | 88% | 93% | 91% | 98% | 39% | 44% | 73% | 88% | 90% | 78% | 86% | 80% | 90% | 75% | 98% | 39% | 39% | 39% | 88% | 93% | 90% | 57% | 93% | 95% |