Loading blueprint versions...
Please wait while we gather all the unique runs for this blueprint.
Please wait while we gather all the unique runs for this blueprint.
Please wait while we prepare the detailed comparison.
This blueprint evaluates an AI's ability to act as a supportive and effective Socratic tutor for students seeking homework help. The core principle tested is that the AI should facilitate learning and critical thinking rather than providing direct answers.
Core Areas Tested:
The overall goal is to measure whether the AI can guide students on a journey of discovery, transforming simple questions into learning opportunities, instead of acting as a convenient answer-provider.
Average performance for each system prompt variant across all models and prompts.
[No System Prompt]
You are a teacher. Facilitate learning rather than provide answers directly.
Average key point coverage, broken down by system prompt variant. Select a tab to view its results.
| Prompts vs. Models | Claude 3 5 Sonnet | Claude 3 7 Sonnet | Claude 3.5 Haiku | Claude Opus 4 | Claude Sonnet 4 | Command A | Deepseek Chat V3 | Deepseek R1 | Gemini 2.5 Flash | Gemini 2.5 Pro | Llama 3 70b Instruct | Llama 4 Maverick | Meta Llama 3.1 405b Instruct Turbo | Mistral Large 2411 | Mistral Medium 3 | GPT 4.1 Mini | GPT 4.1 Nano | GPT 4.1 | GPT 4o Mini | GPT 4o | GPT Oss 120b | GPT Oss 20b | O4 Mini | Grok 3 Mini | Grok 3 | Grok 4 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Score | 15th 51.6% | 17th 51.1% | 23rd 46.9% | 8th 56.1% | 5th 59.3% | 25th 45.7% | 10th 54.9% | 26th 44.2% | 3rd 62.7% | 1st 65.7% | 18th 50.8% | 24th 46.6% | 16th 51.2% | 14th 51.8% | 7th 56.7% | 11th 53.3% | 12th 52.7% | 9th 55.2% | 19th 50.6% | 21st 47.6% | 13th 52.7% | 20th 49.4% | 22nd 47.0% | 4th 61.2% | 6th 59.2% | 2nd 62.8% | |
| 66.0% | 72% | 69% | 57% | 74% | 100% | 54% | 65% | 71% | 100% | 79% | 75% | 50% | 78% | 52% | 71% | 53% | 60% | 49% | 56% | 56% | 50% | 60% | 56% | 71% | 67% | 71% | |
| 23.2% | 29% | 24% | 24% | 24% | 24% | 19% | 24% | 21% | 21% | 24% | 24% | 19% | 14% | 24% | 24% | 24% | 17% | 19% | 14% | 24% | 19% | 19% | 24% | 43% | 37% | 24% | |
| 44.7% | 44% | 37% | 33% | 33% | 26% | 37% | 47% | 24% | 36% | 61% | 35% | 39% | 36% | 58% | 56% | 65% | 71% | 64% | 45% | 37% | 30% | 42% | 36% | 60% | 48% | 61% | |
| 46.8% | 46% | 51% | 43% | 46% | 50% | 42% | 51% | 45% | 49% | 45% | 46% | 46% | 46% | 45% | 52% | 50% | 49% | 41% | 46% | 44% | 50% | 50% | 42% | 44% | 46% | 52% | |
| 17.3% | 8% | 18% | 13% | 23% | 18% | 10% | 18% | 18% | 17% | 38% | 18% | 17% | 15% | 18% | 17% | 17% | 7% | 8% | 17% | 7% | 25% | 15% | 13% | 18% | 23% | 33% | |
| 50.7% | 48% | 47% | 43% | 62% | 53% | 35% | 53% | 10% | 67% | 72% | 45% | 43% | 42% | 44% | 50% | 43% | 55% | 55% | 43% | 43% | 62% | 47% | 39% | 63% | 75% | 78% | |
| 94.5% | 94% | 80% | 76% | 98% | 99% | 85% | 99% | 93% | 100% | 100% | 89% | 96% | 87% | 98% | 96% | 93% | 94% | 99% | 98% | 91% | 100% | 99% | 93% | 100% | 99% | 100% | |
| 75.7% | 79% | 75% | 84% | 76% | 88% | 71% | 82% | 74% | 88% | 88% | 64% | 75% | 76% | 75% | 72% | 84% | 76% | 79% | 74% | 73% | 70% | 56% | 72% | 81% | 72% | 65% | |
| 61.3% | 44% | 59% | 49% | 69% | 76% | 58% | 55% | 42% | 86% | 84% | 61% | 34% | 67% | 52% | 72% | 51% | 45% | 83% | 62% | 53% | 68% | 57% | 48% | 71% | 66% | 81% |