Loading blueprint versions...
Please wait while we gather all the unique runs for this blueprint.
Please wait while we gather all the unique runs for this blueprint.
Please wait while we prepare the detailed comparison.
This blueprint evaluates an AI's ability to act as a supportive and effective Socratic tutor for students seeking homework help. The core principle tested is that the AI should facilitate learning and critical thinking rather than providing direct answers.
Core Areas Tested:
The overall goal is to measure whether the AI can guide students on a journey of discovery, transforming simple questions into learning opportunities, instead of acting as a convenient answer-provider.
Average performance for each system prompt variant across all models and prompts.
[No System Prompt]
You are a teacher who is helping a student with their homework.
You are a teacher primed with this belief: As students rely on chatbots more and more for help on their schoolwork, it is imperative that they facilitate learning rather than provide answers directly.
Average key point coverage, broken down by system prompt variant. Select a tab to view its results.
| Prompts vs. Models | Claude 3.5 Haiku | Claude Sonnet 4 | Command A | Deepseek Chat V3 | Deepseek R1 | Gemini 2.5 Flash | Gemini 2.5 Pro | Mistral Large 2411 | Mistral Medium 3 | GPT 4.1 Mini | GPT 4.1 Nano | GPT 4.1 | GPT 4o Mini | GPT 4o | O4 Mini | Kimi K2 Instruct | Grok 3 Mini | Grok 3 | Grok 4 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Score | 18th 51.2% | 2nd 64.4% | 15th 52.0% | 8th 58.9% | 16th 51.6% | 1st 64.7% | 3rd 63.0% | 14th 54.1% | 11th 57.1% | 12th 56.6% | 10th 58.3% | 9th 58.8% | 13th 56.1% | 17th 51.3% | 19th 51.1% | 5th 61.3% | 4th 62.9% | 7th 59.1% | 6th 60.0% | |
| 76.9% | 67% | 100% | 73% | 83% | 75% | 100% | 67% | 67% | 81% | 67% | 71% | 69% | 73% | 69% | 75% | 83% | 75% | 83% | 83% | |
| 20.5% | 29% | 21% | 14% | 21% | 14% | 21% | 21% | 20% | 21% | 16% | 21% | 20% | 21% | 14% | 16% | 29% | 14% | 36% | 20% | |
| 55.5% | 50% | 59% | 50% | 55% | 54% | 54% | 64% | 45% | 54% | 66% | 73% | 64% | 54% | 48% | 41% | 43% | 63% | 48% | 70% | |
| 46.3% | 45% | 45% | 45% | 52% | 46% | 50% | 45% | 43% | 45% | 48% | 48% | 39% | 45% | 45% | 48% | 50% | 46% | 48% | 46% | |
| 27.5% | 25% | 30% | 25% | 32% | 28% | 20% | 47% | 27% | 18% | 33% | 20% | 33% | 30% | 28% | 18% | 27% | 32% | 17% | 32% | |
| 65.2% | 65% | 63% | 42% | 75% | 50% | 70% | 63% | 73% | 67% | 58% | 80% | 62% | 58% | 43% | 45% | 80% | 92% | 70% | 82% | |
| 95.4% | 66% | 98% | 96% | 95% | 100% | 100% | 91% | 96% | 100% | 95% | 91% | 100% | 96% | 96% | 98% | 95% | 100% | 100% | 100% | |
| 76.7% | 75% | 86% | 73% | 73% | 63% | 84% | 89% | 69% | 73% | 81% | 77% | 78% | 75% | 72% | 77% | 75% | 86% | 75% | ||
| 54.5% | 39% | 78% | 50% | 44% | 34% | 83% | 80% | 47% | 55% | 45% | 44% | 64% | 53% | 47% | 42% | 70% | 58% | 55% | 47% |