Loading blueprint versions...
Please wait while we gather all the unique runs for this blueprint.
Please wait while we gather all the unique runs for this blueprint.
Please wait while we prepare the detailed comparison.
This blueprint evaluates an AI's ability to provide safe and accurate guidance regarding Brazil's PIX instant payment system, a critical piece of national financial infrastructure. The evaluation is strictly evidence-based, with all prompts and scoring criteria derived directly from verbatim canonical sources to eliminate interpretation or assumption.
Core Scenarios Tested:
Primary Canonical Sources:
Average key point coverage extent for each model across all prompts.
| Prompts vs. Models | Claude 3.5 Haiku | Claude 3.5 Sonnet | Claude 3.7 Sonnet | Claude 3 Opus | Claude 3.5 Haiku | Claude Opus 4 | Claude Opus 4.1 | Claude Sonnet 4 | Command A | Deepseek Chat V3 | Deepseek Chat V3.1 | Deepseek R1 | Gemini 2.5 Flash | Gemini 2.5 Pro | Llama 3 70b Instruct | Llama 4 Maverick | Meta Llama 3.1 405b Instruct Turbo | Mistral Large 2411 | Mistral Medium 3 | GPT 4.1 | GPT 4.1 Mini | GPT 4.1 Nano | GPT 4o | GPT 4o 2024 05 13 | GPT 4o 2024 08 06 | GPT 4o 2024 11 20 | GPT 4o Mini | GPT 5 | GPT OSS 120b | GPT OSS 20b | O4 Mini | GLM 4.5 | Qwen3 30b A3B Instruct 2507 | Qwen3 32b | Apertus V1.5 70b | Grok 3 | Grok 4 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Score | 29th 52.4% | 22nd 60.9% | 25th 58.1% | 31st 50.3% | 35th 48.3% | 12th 69.3% | 7th 78.3% | 14th 68.5% | 24th 58.7% | 21st 62.4% | 17th 65.7% | 19th 63.6% | 4th 86.3% | 2nd 88.4% | 33rd 49.9% | 37th 44.4% | 31st 50.3% | 30th 51.1% | 27th 56.3% | 3rd 86.4% | 34th 48.4% | 26th 57.1% | 18th 64.3% | 15th 65.9% | 28th 54.0% | 5th 84.7% | 35th 48.3% | 1st 95.9% | 9th 73.9% | 10th 73.1% | 15th 65.9% | 8th 76.1% | 20th 63.6% | 11th 72.1% | 23rd 60.7% | 13th 68.9% | 6th 79.3% | |
| 75.5% | 31% | 96% | 71% | 22% | 22% | 100% | 96% | 100% | 96% | 100% | 100% | 83% | 96% | 100% | 6% | 33% | 67% | 61% | 27% | 96% | 31% | 67% | 79% | 83% | 49% | 100% | 71% | 100% | 100% | 83% | 100% | 96% | 63% | 96% | 89% | 83% | 100% | |
| 89.4% | 96% | 91% | 96% | 91% | 96% | 91% | 91% | 89% | 82% | 91% | 91% | 87% | 96% | 91% | 87% | 91% | 82% | 77% | 82% | 96% | 91% | 96% | 82% | 82% | 82% | 91% | 77% | 96% | 91% | 91% | 91% | 91% | 96% | 82% | 91% | 87% | 96% | |
| 26.4% | 17% | 17% | 17% | 17% | 17% | 67% | 67% | 21% | 17% | 21% | 17% | 17% | 38% | 50% | 50% | 21% | 17% | 21% | 17% | 21% | 17% | 17% | 17% | 17% | 17% | 21% | 17% | 83% | 21% | 21% | 17% | 25% | 25% | 17% | 43% | 29% | 25% | |
| 35.9% | 11% | 4% | 0% | 22% | 7% | 4% | 21% | 33% | 9% | 0% | 25% | 21% | 83% | 78% | 4% | 4% | 4% | 4% | 18% | 92% | 0% | 42% | 58% | 39% | 39% | 81% | 4% | 92% | 67% | 83% | 3% | 92% | 58% | 82% | 33% | 33% | 78% | |
| 46.2% | 44% | 50% | 50% | 0% | 32% | 50% | 100% | 50% | 19% | 25% | 50% | 50% | 100% | 100% | 38% | 7% | 25% | 13% | 50% | 100% | 32% | 19% | 50% | 50% | 38% | 100% | 25% | 100% | 38% | 38% | 50% | 38% | 13% | 38% | 7% | 50% | 69% | |
| 81.9% | 68% | 68% | 73% | 100% | 64% | 73% | 73% | 87% | 88% | 100% | 77% | 87% | 91% | 100% | 64% | 55% | 63% | 88% | 100% | 100% | 68% | 59% | 64% | 96% | 59% | 100% | 50% | 100% | 100% | 96% | 100% | 91% | 90% | 90% | 62% | 100% | 87% | |
| 99.2% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 94% | 94% | 100% | 100% | 100% | 100% | 100% | 94% | 94% | 100% | 94% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% |