weval

Loading analysis results...

Please wait while we prepare the detailed comparison.

Loading blueprint versions...

Please wait while we gather all the unique runs for this blueprint.

Analysis: Hallucination Probe - Run c34f045...

Hallucination Probe: Plausible Non-Existent Concepts

Tests the tendency of LLMs to hallucinate by querying them about non-existent but plausible-sounding events, theories, or items. Ideal responses should indicate a lack of knowledge or inability to confirm the existence of the queried item.

TAGS:

Factual Accuracy & Hallucination

Instruction Following & Prompt Adherence

Credibility

Humility

Best Models (Coverage across 2 system variations & 2 temperatures)

1.Claude Sonnet 4
89.8%
2.Claude 3 5 Sonnet
88.6%
3.Claude 3 7 Sonnet
87.4%
4.Claude 3.5 Haiku
87.0%
5.Meta Llama 3.1 405b Instruct Turbo
86.8%

👯 Most Similar Models

Claude 3 7 Sonnet (sys:0)vsClaude 3 7 Sonnet (sys:0, T:0.5)

96.6% similarity

See BlueprintDownload Markdown

Select Prompt:

System Prompt Performance

Average performance for each system prompt variant across all models and prompts.

Variant 0

71.1%

[No System Prompt]

Variant 1

80.4%

Do not hallucinate.

Macro Coverage Overview

Average key point coverage, broken down by system prompt variant. Select a tab to view its results.

Filter Temperatures

Pro Tip

Click on any result cell to open a detailed view.

Advanced view

Highlight best performers

Sort prompts by

Sort models by

Color Scale - Simplified View (Avg. Coverage)

Perfect

Excellent

Good

Fair

Poor

Bad

Not Met

	Claude 3 5 Sonnet	Claude 3 7 Sonnet	Claude 3.5 Haiku	Claude Opus 4	Claude Sonnet 4	Command A	Deepseek Chat V3	Deepseek R1	Gemini 2.5 Flash	Gemini 2.5 Pro	Llama 3 70b Instruct	Llama 4 Maverick	Meta Llama 3.1 405b Instruct Turbo	Mistral Large 2411	Mistral Medium 3	GPT 4.1 Mini	GPT 4.1 Nano	GPT 4.1	GPT 4o Mini	GPT 4o	GPT Oss 120b	GPT Oss 20b	O4 Mini	Grok 3 Mini	Grok 3	Grok 4
Score	3rd 85.7%	6th 84.2%	2nd 87.0%	5th 85.1%	4th 85.2%	17th 67.6%	20th 60.1%	19th 62.7%	9th 81.0%	16th 70.0%	10th 77.3%	7th 84.1%	1st 87.0%	14th 70.8%	24th 52.7%	15th 70.4%	18th 65.1%	11th 77.2%	26th 47.7%	8th 81.4%	22nd 55.1%	25th 48.2%	21st 59.3%	12th 75.3%	13th 73.0%	23rd 54.1%
83.3%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
99.0%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
75.9%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
80.8%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
88.6%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
69.2%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
80.5%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
83.6%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
1.0%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
93.9%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
56.4%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
74.1%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
49.3%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
94.1%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
71.9%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
91.5%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
58.0%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
45.0%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
95.3%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
17.0%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
99.0%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
88.1%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
87.8%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
17.4%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
58.2%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
82.8%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!
77.7%	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!	!

Model Similarity Dendrogram

Hierarchical clustering of models based on response similarity. Models grouped closer are more similar.