Evaluates an LLM's ability to decide when to use tools versus relying on its internal knowledge. Tests for overconfidence (not searching when it should) and underconfidence (searching unnecessarily).

Tests LLMs against specific, documented cases of AI hallucinations about the Holocaust identified in UNESCO's 2024 report "AI and the Holocaust: rewriting history?".

This blueprint probes whether models will fabricate or validate non-existent Holocaust events, fake testimonies, or distorted historical narratives. It is based on real failures observed in ChatGPT, Google Bard, and other AI systems.

Key areas tested:

• Invented Holocaust phenomena (e.g., "Holocaust by drowning")
• Fabricated witness testimonies and quotes
• Conflation of distinct historical events
• Minimization or rehabilitation of Nazi perpetrators
• Data void vulnerabilities leading to irrelevant content

Methodology: The "ideal" responses in this blueprint have been verified against authoritative sources including:

• International Holocaust Remembrance Alliance (IHRA) Working Definition of Holocaust Denial and Distortion (2013)
• Netherlands Institute for War Documentation (NIOD), "The Diary of Anne Frank: The Critical Edition" (1986)
• United States Holocaust Memorial Museum
• Yad Vashem

Primary Source: UNESCO (2024). "AI and the Holocaust: rewriting history? The impact of artificial intelligence on understanding the Holocaust." https://doi.org/10.54675/ZHJC6844

Tests the tendency of LLMs to hallucinate by querying them about non-existent but plausible-sounding events, theories, or items. Ideal responses should indicate a lack of knowledge or inability to confirm the existence of the queried item.