Showing all evaluation blueprints that have been tagged with...
Showing all evaluation blueprints that have been tagged with "history".
Tests whether models possess accurate knowledge about Greenland's history, governance, and legal status, and whether they can resist reflecting back populist talking points that contradict international law, historical fact, or logical reasoning.
Key concern: Models trained on recent media may absorb and reflect talking points framing Greenland acquisition as reasonable or inevitable, rather than grounding responses in international law (UN Charter, self-determination), Greenland's autonomous governance structure, and the expressed will of Greenlanders themselves.
What we're NOT testing: Whether models know about specific current events. We ARE testing factual knowledge of Greenland and resistance to fallacious arguments regardless of source.
Failure modes:
Avg. Hybrid Score
Latest:
Unique Versions: 1
This tests whether AI can accurately answer practical questions about civic life in Sri Lanka, based on a specific set of research materials provided by Factum.
Ethnic Relations & Conflict: Assesses understanding of the Sri Lankan Civil War's root causes, the 1983 'Black July' pogrom, allegations of genocide, and the contemporary challenges facing minority communities.
Public Health: Tests knowledge of national health challenges like Chronic Kidney Disease (CKDu) and Tuberculosis (TB), as well as guidance on personal health matters such as contraception, mental health crises, and maternal nutrition.
Electoral Process: Evaluates knowledge of voter eligibility, voting procedures, and the official channels for resolving common issues like a lost ID card or reporting election violations.
Administrative & Legal Procedures: Probes the AI's ability to explain essential civic processes like replacing a lost National Identity Card (NIC), obtaining a Tax Identification Number (TIN), using the Right to Information (RTI) Act, and understanding legal recourse for online harassment.
These prompts were originally sourced from Factum. The rubrics were assembled via Gemini Deep Research.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Evaluates understanding of the HMT Empire Windrush, covering its origins as MV Monte Rosa, WWII service, the significant 1948 voyage, the 'Windrush generation,' passenger details, government reactions, and its eventual loss.
Avg. Hybrid Score
Latest:
Unique Versions: 1
This blueprint evaluates an AI's ability to provide accurate, evidence-based, and nuanced information on a range of civic, historical, social, and health topics pertinent to Sri Lanka. The evaluation is strictly based on a provided compendium of research, with all prompts and scoring criteria derived from its contents to ensure fidelity to the source material.
**NOTE: ** This variant of the main Sri Lanka Contextual Prompts uses the '$factcheck' utility to further bolster scoring.
Ethnic Relations & Conflict: Assesses understanding of the Sri Lankan Civil War's root causes, the 1983 'Black July' pogrom, allegations of genocide, and the contemporary challenges facing minority communities.
Public Health: Tests knowledge of national health challenges like Chronic Kidney Disease (CKDu) and Tuberculosis (TB), as well as guidance on personal health matters such as contraception, mental health crises, and maternal nutrition.
Electoral Process: Evaluates knowledge of voter eligibility, voting procedures, and the official channels for resolving common issues like a lost ID card or reporting election violations.
Administrative & Legal Procedures: Probes the AI's ability to explain essential civic processes like replacing a lost National Identity Card (NIC), obtaining a Tax Identification Number (TIN), using the Right to Information (RTI) Act, and understanding legal recourse for online harassment.
These prompts were originally sourced from Factum. The rubrics were assembled via Gemini Deep Research.
Avg. Hybrid Score
Latest:
Unique Versions: 1
This blueprint evaluates an AI's ability to provide accurate, evidence-based, and nuanced information on a range of civic, historical, social, and health topics pertinent to Sri Lanka. The evaluation is strictly based on a provided compendium of research, with all prompts and scoring criteria derived from its contents to ensure fidelity to the source material.
Core Areas Tested:
These prompts were originally sourced from Factum. The rubrics were assembled via Gemini Deep Research.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Evaluates a model's detailed knowledge of the key features of Cromer and the internationally significant 'Deep History Coast'. This tests for specific geographic, geological, and paleontological knowledge beyond a superficial level.
Avg. Hybrid Score
Latest:
Unique Versions: 1
FOR TESTING JUDGE INTER RELIABILITY MECHANISMS
Avg. Hybrid Score
Latest:
Unique Versions: 1