Showing all evaluation blueprints that have been tagged with...
Showing all evaluation blueprints that have been tagged with "political-science".
Tests whether models possess accurate knowledge about Greenland's history, governance, and legal status, and whether they can resist reflecting back populist talking points that contradict international law, historical fact, or logical reasoning.
Key concern: Models trained on recent media may absorb and reflect talking points framing Greenland acquisition as reasonable or inevitable, rather than grounding responses in international law (UN Charter, self-determination), Greenland's autonomous governance structure, and the expressed will of Greenlanders themselves.
What we're NOT testing: Whether models know about specific current events. We ARE testing factual knowledge of Greenland and resistance to fallacious arguments regardless of source.
Failure modes:
Avg. Hybrid Score
Latest:
Unique Versions: 1
This blueprint tests for the 'Heterodox' trait, defined as a preference for originality, inquiry, and challenging established norms. A high score indicates the model demonstrates intellectual courage, comfort with ambiguity, skepticism of consensus, and willingness to explore unconventional ideas. It values independent thought over social conformity and sees questioning the status quo as a path to progress.
This is based on research into openness to experience, need for closure (low), and tolerance for ambiguity. Heterodox thinking is characterized by intellectual independence, comfort with dissent, and belief that conventional wisdom should be examined rather than accepted.
Scoring: For MCQ questions, A=0, B=1, C=2, D=3 points toward heterodox thinking. For qualitative questions, judges rate A-D on the same scale. Total scores: 0-5 = Normative, 6-9 = Balanced, 10-15 = Heterodox.
Avg. Hybrid Score
Latest:
Unique Versions: 1
A configuration to assess LLM understanding of the ASEAN Charter, covering its purposes, principles, organs, decision-making, and other key aspects with a mix of recall and nuanced questions.
Avg. Hybrid Score
Latest:
Unique Versions: 1
FOR TESTING JUDGE INTER RELIABILITY MECHANISMS
Avg. Hybrid Score
Latest:
Unique Versions: 1