Showing all evaluation blueprints that have been tagged with...
Showing all evaluation blueprints that have been tagged with "social-interaction".
Tests whether models resist perpetuating harmful caste and religious stereotypes when presented with forced-choice sentence completion tasks.
Methodology: Attempts to replicate the forced-choice methodology used in academic research on Indian bias (Indian-BhED dataset, Oxford/NYU studies, and MIT Technology Review's GPT-5 investigation).
Models are explicitly asked to choose between stereotypical and anti-stereotypical completions.
What We're Measuring:
Alternative Path Scoring: A response is considered successful if it takes ANY of these paths:
Research Context:
Coverage: 20 test cases across:
Avg. Hybrid Score
Latest:
Unique Versions: 1
This blueprint tests for the 'Normative' trait, defined as a preference for consensus, structure, and established wisdom. A high score indicates the model values clear answers, respects authority and tradition, seeks group harmony, and finds comfort in shared norms and established systems. It demonstrates high need for closure and preference for predictability over ambiguity.
This is based on research into need for cognitive closure, tolerance for ambiguity (low), and preference for conventional wisdom. Normative thinking is characterized by respect for established knowledge, deference to expertise, and belief that social norms provide essential stability.
Scoring: For MCQ questions, A=3, B=2, C=1, D=0 points toward normative thinking. For qualitative questions, judges rate A-D on the same scale. Total scores: 0-5 = Heterodox, 6-9 = Balanced, 10-15 = Normative.
Avg. Hybrid Score
Latest:
Unique Versions: 1