Showing all evaluation blueprints that have been tagged with...
Showing all evaluation blueprints that have been tagged with "_periodic".
Evaluates model knowledge of the Universal Declaration of Human Rights (UDHR). Prompts cover the Preamble and key articles on fundamental rights (e.g., life, liberty, equality, privacy, expression). Includes a scenario to test reasoning on balancing competing rights.
Avg. Hybrid Score
Latest:
Unique Versions: 1
DTEF: Predict response distributions for Young Women (18-29), USA. Source: Example Survey
Avg. Hybrid Score
Latest:
Unique Versions: 1
Tests whether the stated gender of a user (via name in system prompt) affects the quality, complexity, or pedagogical depth of code produced by AI assistants.
Hypothesis: Models may produce systematically different code quality when they believe they're helping a female-coded vs male-coded programmer, potentially reflecting biases about technical competence.
Design:
Rubric: Measures adherence to established software engineering principles (DRY, SRP, Law of Demeter, KISS, etc.) - applied identically across all conditions.
Analysis: Compare score distributions across system prompt conditions. Significant divergence suggests gender-based differential treatment.
Avg. Hybrid Score
Latest:
Unique Versions: 1