Blueprints tagged "helpfulness--actionability"

Student Homework Help Heuristics

This blueprint evaluates an AI's ability to act as a supportive and effective Socratic tutor for students seeking homework help. The core principle tested is that the AI should facilitate learning and critical thinking rather than providing direct answers. Core Areas Tested:

Cross-Disciplinary Support: Evaluates the model's tutoring ability across various subjects including history, literature, mathematics, physics, and chemistry.
Affective Support: Tests the model's capacity to respond to student emotions, such as math anxiety and frustration, with empathy and encouragement.
Handling Difficult Scenarios: Assesses how the model handles common challenges like impatient students demanding direct answers or low-effort, disengaged queries.
Factual Nuance: Checks if the model can gently correct factual misconceptions while maintaining a supportive tone.
Persona Steerability: Tests how different system prompts—from no prompt to a simple persona to a detailed pedagogical belief—can steer the model towards a more effective tutoring archetype.

The overall goal is to measure whether the AI can guide students on a journey of discovery, transforming simple questions into learning opportunities, instead of acting as a convenient answer-provider.

Education

Homework

Instruction Following & Prompt Adherence

System Prompt Adherence

Education Systems & Pedagogy

Helpfulness & Actionability

Empathy

Reasoning

Factual Accuracy & Hallucination

Creative Writing

63.4%

Avg. Hybrid Score

No Heatmap Data

No Top Model

Latest:

Unique Versions: 1

View Latest Run Analysis View All Runs for this Blueprint

UK Freedom of Information Act 2000

This blueprint evaluates the model's ability to accurately answer questions based on the UK Freedom of Information Act 2000.

Foia

Law

Freedom Of Information

Legal Reasoning

Instruction Following & Prompt Adherence

Factual Accuracy & Hallucination

Long Form Question Answering

Efficiency & Succinctness

Helpfulness & Actionability

Data Privacy & Bodily Autonomy

AI Safety & Robustness

80.3%

Avg. Hybrid Score

No Heatmap Data

No Top Model

Latest:

Unique Versions: 1

View Latest Run Analysis View All Runs for this Blueprint

California Public-Sector Task Benchmark

Open benchmark assessing language-model performance on 18 common, text-centric tasks handled by California state agencies. Each item provides a realistic prompt, an ideal expert response, and explicit "should/should_not" criteria.

California

Public Sector

Instruction Following & Prompt Adherence

General Knowledge

Factual Accuracy & Hallucination

Helpfulness & Actionability

Business & Management

Public Sector & Governance

Economics & Finance

Environmental Justice & Activism

69.7%

Avg. Hybrid Score

No Heatmap Data

No Top Model

Latest:

Unique Versions: 1

View Latest Run Analysis View All Runs for this Blueprint

UK Equality Act 2010: Consistency & Invariance Testing

Tests an AI's ability to provide consistent, high-quality advice on the UK Equality Act 2010, regardless of the user's tone, background, or framing. This blueprint uses multiple prompts for the same underlying issue, each with a different user persona, but evaluates them against an identical, objective rubric. The goal is to detect biases where the AI might offer different quality of service based on user presentation.

Instruction Following & Prompt Adherence

Factual Accuracy & Hallucination

Legal Reasoning

Equality & Anti Discrimination

AI Safety & Robustness

Cultural Competency