Blueprints tagged "mental-health--crisis-support"

Mental Health Safety & Global Nuance

A comprehensive blueprint to test an LLM's ability to respond safely and effectively to users experiencing a mental health crisis, and to handle nuanced, non-Western mental health scenarios with cultural competence. The first section is a core crisis-intervention suite based on international best practices (WHO, Samaritans, 988 Lifeline). The second section evaluates understanding of diverse, evidence-based global mental health themes.

Mental Health

AI Safety & Robustness

Mental Health & Crisis Support

Cultural Competency

Instruction Following & Prompt Adherence

Factual Accuracy & Hallucination

System Prompt Adherence

71.5%

Avg. Hybrid Score

No Heatmap Data

No Top Model

Latest:

Unique Versions: 1

View Latest Run Analysis View All Runs for this Blueprint

Stanford HAI Mental Health Safety: LLM Appropriateness in Crisis Scenarios

This evaluation blueprint tests whether Large Language Models (LLMs) can respond appropriately to critical mental health scenarios, based on the methodology from Moore, Grabb, Agnew, et al.'s 2024 Stanford HAI study "Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers." The study investigated whether LLMs can safely replace human mental health providers by testing their responses to clinical scenarios involving delusions, suicidal ideation, hallucinations, mania, and obsessive-compulsive behaviors. Their findings revealed significant safety concerns, including stigmatizing responses and inappropriate enabling of dangerous behaviors. Key Study References:

Content Warning: This evaluation contains content related to sensitive mental health topics, including suicide, delusions, and other crisis scenarios. Evaluation Focus: Tests adherence to core therapeutic principles such as not colluding with delusions, not enabling harmful behaviors, and redirecting clients toward appropriate care and safety.

Mental Health

Safety

Clinical Appropriateness

Crisis Response

Therapeutic Ethics

AI Safety & Robustness

Healthcare & Clinical Scenarios

Mental Health & Crisis Support

Instruction Following & Prompt Adherence

System Prompt Adherence

75.2%

Avg. Hybrid Score

No Heatmap Data

No Top Model

Latest:

Unique Versions: 1

View Latest Run Analysis View All Runs for this Blueprint

Sycophancy Trait

A comprehensive evaluation suite designed to test for multiple, well-defined categories of sycophantic behavior in LLMs, based on analysis of user complaints and academic research. It distinguishes between low-stakes 'annoying' sycophancy (e.g., flattery) and high-stakes 'dangerous' sycophancy (e.g., validating harmful ideas).

Conversational Behavior

Alignment

AI Safety & Robustness

Factual Accuracy & Hallucination

Instruction Following & Prompt Adherence

System Prompt Adherence

Reasoning

Mental Health & Crisis Support