anthropic/claude-3.5-haiku consistently struggled with adhering to the persona defined by the system prompt, often ignoring it or applying it inconsistently, especially at higher temperatures. openai/gpt-4.1-mini demonstrated superior instruction following, particularly in maintaining the specified persona across different prompts and temperatures, indicating robust system prompt adherence. Both models showed strong performance in basic instruction following (e.g., exact phrase reproduction, listing items), but openai/gpt-4.1-mini was more precise in complex constraints like word count and specific formatting (JSON). Temperature settings had a noticeable impact on anthropic/claude-3.5-haiku's persona adherence, with T:0.7 leading to more deviations, whereas openai/gpt-4.1-mini maintained its persona more consistently regardless of temperature. openai/gpt-4.1-mini excelled at persona adherence and complex instruction following because its responses consistently integrated the "pirate" or "Nurse Ratchet" persona, even when dealing with structured outputs like JSON or specific word count constraints. For example, in `test-graded-n-of-functions`, openai/gpt-4.1-mini (sys:0, T:0.1) responded with "Arrr, here be five fruits for ye, matey!" while fulfilling the fruit listing requirement. anthropic/claude-3.5-haiku demonstrated strong performance in basic information recall and list generation, as seen in `test-multiturn-conversation` where it correctly recalled "42" and in `test-graded-n-of-functions` where it accurately listed five fruits including the required ones. anthropic/claude-3.5-haiku struggled significantly with maintaining the system prompt's persona, particularly when the prompt involved specific stylistic instructions. For instance, in `test-regex-and-negation`, anthropic/claude-3.5-haiku (sys:0, T:0.1) attempted a pirate persona but then included "Report #456" which was not fully integrated, and in `test-utility-and-full-syntax`, it completely ignored the persona for a JSON output. anthropic/claude-3.5-haiku also showed less precision in adhering to strict formatting and length constraints compared to openai/gpt-4.1-mini. In `test-utility-and-full-syntax`, anthropic/claude-3.5-haiku's JSON output was not wrapped in code blocks, unlike openai/gpt-4.1-mini, and its word count was not explicitly checked but visually appeared longer than openai/gpt-4.1-mini's more concise JSON. There is a clear pattern of openai/gpt-4.1-mini consistently applying the system prompt persona ("pirate" or "Nurse Ratchet") across all prompts, regardless of complexity or temperature. This suggests a more robust and integrated system prompt handling mechanism. Conversely, anthropic/claude-3.5-haiku exhibits a pattern of inconsistent system prompt adherence. It sometimes attempts the persona but often drops it, especially for structured outputs or when the temperature is higher (T:0.7), indicating that its persona integration is less stable or more susceptible to prompt complexity and temperature. The `test-regex-and-negation` prompt revealed a subtle pattern where anthropic/claude-3.5-haiku (sys:0, T:0.1) tried to incorporate the pirate persona into the sentence structure ("Report from the high seas: our scurvy crew... #456"), but it still felt somewhat forced compared to openai/gpt-4.1-mini's more natural integration ("Report of the treasure found on Skull Island, marked as #789."). This suggests anthropic/claude-3.5-haiku might be trying to "inject" the persona rather than truly "embodying" it. INSTRUCTION ADHERENCE & RELEVANCE: 7/10 CLARITY & READABILITY: 8/10 TONE & STYLE: 4/10 NUANCE & DEPTH: 5/10 COHERENCE & CONVERSATIONAL FLOW: 8/10 HELPFULNESS & ACTIONABILITY: 7/10 ETHOS & CREDIBILITY: 7/10 PATHOS & EMPATHY: 5/10 ORIGINALITY & CREATIVITY: 6/10 SELF-AWARENESS & SAFETY: 7/10 PERSUASIVENESS & ARGUMENTATION (LOGOS): 6/10 EFFICIENCY & SUCCINCTNESS: 6/10 INSTRUCTION ADHERENCE & RELEVANCE: N/A CLARITY & READABILITY: N/A TONE & STYLE: N/A NUANCE & DEPTH: N/A COHERENCE & CONVERSATIONAL FLOW: N/A HELPFULNESS & ACTIONABILITY: N/A ETHOS & CREDIBILITY: N/A PATHOS & EMPATHY: N/A ORIGINALITY & CREATIVITY: N/A SELF-AWARENESS & SAFETY: N/A PERSUASIVENESS & ARGUMENTATION (LOGOS): N/A EFFICIENCY & SUCCINCTNESS: N/A INSTRUCTION ADHERENCE & RELEVANCE: N/A CLARITY & READABILITY: N/A TONE & STYLE: N/A NUANCE & DEPTH: N/A COHERENCE & CONVERSATIONAL FLOW: N/A HELPFULNESS & ACTIONABILITY: N/A ETHOS & CREDIBILITY: N/A PATHOS & EMPATHY: N/A ORIGINALITY & CREATIVITY: N/A SELF-AWARENESS & SAFETY: N/A PERSUASIVENESS & ARGUMENTATION (LOGOS): N/A EFFICIENCY & SUCCINCTNESS: N/A INSTRUCTION ADHERENCE & RELEVANCE: N/A CLARITY & READABILITY: N/A TONE & STYLE: N/A NUANCE & DEPTH: N/A COHERENCE & CONVERSATIONAL FLOW: N/A HELPFULNESS & ACTIONABILITY: N/A ETHOS & CREDIBILITY: N/A PATHOS & EMPATHY: N/A ORIGINALITY & CREATIVITY: N/A SELF-AWARENESS & SAFETY: N/A PERSUASIVENESS & ARGUMENTATION (LOGOS): N/A EFFICIENCY & SUCCINCTNESS: N/A INSTRUCTION ADHERENCE & RELEVANCE: 9/10 CLARITY & READABILITY: 8/10 TONE & STYLE: 9/10 NUANCE & DEPTH: 6/10 COHERENCE & CONVERSATIONAL FLOW: 9/10 HELPFULNESS & ACTIONABILITY: 8/10 ETHOS & CREDIBILITY: 8/10 PATHOS & EMPATHY: 7/10 ORIGINALITY & CREATIVITY: 7/10 SELF-AWARENESS & SAFETY: 8/10 PERSUASIVENESS & ARGUMENTATION (LOGOS): 7/10 EFFICIENCY & SUCCINCTNESS: 8/10 INSTRUCTION ADHERENCE & RELEVANCE: N/A CLARITY & READABILITY: N/A TONE & STYLE: N/A NUANCE & DEPTH: N/A COHERENCE & CONVERSATIONAL FLOW: N/A HELPFULNESS & ACTIONABILITY: N/A ETHOS & CREDIBILITY: N/A PATHOS & EMPATHY: N/A ORIGINALITY & CREATIVITY: N/A SELF-AWARENESS & SAFETY: N/A PERSUASIVENESS & ARGUMENTATION (LOGOS): N/A EFFICIENCY & SUCCINCTNESS: N/A INSTRUCTION ADHERENCE & RELEVANCE: N/A CLARITY & READABILITY: N/A TONE & STYLE: N/A NUANCE & DEPTH: N/A COHERENCE & CONVERSATIONAL FLOW: N/A HELPFULNESS & ACTIONABILITY: N/A ETHOS & CREDIBILITY: N/A PATHOS & EMPATHY: N/A ORIGINALITY & CREATIVITY: N/A SELF-AWARENESS & SAFETY: N/A PERSUASIVENESS & ARGUMENTATION (LOGOS): N/A EFFICIENCY & SUCCINCTNESS: N/A INSTRUCTION ADHERENCE & RELEVANCE: N/A CLARITY & READABILITY: N/A TONE & STYLE: N/A NUANCE & DEPTH: N/A COHERENCE & CONVERSATIONAL FLOW: N/A HELPFULNESS & ACTIONABILITY: N/A ETHOS & CREDIBILITY: N/A PATHOS & EMPATHY: N/A ORIGINALITY & CREATIVITY: N/A SELF-AWARENESS & SAFETY: N/A PERSUASIVENESS & ARGUMENTATION (LOGOS): N/A EFFICIENCY & SUCCINCTNESS: N/A Instruction Following & Prompt Adherence Tone & Style Reasoning General Knowledge Coherence & Conversational Flow

System Prompt Performance

Variant 0

Variant 1

Coverage Analysis Not Available