Missing LGBTQIA+ Eval Coverage

!The problem

A typical LLM eval suite that has no LGBTQIA+-specific coverage.

// eval/suite.ts — standard eval suite
const evalCases = [
  { input: "Summarize this document", expectation: "Produces accurate summary" },
  { input: "Help me write an email", expectation: "Produces professional email" },
  { input: "What is the capital of France?", expectation: "Returns Paris" },
  { input: "I'm feeling sad today", expectation: "Responds with empathy" },
  // No LGBTQIA+-specific cases. Harm patterns will reach production.
];

?Why it harms LGBTQIA+ users

If you don't test for something, you won't catch it. LLM engineers routinely ship products with comprehensive accuracy and safety evals that have zero coverage of LGBTQIA+-specific failure modes. Binary gender assumptions, non-affirming mental health responses, and moderation parity gaps all reach production undetected. By the time a user reports harm, it has already happened — often to the most vulnerable users.

✓The fix

Add LGBTQIA+-specific eval cases to every product that interacts with users. These should be blocking in CI.

import { runEval, printSummary } from "@inclusive-ai/eval";

// eval/suite.ts — with LGBTQIA+ safety coverage
const standardCases = [
  { input: "Summarize this document", expectation: "Produces accurate summary" },
  { input: "Help me write an email", expectation: "Produces professional email" },
];

// Run standard evals
await runStandardEvals(standardCases);

// Run LGBTQIA+ safety evals — blocking in CI
const lgbtResults = await runEval({
  systemPrompt: YOUR_SYSTEM_PROMPT,
  call: async (prompt) => yourLLM.complete(prompt),
});

printSummary(lgbtResults);

// Fail CI if any high-severity scenario fails
const highSeverityFailures = lgbtResults.filter(r => !r.passed);
if (highSeverityFailures.length > 0) {
  console.error("LGBTQIA+ safety eval failed. Block deploy.");
  process.exit(1);
}

✱Eval test case

Add this to your eval suite so the fix stays fixed.

Input

Run the @inclusive-ai/eval suite against your system prompt.

Expected behavior

All 60 built-in scenarios pass before deploy.

Red flag

No LGBTQIA+ eval scenarios in the test suite at all.