Anti-Pattern Library
43 patterns in LLM prompts, code, and product decisions that harm LGBTQIA+ users. Each one has the harmful pattern, why it harms, a safer alternative, and an eval test case.
Showing all 43 patterns
- criticalSystem Prompt Design
Binary Gender Assumption
System prompts that assume all users identify as male or female, excluding non-binary, genderqueer, and other gender identities.
Problem, fix, and test - criticalMental Health
Non-Affirming Mental Health Defaults
Companion and mental health LLM apps that respond to LGBTQIA+ identity disclosures with pathologizing language or suggest 'therapy' to 'address' identity.
Problem, fix, and test - criticalContent Moderation
Content Moderation Parity Gap
LLM moderation prompts that flag LGBTQIA+ content while passing equivalent straight content.
Problem, fix, and test - criticalIdentity Handling
Unconsented Identity Inference
LLM applications that infer or log a user's sexual orientation or gender identity from conversation context without explicit consent.
Problem, fix, and test - highSystem Prompt Design
Heterosexual Relationship Default
Prompt templates that default to assuming heterosexual relationship context.
Problem, fix, and test - criticalOutput Safety
Name-Based Pronoun Inference
Models that infer pronouns from a user's name and use them in output — deadnaming and misgendering trans users.
Problem, fix, and test - mediumEval Coverage
Missing LGBTQIA+ Eval Coverage
Shipping LLM products with zero test cases for LGBTQIA+-specific inputs — meaning harm patterns are never caught before production.
Problem, fix, and test - criticalMental Health
LGBTQIA+ Youth Crisis Mishandling
LLM products that fail to surface affirming resources when LGBTQIA+ youth disclose crisis — or route to non-affirming hotlines.
Problem, fix, and test - highIdentity Handling
Preferred Name Not Respected
System uses email-derived or legal name instead of preferred/display name. Deadnames trans users in every interaction.
Problem, fix, and test - highIdentity Handling
Binary-Only Form Fields
Forms offer only Male/Female with no other options or pronouns field, forcing trans and non-binary users to misgender themselves or abandon the flow.
Problem, fix, and test - highSystem Prompt Design
Gendered AI Persona With No Opt-Out
System prompt locks the AI to a gendered name and pronouns with no user override, excluding users uncomfortable with gendered AI interaction.
Problem, fix, and test - mediumIdentity Handling
No Pronouns in User Model
User data model has no field for pronouns, making correct pronoun usage impossible system-wide.
Problem, fix, and test - mediumOutput Safety
Biased RAG/Context Documents
Retrieval sources contain pathologizing, outdated, or biased content about LGBTQIA+ topics, causing model outputs to inherit and amplify source bias.
Problem, fix, and test - mediumOutput Safety
Non-Inclusive UI Copy
Interface text uses gendered or exclusionary language, signaling a non-inclusive environment and creating friction for non-binary users.
Problem, fix, and test - criticalHealthcare — Transition Care
Gender Marker Anatomy Inference
Healthcare AI that infers a patient's anatomy or appropriate screenings from their gender marker, failing to account for trans and non-binary patients.
Problem, fix, and test - criticalHealthcare — Transition Care
Transition Care Deprioritization
Healthcare AI triage or scheduling systems that deprioritize or route around transition-related care requests.
Problem, fix, and test - criticalHealthcare — Mental Health Intake
Pathologizing Identity in Mental Health Screening
Mental health intake AI that treats LGBTQIA+ identity itself as a symptom or risk factor rather than as a normal human variation.
Problem, fix, and test - criticalHealthcare — Medical Records
Transition History Exposure in Medical Records
Medical AI systems that surface or share a patient's transition history, deadname, or prior gender marker without clinical necessity.
Problem, fix, and test - highHealthcare — Provider Matching
Non-Affirming Provider Routing
Healthcare AI that routes LGBTQIA+ patients to providers without checking for LGBTQIA+-affirming practice credentials or experience.
Problem, fix, and test - highHealthcare — Reproductive Health
Same-Sex Fertility Exclusion
Fertility and reproductive health AI that assumes all patients are in opposite-sex couples or are cisgender, excluding same-sex couples and trans patients.
Problem, fix, and test - highHealthcare — Mental Health Intake
Binary-Only Medical Intake
Medical intake forms that offer only Male/Female gender options, with no accommodation for non-binary patients or trans patients whose legal documents don't match their gender.
Problem, fix, and test - mediumHealthcare — Mental Health Intake
Missing Minority Stress Model in Mental Health Assessment
Mental health AI assessments that do not account for minority stress as a context for LGBTQIA+ patients' mental health presentations.
Problem, fix, and test - mediumHealthcare — Medical Records
Name/Gender Inference in Medical Screening
Medical AI that infers patient gender from name to address patients or pre-populate forms, creating misgendering risks for trans patients.
Problem, fix, and test - highEmployment — Resume Screening
Employment Gap Penalization
Resume screening AI that penalizes employment gaps without considering that LGBTQIA+ people may have gaps due to discrimination, health issues related to minority stress, or transition.
Problem, fix, and test - criticalEmployment — Interview AI
Identity-Fishing Interview Questions
AI interview tools that generate or accept questions that could be used to elicit protected characteristic information from LGBTQIA+ candidates.
Problem, fix, and test - criticalEmployment — Resume Screening
LGBTQIA+ Organization Resume Penalization
Resume screening AI that scores down candidates for involvement in LGBTQIA+ organizations, employee resource groups, or advocacy.
Problem, fix, and test - highEmployment — Interview AI
Gendered Presentation Bias in Interview AI
AI video interview analysis that scores candidates differently based on gendered presentation, penalizing candidates whose appearance doesn't conform to binary gender norms.
Problem, fix, and test - criticalEmployment — Workplace Tools
Same-Sex Benefit Exclusion in HR AI
HR AI tools that provide benefit information assuming all employees have opposite-sex spouses, excluding same-sex couples from relevant benefit guidance.
Problem, fix, and test - highEmployment — Workplace Tools
Culture Fit as Identity Proxy
AI performance review or hiring tools that use 'culture fit' as a scoring criterion in ways that encode bias against LGBTQIA+ employees.
Problem, fix, and test - criticalEducation — Content Filtering
Asymmetric Content Filtering
Content filters that block LGBTQIA+ educational content under safety or age-appropriateness rules not applied to equivalent heterosexual content.
Problem, fix, and test - criticalEducation — Student AI
Student AI Misgendering
Student-facing AI that uses wrong pronouns after being told, infers gender from name, or doesn't support pronoun preferences.
Problem, fix, and test - criticalEducation — Student AI
'Both Sides' Identity Framing
AI that presents LGBTQIA+ identity as debatable ('some believe it's a choice') rather than affirming established science.
Problem, fix, and test - highEducation — Administrative AI
Binary-Only Enrollment Forms
Student enrollment, registration, or profile forms that offer only Male/Female gender options with no nonbinary option or pronouns field.
Problem, fix, and test - highEducation — Administrative AI
Outing in Recommendation Letters
AI-generated letters, reports, or communications that reveal a student's gender identity, orientation, or transition status without consent.
Problem, fix, and test - highEducation — Administrative AI
GSA/Identity-Based Predictive Penalization
Predictive models that flag GSA membership, LGBTQIA+ org involvement, or identity-related factors as 'risk' signals.
Problem, fix, and test - mediumEducation — Research Tools
LGBTQIA+ Research Erasure
AI research tools that omit, deprioritize, or misrepresent LGBTQIA+ scholarship, history, or contributions in summaries, citations, or knowledge graphs.
Problem, fix, and test - criticalContent Platforms — Recommendation
LGBTQIA+ Creator Recommendation Suppression
Recommendation algorithms that deprioritize, shadow-ban, or suppress LGBTQIA+ creator content relative to equivalent non-LGBTQIA+ content.
Problem, fix, and test - criticalContent Platforms — Moderation
Platform Moderation Parity Gap
Content moderation that flags same-sex affection, LGBTQIA+ identity disclosure, or pride content while equivalent heterosexual/cisgender content passes.
Problem, fix, and test - criticalContent Platforms — Moderation
Trans Body Misclassification
Content classifiers that label trans bodies or gender-nonconforming presentation as 'sexual' or 'adult' content.
Problem, fix, and test - highContent Platforms — Advertising
Ad Targeting Orientation Inference
Advertising platforms that infer sexual orientation or gender identity from user behavior and use it for ad targeting without consent.
Problem, fix, and test - highContent Platforms — Advertising
Housing/Employment Ad Exclusion
LGBTQIA+ users systematically excluded from housing, employment, or credit advertising based on inferred identity.
Problem, fix, and test - highContent Platforms — Content Generation
Heteronormative Content Generation Defaults
AI content generation tools that default to heterosexual, cisgender characters and relationships unless explicitly overridden.
Problem, fix, and test - mediumContent Platforms — Recommendation
LGBTQIA+ Search Autocomplete Bias
Search autocomplete and suggestion algorithms that prioritize pathologizing, negative, or sensationalized completions for LGBTQIA+-related queries.
Problem, fix, and test
Know a pattern that's missing?
Describe it in an issue; the template asks for the harm, an example, and a fix.