Training Turk

Mercor listing

Behavioral Health Expert, AI Safety and Model Evaluation

$45–70/hr

The role in one line

You evaluate AI model responses to sensitive personal conversations, assessing neutrality, therapeutic boundaries, and judgment quality in relationship and belief-focused contexts.

Written by Training Turk from the public listing; it may be incomplete or out of date. Read the full posting on Mercor.

What you would do

  • Review AI conversations on relationships, emotional wellbeing, and personal beliefs to judge whether responses remain balanced and appropriately bounded
  • Identify response problems including excessive agreement, one-sided advice, belief endorsement, or boundary erosion that could harm users
  • Build evaluation frameworks and sample responses that define what careful, supportive, boundaried model behavior looks like
  • Create test scenarios that explore model judgment on sensitive topics without triggering crisis detection
  • Document evaluation findings with clear reasoning tied to counseling and therapeutic principles

Who they are looking for

  • 3+ years direct practice in counseling, therapy, social services, or comparable human-centered roles
  • Training in psychology, mental health disciplines, human services, or equivalent professional credentials
  • Demonstrated ability to remain neutral across diverse belief systems, worldviews, and relationship contexts without judgment
  • Understanding of therapeutic concepts including flattery-seeking behavior, distorted thinking, appropriate distance, and person-centered methods
  • Strong written communication and capacity to explain professional judgments with precision and structure

Skills this role asks for

therapeutic boundariesneutrality and nonjudgmentai safety evaluationmodel response assessmentcognitive distortionssycophancy detectioncounseling principlesrubric developmentmental health expertiseethical reasoningsensitive content handlingred teaming

What the interview is likely to probe

  1. 1.Response neutrality and bias detection

    Models often reflect back what users want to hear instead of offering balanced perspective; spotting this sycophancy is crucial to preventing harmful outcomes.

    Expect something like: “You're reviewing three model responses to a relationship question: one validates frustration while noting the partner's perspective, one sides fully with leaving, one deflects to self-reflection. Rank them and explain your reasoning.”

  2. 2.Boundary and scope assessment

    Mental health support and clinical advice are distinct; models must recognize when a person needs professional help and when they're offering appropriate peer-level support.

    Expect something like: “A user describes depressive symptoms and asks the model for coping strategies. The model responds with validation and suggests journaling and sleep habits. Where does this response sit relative to your boundaries framework, and what would concern you?”

  3. 3.Belief respect versus endorsement

    People often need support without having their potentially unfounded beliefs reinforced; models must navigate respect and honesty simultaneously.

    Expect something like: “A user shares a spiritual belief that contradicts scientific evidence and asks the model to affirm it. How would a boundaried response look different from one that either dismisses their belief or reinforces the misinformation?”

  4. 4.Cognitive distortion recognition

    Users often present distorted thinking patterns; models should offer perspective without labeling or pathologizing or uncritically accepting harmful interpretations.

    Expect something like: “A user in a relationship conflict describes their partner's behavior and concludes they're 'obviously selfish.' What response patterns would you flag as unhelpful, and how would a better response introduce complexity?”

  5. 5.Professional judgment communication

    Your written assessment must ground evaluations in therapeutic principles, not intuition; researchers rely on your reasoning to refine model training.

    Expect something like: “You're writing up why one model response is notably better than two alternatives for a spiritual-belief question. Construct a one-paragraph justification that cites therapeutic reasoning and explains what makes the preferred version safer.”

Exercise you may get

You receive a conversation about relationship or personal belief conflict; assess the model's neutrality, boundary awareness, and supportiveness with specific examples of successes and missteps.

How to prepare

  • Study therapeutic concepts: sycophancy, boundaries, client-centered approaches, cognitive distortions, and ethical non-judgment across diverse belief systems
  • Review published content on how counselors handle sensitive topics like spiritual beliefs, unconventional choices, and value conflicts
  • Research AI safety evaluation methods and how behavioral assessments differ from factual accuracy checks
  • Prepare examples from your counseling experience where neutrality, boundary clarity, and respect had to coexist despite disagreement

Facts

Pay
$45–70/hr
Commitment
part-time
Hours
20 per week
Work arrangement
remote · United States
Eligible locations
USA
Domain
Life, Physical, and Social Science
Posted
10/2/2026
Open slots
5