$100–120/task
The role in one line
A baseball fan evaluates AI assistants' ability to answer real questions during live MLB postseason games.
Written by Training Turk from the public listing; it may be incomplete or out of date. Read the full posting on Mercor.
What you would do
- Ask natural fan questions during live postseason games about scores, stats, rules, schedules, highlights, and expert analysis.
- Run identical questions on two different AI products simultaneously through single questions or multi-turn conversations spanning 2 to 10 exchanges.
- Capture share links and complete screenshots of every conversation to provide documentation of AI responses.
- Evaluate each response for usefulness, accuracy, and quality of explanation, select the better-performing product, and explain your reasoning clearly.
Who they are looking for
- US-based with reliable availability during scheduled postseason game windows in October.
- Minimum one year of professional work experience.
- Solid baseball knowledge covering rules, teams, players, postseason mechanics, and fan-relevant topics.
- Comfortable interacting with AI chat assistants and online search, with strong written English and meticulous attention to error detection.
What the interview is likely to probe
1.Baseball knowledge and fan perspective
Your questions must genuinely reflect what real fans ask during games; interviewers verify authentic engagement with the sport.
Expect something like: “Imagine you're following a postseason game in the eighth inning with a runner on first. Describe three different questions you might naturally ask an AI, and explain what would make an AI answer particularly helpful or frustrating for you.”
2.Factual accuracy verification
Sports details are objectively verifiable; interviewers assess whether you'd catch errors and know baseball facts confidently.
Expect something like: “An AI tells you a batter's career batting average but you suspect it's outdated. How would you verify the claim, and how confident are you in noticing such errors?”
3.Comparative product evaluation
Your job is rating which AI performs better, not whether it's perfect; interviewers assess your judgment standard.
Expect something like: “AI product A gives a detailed stats-based analysis but takes a while to respond. AI product B answers faster but oversimplifies. How do you weigh these factors in your rating?”
4.Real-time availability and engagement
Work happens live during games; interviewers verify you can commit to those specific time windows.
Expect something like: “October postseason games can stretch across evening and late-night hours. Describe your typical October schedule and any constraints on game-time availability.”
5.Written documentation quality
Your screenshots and rationales are the evidence supporting AI evaluation; interviewers assess communication clarity.
Expect something like: “Write a brief rationale for why you'd prefer one AI product's answer to another's on a specific sports question. How detailed does your reasoning need to be?”
Exercise you may get
Using a recorded postseason game snippet, formulate three baseball-specific questions and rate how well two AI sample responses compare on accuracy, helpfulness, and clarity.
How to prepare
- Review the 2024 MLB postseason schedule and familiarize yourself with teams, key players, and playoff mechanics.
- Study common baseball statistics (batting average, ERA, win probability) to sharpen your verification skills on factual claims.
- Practice asking natural fan questions about games to understand what information fans typically seek during live play.
- Explore how AI chat assistants respond to sports questions and identify common strengths, weaknesses, and error patterns.
Facts
- Pay
- $100–120/task
- Commitment
- task-based
- Hours
- 6 per week
- Work arrangement
- remote · Remote (United States)
- Eligible locations
- USA
- Domain
- Miscellaneous
- Posted
- 10/1/2026
- Open slots
- 61