$120/hr · Mercor · Hourly, 40 hours a week
A research professional evaluates survey design for methodological rigor and decision-usefulness, coaching AI systems on research standards and survey quality assessment.
What you would do
- Review survey instruments for questionnaire design quality, including wording, response options, scales, skip logic, and overall structure soundness
- Assess surveys for methodological problems such as leading questions, ambiguous wording, double-barreled items, response-option bias, and question ordering effects
- Identify potential bias sources across sampling, measurement, response, and selection that could compromise data reliability or interpretability
- Evaluate fielding approaches including respondent recruitment, panel composition, quota strategies, and data-quality monitoring practices
- Provide clear, structured feedback explaining methodological issues and specific design recommendations
Who they want
- 3+ years professional experience across survey methodology, market insights, consumer behavior research, applied behavioral science, user research, or polling organizations
- Meaningful hands-on experience designing and fielding surveys in professional research contexts
- Strong understanding of questionnaire design principles and various distortion mechanisms in survey data collection
- Experience with respondent panels, sampling strategies, screeners, quotas, and fieldwork is valuable
- Ability to critically evaluate survey methodology and explain why design choices strengthen or weaken research quality
Main skills
What the interview asks about
1.Questionnaire design flaws
Identifying wording problems, response-option issues, and scale construction weaknesses requires deep expertise in how survey design affects data quality.
For example: “A survey asks 'Do you agree or disagree that our customer service is excellent and responsive?' Identify the questionnaire design problems and suggest how to rewrite the item.”
2.Bias source identification
Recognizing sampling, measurement, response, and selection bias demonstrates systematic understanding of where survey reliability breaks down.
For example: “A satisfaction survey is administered only to customers who remain with the company, not those who switched to competitors. Assess what bias this introduces and its impact on insights.”
3.Fielding evaluation rigor
Assessing respondent recruitment, quota systems, and fieldwork controls requires practical knowledge of how fieldwork execution affects data quality.
For example: “A survey uses online-only recruitment from a customer email list, but the company wants insights from the broader market. Analyze the sampling and response bias implications.”
4.Practical research judgment
Many survey design decisions involve trade-offs without single correct answers; applying nuanced judgment reflects professional research competence.
For example: “A researcher must choose between a longer, comprehensive questionnaire that risks respondent fatigue or a shorter version that omits desired content. Discuss your reasoning framework.”
5.Methodological rationale clarity
Communicating clear, evidence-based reasoning for survey design feedback ensures research teams understand principles underlying quality assessments.
For example: “Write feedback on a survey that uses a five-point scale but offers a six-point option for one question. Explain the methodological concern and recommend a specific solution.”
A task you may get
Candidates evaluate 3-4 surveys of varying quality and complexity, assess methodological strengths and weaknesses, identify bias sources, and provide structured improvement recommendations.
How to prepare
- Study questionnaire design principles including question wording, response-option construction, scale design, and skip-logic patterns that affect data quality
- Research common sources of survey bias including sampling, measurement, response, selection, and how each affects data interpretability
- Review fielding best practices for respondent recruitment, panel management, quota systems, and fieldwork quality controls
- Examine published research methodology papers and survey instruments from reputable firms to understand professional standards and design choices
The facts
- Pay
- $120/hr
- Hours
- Hourly, 40 hours a week
- Where
- Remote
- Posted
- 9/4/2026
We wrote this page from the public Mercor listing. It may be out of date, so read the full posting before you apply.