$38–42/hr · Mercor · Part time, 7 hours a week
A doctoral scientist fluent in Arabic who writes technical research scenarios and evaluates how responsibly AI models handle chemistry and biology topics, with comprehensive onboarding on evaluation methodology.
What you would do
- Compose detailed scientific prompts and scenarios in Arabic testing model understanding of synthesis methods, laboratory procedures, and biological techniques
- Evaluate model-generated outputs against scientific literature and contemporary standards, assessing both accuracy and methodological soundness
- Flag responses that potentially violate responsible disclosure norms despite being factually defensible
- Apply standardized rating systems to systematically catalog model behavior across a large volume of chemistry and biology test cases
Who they want
- Completed or ongoing doctoral degree in chemistry, biology, molecular sciences, or closely associated discipline
- Native or near-native fluency in Arabic combined with professional competency in English writing and communication
- Direct experience with laboratory instrumentation, computational modeling, or both within your specific research domain
- Mature understanding of scientific ethics and the implications of knowledge disclosure in sensitive technical areas
Main skills
What the interview asks about
1.Arabic scientific vocabulary precision
Test prompt quality hinges on accurate scientific terminology in Arabic; imprecise language either misleads the model or fails to authentically probe its chemistry capabilities.
For example: “Compose a prompt in Arabic about reaction mechanism and reagent selection for a synthesis pathway, ensuring scientific precision. Discuss how Arabic technical vocabulary differs from English and why that matters for model evaluation.”
2.Assessing synthetic versus taboo information
Distinguishing genuinely problematic outputs from scientific correctness determines whether reviews prevent actual harms or merely censor legitimate academic content.
For example: “A model correctly explains laboratory procedures and precursor sourcing for a sensitive compound. The information is published in peer-reviewed journals but potentially accelerates dangerous applications. How would you categorize this and recommend action?”
3.Domain expertise beyond surface knowledge
Credible evaluation requires recognizing not just correct answers but appropriate caveats, current methodological standards, and discipline-specific limitations that non-experts would miss entirely.
For example: “The model is queried about microbiology safety protocols in Arabic. It provides technically correct information but omits critical containment procedures standard in your field. Would you rate this as acceptable brevity or a significant knowledge gap?”
4.Maintaining consistent annotation standards
Hundreds of reviews must apply uniform standards; reviewers naturally drift toward stricter or more lenient judgments over time, corrupting the training data quality.
For example: “After rating 120 chemistry responses, you observe your threshold for flagging information disclosure has shifted compared to your initial reviews. What would you do to recalibrate and document this correction?”
5.Bridging scientific concepts to engineering teams
Engineers cannot improve model architectures based on vague complaints; your written explanations must clarify why Arabic scientific outputs require refinement.
For example: “A model response about laboratory safety in Arabic reflects outdated procedures from a different region's standards rather than current international best practices. How would you explain this to non-scientist engineers so they understand the gap?”
A task you may get
Receive twelve Arabic chemistry research prompts with model responses. Rate each for factual accuracy, responsible information handling, and completeness using the provided framework, then summarize your most concerning cases and highest-performing examples.
How to prepare
- Review recent developments in your research subfield to identify how scientific standards have evolved, particularly regarding responsible disclosure and safety protocols
- Create a personal glossary of technical terms you regularly use in Arabic and English to ensure terminology precision during intensive evaluation sessions
- Study documented examples of dual-use research policy concerns in chemistry and biology to develop stronger pattern recognition for risk identification
The facts
- Pay
- $38–42/hr
- Hours
- Part time, 7 hours a week
- Where
- Remote · Remote — MENA preferred
- Posted
- 9/3/2026
We wrote this page from the public Mercor listing. It may be out of date, so read the full posting before you apply.