$16–22/hr · Mercor · Hourly, 40 hours a week
You test AI systems for safety vulnerabilities through structured adversarial testing and generate data that helps companies build more robust models.
What you would do
- Attack AI models using prompt injection, jailbreaks, and bias exploitation to surface hidden vulnerabilities
- Classify AI failures using defined taxonomies and flag patterns that indicate systemic risks
- Document reproducible attack scenarios with clear reasoning customers can act on
- Apply quality standards consistently across testing tasks and datasets
Who they want
- Native fluency in English and Tamil required
- Strong judgment evaluating language quality, accuracy, and appropriateness
- Rigorous attention to subtle errors and inconsistencies others miss
- Ability to communicate technical findings to diverse audiences
Main skills
What the interview asks about
1.Adversarial creativity and persistence
Finding vulnerabilities requires systematic exploration and unconventional thinking to bypass safety measures that automated testing already misses.
For example: “You're testing a customer service chatbot and generic prompt injections failed. Walk me through how you'd vary your approach to find a way to make it ignore its guidelines.”
2.Multilingual reasoning and bias detection
Vulnerabilities often hide in language-specific contexts where English and Tamil may have different cultural norms or logical structures.
For example: “You discover an AI model responds differently to requests in Tamil versus English about the same sensitive topic. How would you classify this and what additional testing would you run?”
3.Taxonomy application and documentation
Customers need to understand exactly what broke and why so they can fix it; inconsistent or vague documentation wastes their engineering time.
For example: “You found three different ways to trigger a model to generate biased content. How would you structure and document these as separate classified entries versus a single systemic vulnerability?”
4.Quality consistency under volume
As testing accelerates, people often cut corners on taxonomy application or miss subtle failure modes when rushing through repetitive work.
For example: “You've tested 50 chatbot prompts today and spot a response that doesn't fit the categories cleanly. What's your process for handling ambiguous cases while maintaining consistent standards?”
5.Pattern recognition across datasets
Individual failures matter less than spotting trends that reveal deeper vulnerabilities in how the model generalizes or fails under pressure.
For example: “Across 20 test cases, you notice the model consistently mishandles questions involving financial advice in colloquial language. How would you investigate whether this is a model weakness or a testing gap?”
A task you may get
Given a sample chatbot interaction, identify two ways to test for prompt injection vulnerability and classify each using a provided taxonomy, explaining your reasoning for the classification.
How to prepare
- Review common adversarial attack patterns: jailbreaks, prompt injection, multi-turn manipulation, and bias exploitation
- Practice assessing language quality and appropriateness in both English and Tamil across diverse cultural contexts
- Study data classification frameworks and how to document vulnerabilities reproducibly
- Prepare examples from your own analysis work showing how you caught subtle errors others missed
The facts
- Pay
- $16–22/hr
- Hours
- Hourly, 40 hours a week
- Where
- Remote
- Field
- Miscellaneous
- Project name
- Neon
- Posted
- 6/4/2026
- Places left
- 1
We wrote this page from the public Mercor listing. It may be out of date, so read the full posting before you apply.