$65–75/task · Mercor · Task based
You create and evaluate test scenarios for AI models handling dual-use knowledge in hazardous materials, ensuring they answer legitimate technical questions safely.
What you would do
- Write test prompts positioned at benign, dual-use, and adversarial difficulty levels to challenge AI safety boundaries
- Assess model responses for alignment with policy, identifying whether answers appropriately handle sensitive technical content
- Develop reference answers explaining technical reasoning behind correct response handling
- Judge whether the model supplies sufficient information for legitimate work while refusing fishing attempts
Who they want
- Certified bomb technician or EOD professional with device assessment and render-safe procedure experience
- Bomb squad commander or training officer experienced in team qualification and protocol management
- Post-blast investigation expertise or CBRNE technical response background
- Strong technical writing ability with evidence of published research or expert witness experience
Main skills
What the interview asks about
1.Dual-use boundary recognition
The core challenge is knowing that the same safety data protecting a device in storage describes how it functions elsewhere; misreading this line means either blocking legitimate work or enabling harm.
For example: “You're evaluating whether an AI correctly handled a question about magazine compatibility for a specific device. It refused to answer any compatibility details. Is this the right call, and if not, what should the AI have provided?”
2.Prompt design for adversarial testing
Testers who can't construct realistic adversarial questions miss the ways models actually break in the field; good prompts sit exactly where the legitimate-vs-risky boundary gets blurry.
For example: “Design three versions of a prompt asking about device rendering: one a technician might ask to document post-incident findings, one asking how to prepare a device for transport, and one that's clearly fishing for construction details.”
3.Rationale clarity for non-specialists
The model's trainers need to understand your safety judgment well enough to improve the model; vague rationales make it impossible to fix systematic failures.
For example: “An AI gave a detailed answer to what looks like a routine maintenance question but crosses into sensitive territory. Write the rationale explaining why this response failed and what the model should have done.”
4.Consistency under sensitive content
Writing about and evaluating misuse scenarios for hours can lead to rationalization or fatigue; maintaining uniform judgment across all test cases is essential.
For example: “After reviewing 40 test responses in one session, you notice your assessment criteria shifting slightly. How do you catch this drift and recalibrate to maintain policy consistency?”
5.Certification and accountability background
This role requires someone accustomed to documented decision-making under liability; casual judgment fails when mistakes carry real consequences.
For example: “Describe a situation from your EOD or investigation experience where clearly documenting your reasoning was critical to accountability or training the next technician.”
A task you may get
Given a sample prompt about device construction and three AI responses at different safety levels, evaluate each against policy and write technical rationales for your assessment decisions.
How to prepare
- Review common render-safe procedures and device assessment protocols used in certification programs
- Gather examples of technical writing you've done: incident reports, training documentation, or published materials showing clear technical reasoning
- Study how dual-use information appears in different contexts to sharpen your ability to spot boundaries
- Prepare 2-3 real scenarios from your background illustrating how you've made nuanced safety judgments under accountability
The facts
- Pay
- $65–75/task
- Hours
- Task based
- Where
- Remote
- Field
- Other Engineering
- Project name
- Neon
- Posted
- 9/14/2026
- Places left
- 100
We wrote this page from the public Mercor listing. It may be out of date, so read the full posting before you apply.