$60–90/hr · Mercor · Hourly, 30 hours a week
You design engineering simulations and specifications that evaluate whether AI models can reason from first principles to solve design problems.
What you would do
- Define realistic engineering problems in your domain with quantitative specifications and pass-fail thresholds
- Write Python simulations that let AI models probe the design space by adjusting parameters like controller gains, circuit values, or geometry
- Specify acceptance criteria as measurable constraints that an agentic grader can use to score whether AI-submitted designs satisfy requirements
- Design problems that require reasoning through physics and engineering mathematics rather than retrieval of standard solutions
- Collaborate with AI researchers to interpret model exploration patterns and refine problem difficulty and scope
Who they want
- Expert knowledge spanning circuit design (analog/RF), control theory, power conversion, or mechanical engineering; PhD or industry equivalent
- Skilled at implementing Python simulations and translating engineering requirements into measurable, verifiable success criteria
- Ability to design problems you cannot shortcut yourself but can still judge rigorously
- Experience working in browser-based tools and GitHub for collaborative development
- Hands-on familiarity with model evaluation or similar research evaluation work
Main skills
What the interview asks about
1.First-principles problem design
AI trained on solutions to standard problems learns patterns, not reasoning. Your ability to design novel problems that require stepping through equations distinguishes valuable training from easy pattern-matching.
For example: “Design a control systems task where an AI model must reason about feedback loop stability and trade-offs between response speed and overshoot. What parameters would you vary to ensure it cannot succeed by memorizing standard controller designs?”
2.Specification precision and scoring
Ambiguous specs let AI generate plausible-sounding but untestable solutions. Rigorous specs with measurable pass-fail criteria are what enable meaningful evaluation.
For example: “Write the quantitative specifications for a power electronics design task where an AI must choose component values to meet efficiency, thermal, and output voltage requirements. What would make this specification rigorous enough to score objectively?”
3.Problem difficulty calibration
Problems too easy do not measure reasoning; too hard frustrate rather than teach. You must calibrate so AI systems can explore meaningfully while still demonstrating understanding.
For example: “You design a mechanical design task and notice AI models either solve it trivially or spend all their simulation calls making no progress. How would you adjust specifications to create a problem where reasoning matters?”
4.Simulation robustness and edge cases
AI will probe your simulation in unexpected ways; you must anticipate edge cases so the simulation and scoring remain valid when models try unconventional approaches.
For example: “You have written a circuit design simulation and an AI model tries to submit component values far outside normal ranges. How do you handle edge cases in your scoring without breaking the problem?”
5.Collaborative iteration with researchers
Researchers need to understand whether low model performance is due to reasoning gaps or specification problems; your domain expertise guides problem refinement.
For example: “Researchers show you that multiple AI attempts failed at a design task. How would you analyze whether the problem needs engineering adjustment vs. whether it is correctly revealing model reasoning gaps?”
A task you may get
Design a complete evaluation task in your domain: write the problem statement, Python simulation with pass-fail criteria, and three example starting points for AI models to explore.
How to prepare
- Prepare a real engineering challenge from your work; analyze which aspects require first-principles reasoning and which could be solved by pattern-matching.
- Gather 2-3 examples of engineering problems with standard solutions; for each, write a variant that requires reasoning steps to solve successfully.
- Draft a Python simulation sketch for a problem in your domain; identify edge cases and how you would handle them in objective scoring.
The facts
- Pay
- $60–90/hr
- Hours
- Hourly, 30 hours a week
- Where
- Remote
- Field
- Other Engineering
- Posted
- 8/31/2026
- Places left
- 11
We wrote this page from the public Mercor listing. It may be out of date, so read the full posting before you apply.