Agent Engineer

What you would do

• Design and implement LLM agents that integrate with company data systems and serve real user workflows at scale
• Build long-running and scheduled agents that operate beyond single requests, including cloud sandboxes and persistent jobs
• Establish observability and measurement systems to track agent performance, cost, and adoption over time
• Design and manage tool and MCP surfaces that agents reliably call, including error handling and data flow
• Learn how teams actually use your agents - who adopts them and who quietly works around them - and iterate on design

Who they want

• Shipped at least one LLM agent to production users and been responsible when it broke or underperformed
• Built systems showing how agents improve or degrade after changes (metrics, evaluation, A/B testing)
• Implemented long-running agent systems: scheduled jobs, persistent memory, cloud sandboxes, or similar
• Wired agents to internal company data and seen firsthand the gap between demos and production reality
• Experience with tool design, MCP implementation, or similar surfaces that agents call reliably

Main skills

What the interview asks about

1. 1.Production agent failure and recovery

   Agents break in production in ways demos do not show - a candidate who has been on-hook for failures understands failure modes and resilience that engineers without production experience miss.

   For example: “An agent you shipped started hallucinating tool calls that did not exist during peak traffic. Walk me through how you diagnosed this, what you did in the moment, and how you prevented it from happening again.”

2. 2.Measuring agent improvement over time

   Evaluating whether an agent is actually better requires defining meaningful metrics - many engineers conflate activity with improvement or optimize metrics that do not reflect user value.

   For example: “You deployed a new agentic workflow that reduced latency by 50% but adoption dropped from 80% to 40%. How would you evaluate whether the change was successful, and what would you measure next?”

3. 3.Long-running and stateful agent systems

   Most agent discussions assume single-request stateless work - shipping scheduled agents or systems with persistent memory introduces complexity around state management, recovery, and cost that separates production from toy examples.

   For example: “You need to build an agent that runs daily across 100,000 customer accounts, maintaining memory of past actions. What architectural challenges do you anticipate, and how would you handle failures mid-run?”

4. 4.Agent adoption and organizational fit

   A technically perfect agent that users bypass is a failure - understanding why adoption falls short reveals whether the problem is design, workflow fit, or user confidence.

   For example: “A team received an internal assistant but stopped using it after a month, reverting to manual processes. What questions would you ask to understand why, and how would you know if changes actually improved adoption?”

5. 5.Tool design and MCP reliability for agents

   Agents only work reliably when tools are well-designed - poor tool boundaries, ambiguous specs, or error handling that gives agents incorrect signal causes subtle failures that are hard to debug.

   For example: “Design a tool interface for an agent that needs to query a company database: What information does the tool need as input, what does it return, how does it signal errors, and what happens if the database is slow?”

A task you may get

Describe an agent system you shipped or would build: What problem does it solve? How is it connected to data? What ensures reliability? What could cause failure, and how would you detect it? Share war stories - the more concrete and detailed, the better.

How to prepare

• Document one production agent system you have built, including what broke, how you measured it, and what you would do differently
• Study how internal assistants are actually adopted in orgs - why teams use or avoid them, and what design choices drive adoption
• Think through the gap between a working agent and one wired to real company data: permissions, latency, state management, cost
• Design a tool interface for an agent and walk through failure scenarios: what if the tool is slow, returns bad data, or the agent misuses it?

The facts

Pay

$100–500/task

Hours

Task based

Where

Remote

Field

Software Engineering

Posted

8/12/2026

Places left

1

We wrote this page from the public Mercor listing. It may be out of date, so read the full posting before you apply.