AI trainer, chatbot response evaluation
Scale AI · Apr 2024 · Data annotation and evaluation
A new Remotasks contributor recorded a video interview, read long guidelines and joined training calls. The evaluation of chatbot answers came next, and they failed with no feedback.
- Community report, paraphrased
- 5 rounds
Read the full experienceHide the full experience
How it ran
- A recorded video interview.
- A long set of guidelines to read.
- An hour-long training video.
- An hour-long group call with trainers.
- An evaluation test rating chatbot responses.
What was asked
- Assess chatbot responses against the project guidelines.
What the interviewer or test was like
- A recorded video interview at the start, with no live interviewer.
- The test seemed to be graded automatically within an hour.
Tips
- Study the guidelines closely, since the test is judged against them.
Outcome
Failed the evaluation with no feedback.