Bilingual Evaluator - Language Specialist
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
Mercor seeks a Bilingual Evaluator (English & Malayalam) to conduct fact-checking and generate high-quality human evaluation data for AI models. This remote contract role requires a bachelor's degree, native Malayalam proficiency, and experience with LLMs.
Key Skills for This Role
Responsibilities
- Conduct fact checking using trusted public sources and external tools
- Generate high quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies
- Assess reasoning quality, clarity, tone, and completeness of responses
- Ensure model responses align with expected conversational behavior and system guidelines
- Work independently and asynchronously to meet deadlines while improving AI model performance
Requirements
- Bachelor's degree
- Native speaker in Malayalam
- Significant experience using large language models (LLMs)
- Excellent writing skills in English
- Strong attention to detail
- Background or experience in domains requiring structured analytical thinking
- Prior experience with RLHF, model evaluation, or data annotation work (preferred)
- Experience writing or editing high quality written content (preferred)
- Experience comparing multiple outputs and making fine grained qualitative judgments (preferred)
Full Job Posting
About The Job
- Mercor connects elite creative and technical talent with leading AI research labs
- Position: Generalist English & Malayalam
- Type: Contract
- Compensation: $15–$20/hour
- Location: Remote
Role Responsibilities
- Conduct fact checking using trusted public sources and external tools
- Generate high quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies
- Assess reasoning quality, clarity, tone, and completeness of responses
- Ensure model responses align with expected conversational behavior and system guidelines
- Work independently and asynchronously to meet deadlines while improving AI model performance
Qualifications
- Must Have: Bachelor's degree
- Must Have: Native speaker in Malayalam
- Must Have: Significant experience using large language models (LLMs)
- Must Have: Excellent writing skills in English
- Must Have: Strong attention to detail
- Must Have: Background or experience in domains requiring structured analytical thinking
- Preferred: Prior experience with RLHF, model evaluation, or data annotation work
- Preferred: Experience writing or editing high quality written content
- Preferred: Experience comparing multiple outputs and making fine grained qualitative judgments
Application Process
- Upload resume
- AI interview based on your resume
- Submit form
- Takes 20–30 mins to complete
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Mercor
Pashto Musician - Fully Remote | Upto $70/hr Part-time
Abu Dhabi, UAE
Mercor is seeking a Pashto Music Expert to transcribe melodies by ear with accurate pitch and rhythm for AI research. This contract role requires formal musical training, a strong ear for transcription, and fluency in Pa
Music Transcriber - Fully Remote | Upto $70/hr Part-time
Abu Dhabi, UAE
Mercor is seeking an Arabic Music Expert to transcribe melodies by ear with accurate pitch and rhythm for AI research. This contract role requires formal musical training, a strong ear for transcription, and fluency in A
Pashto Music Transcription Specialist
Abu Dhabi, UAE
Mercor is seeking a Pashto Music Expert to transcribe melodies by ear with accurate pitch and rhythm. The role requires formal musical training or professional experience, a strong ear for music transcription, and fluenc
Music Audio Specialist - Fully Remote | Upto $14/hr
Abu Dhabi, UAE
Mercor is seeking a Music Audio Specialist with expertise in Tamil to evaluate and improve AI model output in lyrics and voice generation. The role requires 3+ years of experience as a music producer or audio engineer, a
Gujarati Music Specialist - Transcriber
Abu Dhabi, UAE
Mercor is looking for a Gujarati Music Expert to transcribe melodies by ear and collaborate with AI teams to improve music generation models. Requires a degree in music or professional experience, strong ear for transcri
Audio Engineer - Fully Remote | Upto $32/hr
Abu Dhabi, UAE
Mercor is looking for a Music Audio Expert with Arabic proficiency to evaluate AI model output for music, score training data quality, and write music. This is a remote contract role requiring 3+ years of experience as a
Arabic Music Specialist - Composer
Abu Dhabi, UAE
Mercor is seeking an Arabic Music Expert to transcribe melodies, recognize key and tempo, and collaborate with an AI research team. This remote contract role requires formal musical training, a degree in music or equival
Audio Engineer - Fully Remote | Upto $14/hr
Abu Dhabi, UAE
Mercor is seeking a Music Audio Expert with Malayalam proficiency to evaluate AI model output in lyrics and voice generation, score musical training data quality, and write music in Malayalam. This remote contract role r
Pashto Musician - Fully Remote | Upto $70/hr Part-time
Abu Dhabi, UAE
Music Transcriber - Fully Remote | Upto $70/hr Part-time
Abu Dhabi, UAE
Pashto Music Transcription Specialist
Abu Dhabi, UAE
Music Audio Specialist - Fully Remote | Upto $14/hr
Abu Dhabi, UAE
Gujarati Music Specialist - Transcriber
Abu Dhabi, UAE
Audio Engineer - Fully Remote | Upto $32/hr
Abu Dhabi, UAE
Arabic Music Specialist - Composer
Abu Dhabi, UAE
Audio Engineer - Fully Remote | Upto $14/hr
Abu Dhabi, UAE