linkedin
AI Quality Analyst (Arabic)
MillionLogics
Abu Dhabi, UAE
Contract
Mid
Remote
1 months ago
ArabicData AnnotationAI Quality EvaluationPrompt EngineeringAnalytical ThinkingAttention to Detail
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
ArabicData AnnotationAI Quality Evaluation
About the Role
MillionLogics is hiring an AI Quality Analyst to evaluate a personalization feature for Gemini. The role involves designing prompts, evaluating model responses for grounding and helpfulness, and providing detailed rationales.
Key Skills for This Role
ArabicData AnnotationAI Quality EvaluationPrompt EngineeringAnalytical ThinkingAttention to Detail
Responsibilities
- Design and execute multi turn conversational prompts requiring AI to use personal information
- Evaluate model responses for personalization quality, grounding, integration, and helpfulness
- Stack rank two model responses side by side and write clear rationales
- Extract and verify debug info to confirm proper use of chat summaries and data sources
- Maintain strict data hygiene by deleting evaluation conversations
Requirements
- Arabic proficiency: ability to read and write in Arabic with high competence
- Willingness to use personal Google account and enable personal data sources
- Full time availability in local time zone
- Exceptional analytical thinking and ability to evaluate nuanced AI responses
- Experience in data annotation, AI quality evaluation, content moderation, or related role strongly preferred
- BS/BA degree or equivalent experience in relevant field
Full Job Posting
Role Overview
- As an AI Quality Analyst, you will evaluate a new personalisation feature for Gemini.
- You will assess how well the model uses information from past Gemini conversations, Gmail, Google Search, and YouTube activity.
Description
- Design and execute multi turn conversational prompts (typically 1 5 turns) that require the AI to utilise your personal information.
- Evaluate model responses based on your intent from the starting prompt.
- Analyse responses for Grounding issues, ensuring claims are supported by evidence.
- Assess Integration quality to ensure personal data is woven naturally.
- Rigorously evaluate and stack rank two model responses side by side.
- Write clear, defensible rationales for your comparisons.
- Extract and verify Debug Info from the model.
- Maintain strict data hygiene by deleting evaluation conversations.
Key Qualifications
- Arabic proficiency: ability to read and write in Arabic with high competence.
- Willingness to use primary personal Google account and enable personal data sources.
- Full time availability in local time zone.
- Exceptional analytical thinking and ability to evaluate nuanced AI responses.
- Creative prompt engineering experience.
- Strong evaluation acumen for personalisation concepts.
- Meticulous attention to detail for side by side comparisons.
- Excellent written communication skills.
- Ability to provide constructive feedback and detailed annotations.
- Self motivated and able to work independently in a remote setting.
- Desktop/Laptop with good internet connection.
Offer Detail
- Duration: 12 months
- Pay: $1500 per month (net/take home)
- Mode of work: Fully Remote
- Number of positions: 40
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career