Data Quality Engineer, AI Business

Investigate quality and integrity issues end to end, running root-cause analysis across guidelines, UX, screening, training, and operations, and driving corrective and preventive actions (CAPAs).

Build dashboards and operating cadences to track defect rates, rework, throughput versus quality trade-offs, integrity events, and SLA adherence.

Lead calibration sessions and coach QA leads and reviewers to improve decision consistency, rubric application, and overall quality judgement.

Translate one-off quality fixes into repeatable, scalable playbooks across customers, programs, and study types.

What You’ll Bring to the Role

5+ years of experience in quality engineering, data or annotation quality, analytics engineering, trust and integrity, or ML/LLM evaluation operations.

Strong proficiency in Python and SQL, with comfort applying statistical concepts such as sampling strategies, confidence levels, and agreement metrics.

A proven track record of turning ambiguous or messy quality problems into clear metrics, automated checks, and durable process improvements.

Strong quality systems thinking, with the ability to translate complex edge cases into clear rules, tests, rubrics, and governance mechanisms.

Hands-on experience instrumenting workflows and implementing pragmatic automation that catches quality and integrity issues early.

Demonstrated ability to influence cross-functional teams (Product, Engineering, Operations, Client teams) and drive change without direct authority.

Strong customer empathy, with a clear understanding of what “useful, trustworthy data” means for research, AI training, and evaluation use cases. Even Better if you have:

Familiarity with data collection mechanics (screeners, quota/routing constraints, study design patterns).

LLM evals, red teaming, or policy-based annotation experience.

Data/versioning discipline (dataset lineage, change control, reproducibility).

Experience with integrity/fraud detection systems and anti-abuse tooling

Why Prolific is a great place to work

We've built a unique platform that connects researchers and companies with a global pool of participants, enabling the collection of high-quality, ethically sourced human behavioral data and feedback. This data is the cornerstone of developing more accurate, nuanced, and aligned AI systems.

We believe that the next leap in AI capabilities won't come solely from scaling existing models, but from integrating diverse human perspectives and behaviors into AI development. By providing this crucial human data infrastructure, Prolific is positioning itself at the forefront of the next wave of AI innovation – one that reflects the breath and the best of humanity.

Working for us will place you at the forefront of AI innovation, providing access to our unique human data platform and opportunities for groundbreaking research.

Join us to enjoy a competitive salary, benefits, and remote working within our impactful, mission-driven culture. At Prolific, our compensation packages for eligible roles include base salary, equity, and benefits. Many roles also include the opportunity to earn a cash variable element, such as a bonus or commission. Each job posting shows a salary range that reflects the minimum and maximum target for new hires, based on the role’s location as well as your skills, experience, and relevant education or training. Your recruiter will also be happy to share the specific salary range for your preferred location during the hiring process.