Senior Life Sciences Analyst, AI Model Training

About Norstella:

Job Description:

Responsibilities:

• Evaluate model outputs across clinical, regulatory, and life sciences tasks by rating individual responses, ranking pairs and lists, and rewriting outputs to demonstrate the preferred response.
• Define and maintain evaluation rubrics, preference dimensions, and quality criteria (factual accuracy, clinical reasoning quality, etc) that govern model outputs and preference alignment.
• Act as an expert-in-the-loop for how life science models are commercialized with clients, including ongoing support for implementation.
• Advise and assist with interpretation of model behavior, bridging the desired behavior back to the dataflow inflection point, working with the relevant teams to adjust as needed.
• Partner with data scientists and MLEs to walk feedback into data collection pipelines.
• Conduct red-team and adversarial evaluation of models, surfacing subtle failure modes — hallucinations, citation errors, unsupported clinical claims, regulatory missteps - that automated metrics cannot reliably catch.
• Analyze preference data and model behavior to identify systematic gaps, then partner with the fine-tuning SMEs and the data science team to decide how each gap is addressed through preference data.
• Conduct new proofs of concept for novel domain capabilities.
• Contribute to Norstella’s knowledge graph and taxonomy work and help design new agentic workflows based on domain-grounded language models.

Qualifications:

• Graduate degree in life sciences, medical sciences, computer science or equivalent professional experience.
• At least 3 years of professional experience in production-grade life science datasets, including with AI-enabled applications.
• Experience working with structured publishing platforms and data tools; comfort with automation concepts
• Experience working with and statistically analysing large and complex data sets, including data cleaning and preprocessing.
• Excellent problem-solving skills and the ability to work independently.
• Excellent communication skills, especially between technical and non-technical teams.

Preferred:

Our Guiding Principles for success at Norstella:

Benefits:

• 25 days annual leave, 4 days for volunteering and a personal day.
• 5% pension match.
• Group Life Assurance (100% employer funded).
• Group Income Protection (100% employer funded).
• Other voluntary benefits such as: Dental, Cash Plan, PMI Excess Cover, Health Screening & Critical Illness.