Principal AI Engineer

• Stay hands-on and close to the work: run direct ideation and feedback loops with Prodigy's end users (abstractors, registry, and quality teams) and with production analytics and monitoring systems — turning real usage signals into prioritized improvements that demonstrably move value, not just model metrics.
• Evolve the feedback loop architecture that captures human corrections and routes them into continuous model improvement — ensuring Prodigy gets measurably better with every customer interaction.
• Define the evals framework for Prodigy: how accuracy is measured, how regression is detected, and what signals trigger retraining or prompt revision.
• Establish guardrails for hallucination detection and factual grounding specific to clinical registry use cases, where answer accuracy has direct downstream compliance implications.

• Architect AI infrastructure across GCP (Vertex AI, BigQuery, Dataflow) and AWS (Bedrock), ensuring the pipeline is scalable, cost-efficient, and operationally observable.
• Collaborate with data engineering to maintain high-quality, well-governed clinical and FHIR data inputs; define feature engineering and chunking strategies that optimize retrieval precision.
• Define MLOps standards for Prodigy: model versioning, deployment gates, rollback procedures, drift monitoring, and audit trail requirements consistent with HIPAA compliance.

• Act as the AI technical mentor for the Prodigy squad and adjacent engineering teams — guiding developers on RAG patterns, LLM integration, responsible AI practices, and clinical data handling.
• Collaborate with Security and Compliance to ensure Prodigy's AI layer meets HIPAA requirements, including PHI handling in prompts, data residency, and model audit logging.
• Foster AI literacy across the broader engineering organization, helping teams understand when and how to apply AI safely in a regulated healthcare context.
• Partner with Product Management to translate registry workflow complexity and customer feedback into technically sound AI capability improvements.

• Bachelor's in Computer Science, AI/ML, or related field; Master's or PhD preferred — or equivalent depth proven through shipped AI systems.
• Strong ML / data science / statistics theory foundation with the ability to read research, assess applicability, and execute.

• Hands-on LLM integration: prompt engineering, grounding, citation, hallucination mitigation, and output validation at clinical accuracy standards.
• Experience with LangChain, LlamaIndex, or equivalent orchestration frameworks.
• Built confidence scoring and auto-acceptance thresholds that govern when answers route to human review.
• Designed human-in-the-loop feedback systems that capture corrections and feed them back into model improvement.
• Production experience building RAG pipelines — document ingestion, chunking, embedding model selection, vector store management, and retrieval evaluation.

• Full ML lifecycle ownership in production: versioning, deployment gates, drift monitoring, rollback, and audit trails.
• Strong Python and software engineering fundamentals — CI/CD, testing, code review.
• Hands-on with vector databases (pgvector, Pinecone, Weaviate, or equivalent) and hybrid search.
• Built evals frameworks that measure accuracy, precision, recall, and F1 to inform product decisioning

• Solid AWS and GCP experience: Bedrock, SageMaker, Vertex AI, BigQuery, Dataflow.
• Azure familiarity a plus.
• Experience building pipelines over mixed unstructured and structured data sources.
• FHIR/HL7 and clinical document format familiarity strongly preferred.

• Clinical NLP or healthcare AI experience — medical terminology, document structure, and regulated accuracy standards are not new territory.
• Prefer direct experience with clinical documentation and abstraction workflows
• Knows how HIPAA applies to AI systems specifically: PHI in prompts and embeddings, data residency, audit logging, de-identification.
• Familiar with AI governance in practice: bias detection, explainability, responsible AI in compliance-sensitive contexts.

• Has owned AI technical vision before — not just contributed to it.
• Can write an ADR, set an engineering standard, and make it stick across teams.
• Communicates tradeoffs clearly to both engineers and non-technical stakeholders.
• Track record of mentoring engineers and raising AI maturity on a team.

Total Compensation Base pay is one element of the total compensation package.

Eligible employees may also receive an annual cash bonus and have access to a comprehensive benefits offering, including medical, dental, vision, life insurance, and a 401(k) plan.

Range It is not typical for an individual to be hired at or near the top of the range.

Individual pay may be influenced by factors such as skills, qualifications, experience, licensure, certifications, geographic location, and internal equity.