ashby
Senior MLOps Engineer
CreatorIQ
San Francisco, USA
Full-time
Senior
Hybrid
USD 198000-230000 yearly / year
Discovered 1 weeks ago
S3Label StudioPython
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
S3Label StudioPython
Key Skills for This Role
S3Label StudioPython
Full Job Posting
Senior MLOps Engineer (Applied AI Focus)
As a Senior MLOps Engineer on our Product Innovations team, you will serve as the technical lead for Applied MLOps, bridging the gap between experimental data science and production-grade efficiency. You will focus heavily on ground truth generation, model evaluation, and pre/post processing decisions around a scaled vector embeddings ecosystem.
In this role, you’ll get to
Architect Annotation & Measurement Pipelines: Own the design and implementation of human-in-the-loop and auto-annotation workflows. You will set up systems (e.g., Label Studio) to establish tight confidence metrics and rigorous Inter-Annotator Agreement (IAA).
Drive Cost-Efficiency via Evaluation: Create the ground truth, golden sets, and by-model evaluation criteria that allows us to benchmark models and make informed data science decisions. Your work will directly impact margin by enabling us to deploy the right model for the right task.
Enforce Applied MLOps Standards: Implement "model-as-a-judge" frameworks, deterministic PII scrubbing, and other best practices within our production pipelines to ensure our model-driven products are robust and compliant.
Collaborate on Model Strategy: Work closely with our Data Science team to make applied generative decisions that result in stronger product outcomes and cost efficiencies.
Build & Integrate: Collaborate with engineering to integrate measurement loops into our broader infrastructure (AWS/GCP), ensuring our model lifecycle is automated and observable.
Who you are and what you’ll need for this position
Annotation & Evaluation Expert: You have deep experience setting up annotation workstreams (using tools like Label Studio, Scale AI, or custom solutions) and know how to calculate and improve ground truth quality.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at CreatorIQ
Senior Fullstack Engineer, Agentic Experience
, USA
SeniorFull-time
CreatorIQ is the operating system for creator-led growth trusted by more than 1,300 global brands and agencies. We’re on a mission to make businesses more human, and humans more impactful. We operate by our values — be i
Discovered TodayView →
Senior Fullstack Engineer, Agentic Experience
, CAN
SeniorFull-time
CreatorIQ is the operating system for creator-led growth trusted by more than 1,300 global brands and agencies. We’re on a mission to make businesses more human, and humans more impactful. We operate by our values — be i
Discovered TodayView →
Vice President, GTM Operations
, USA
ExecutiveFull-time
CreatorIQ is the operating system for creator-led growth trusted by more than 1,300 global brands and agencies. We’re on a mission to make businesses more human, and humans more impactful. We operate by our values — be i
Discovered TodayView →
Director, Customer Success
, GBR
DirectorFull-time
CreatorIQ is the operating system for creator-led growth trusted by more than 1,300 global brands and agencies. We’re on a mission to make businesses more human, and humans more impactful. We operate by our values — be i
Discovered TodayView →
Product Education Lead
, USA
LeadFull-time
CreatorIQ is the operating system for creator-led growth trusted by more than 1,300 global brands and agencies. We’re on a mission to make businesses more human, and humans more impactful. We operate by our values — be i
Discovered TodayView →
Senior Fullstack Engineer, Agentic Experience
, USA
Discovered TodayFull-time
Senior Fullstack Engineer, Agentic Experience
, CAN
Discovered TodayFull-time
Customer Success Partner
Los Angeles, USA
Discovered TodayFull-time
Vice President, GTM Operations
, USA
Discovered TodayFull-time
Director, Customer Success
, GBR
Discovered TodayFull-time
Product Education Lead
, USA
Discovered TodayFull-time
Analytics Engineer
Los Angeles, USA
Discovered YesterdayFull-time
Customer Success Partner
New York City, USA
Discovered 2 days agoFull-time