Senior Technical Program Manager, Human-in-the-Loop Operations

• 5+ years of experience in Technical Program Management, Data Operations, or Operations Management within an AI/ML or data-intensive environment.
• Deep hands-on experience with data labeling, annotation, or model evaluation — including designing annotation schemas, writing labeling guidelines, and managing quality control at scale.
• Proven experience managing external vendor relationships and distributed contractor workforces, including workforce planning, quality oversight, and budget management.
• Strong understanding of the ML lifecycle and the role of human-labeled data in training, fine-tuning, and evaluating models.
• Proficiency in SQL and Python for querying datasets, analyzing label quality, and building operational dashboards. At a minimum, be able to use AI tools to generate queries.
• Excellent written and verbal communication skills — able to write rigorous technical specifications and present complex data concepts to both ML researchers and business stakeholders.
• Proven ability to drive cross-functional alignment and exercise strong judgment, knowing when to champion collaborative efforts versus leading independently.
• Exceptional critical thinking skills with a demonstrated capacity to navigate ambiguity and execute effectively in 0-to-1 environments without structured guidance.
• Solid grasp of software development life cycles and hands-on experience leveraging AI-assisted tools like Cursor, Claude, or Gemini to accelerate workflows.

Nice to Have

• Experience with LLM-based labeling pipelines, LLM-as-a-Judge evaluation, or human-AI calibration loops.
• Familiarity with labeling platforms such as Label Studio, Scale AI, or Snorkel AI.
• Experience evaluating 3D, games, videos, or multimodal data beyond standard text and image annotation.
• Background in vendor platform evaluation or build vs. buy analysis.
• Experience with data operations at a consumer platform operating at massive scale (100M+ users).
• Knowledge of Roblox platform and its games, or general gaming experience.

For roles that are based at our headquarters in San Mateo, CA: The starting base pay for this position is as shown below.

The actual base pay is dependent upon a variety of job-related factors such as professional background, training, work experience, location, business needs and market demand.

Therefore, in some circumstances, the actual salary could fall outside of this expected range.

This pay range is subject to change and may be modified in the future.

All full-time employees are also eligible for equity compensation and for benefits as described on this page.

Roles that are based in an office are onsite Tuesday, Wednesday, and Thursday, with optional presence on Monday and Friday (unless otherwise noted).

Roblox provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws. Roblox also provides reasonable accommodations to candidates with qualifying disabilities or religious beliefs during the recruiting process.

For US based roles only, please note the Company may not be able to employ candidates for this role who have United States work authorization related to certain U.S. visa categories, or support future H-1B sponsorship at this time.