Robot data quality assurance. Review videos and associated annotations from a high mix of industrial robot tasks. Identify missing, inaccurate, or inconsistent annotations and determine whether collected data meets established quality standards.
Annotation feedback and improvement. Provide clear, structured feedback to internal and external annotation teams. Track recurring sources of error and help ensure that feedback leads to measurable improvements in annotation quality.
Annotation guidelines. Help design, test, and refine annotation instructions, examples, rubrics, and edge-case guidance so that annotators can make consistent decisions at scale.
Pilot observation and coaching. Observe teleoperation pilots as they train on new tasks, identify performance gaps, and provide timely, actionable coaching.
Pilot readiness and evaluation. Help evaluate whether pilots are ready to perform tasks independently. Support the development of task-specific training materials, grading criteria, and certification processes.
Quality monitoring and reporting. Track data-quality and pilot-performance metrics, investigate unexpected changes, and communicate findings to operations and technical stakeholders.
Process improvement. Identify inefficient or unreliable parts of the data collection and review process, propose improvements, and help implement systems that maintain quality as operations scale.
Cross-functional collaboration. Partner with the Autonomy team to understand which mistakes matter most for model training and evaluation, then translate those needs into practical guidance for pilots and annotators.
What we’re looking for
Exceptional attention to detail and the ability to maintain consistent judgment while reviewing large volumes of complex information.
Strong written and verbal communication skills, especially the ability to explain errors clearly and provide direct, constructive feedback.
Comfort reviewing repetitive work without losing focus, while still recognizing unusual edge cases and broader patterns.
A systems-oriented mindset: you look beyond individual mistakes to identify why they are happening and how the process can be improved.
Comfort working with technical tools, structured data, spreadsheets, dashboards, and unfamiliar software systems.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
Good judgment and a willingness to make decisions when guidelines do not perfectly cover the situation.
An interest in robotics, artificial intelligence, industrial operations, or the role high-quality data plays in improving machine-learning systems.
A hands-on attitude. You are willing to perform a process yourself, understand it deeply, and help improve it before attempting to automate or delegate it.
Comfort traveling to Mexico on a semi-regular basis, typically once a month or every other month, to work directly with pilot and data-operations teams.
Nice to have
Experience in data operations, quality assurance, data annotation, technical operations, robotics operations, manufacturing, logistics, or another detail-oriented operational environment.
Experience reviewing or producing labeled datasets for machine learning.
Experience in robotics, teleoperation, manufacturing, warehouse operations, or industrial environments.
Experience training, coaching, or evaluating operators.
Experience working with external annotation or operations vendors.
Familiarity with data-quality metrics, sampling methods, inter-annotator agreement, or quality-control workflows.
Spanish-language proficiency.
What success looks like
About Ultra
Software & SaaS
Ultra is a private AI networking platform that connects users with co-founders, investors, experts, and talent.