smartrecruiters
Senior Software Engineer, Systems Infrastructure - Agent Evaluation
LinkedIn
Toronto, CAN
Full-time
Senior · 2+ years experience
Hybrid
CAD 132000-179000 yearly / year
Discovered 1 weeks ago
JavaC++PythonGoRustC#
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
JavaC++Python
Key Skills for This Role
JavaC++PythonGoRustC#
Full Job Posting
Responsibilities
- Own the technical vision, architecture, and execution of the Evaluation Operating System (EOS), solving complex, open-ended challenges at the intersection of distributed systems, data infrastructure, and machine learning.
- Design and build large-scale evaluation infrastructure that enables LinkedIn teams to measure, understand, and continuously improve the quality, reliability, safety, and performance of AI agents and GenAI products.
- Work on reliable and scalable Tracing Infrastructure for LinkedIn AI Agents along with trace debuggability features.
- Architect scalable data pipelines and platforms for capturing, processing, labeling, and managing large volumes of AI interactions, evaluation data, golden datasets, and synthetic data.
- Build and evolve evaluation systems powered by LLM-as-judge, reward models, and other automated evaluators to assess AI systems across multiple dimensions of quality and performance.
- Develop experimentation and testing frameworks, including adversarial testing, champion/challenger experiments, and agent arena capabilities, to identify weaknesses and drive continuous improvement of AI systems.
- Establish real-time observability, monitoring, and feedback loops that detect regressions, model drift, quality degradation, and unexpected behavior in production AI systems.
- Partner closely with AI product teams, ML engineers, and infrastructure organizations to integrate evaluation deeply into the AI development lifecycle and establish consistent evaluation standards across LinkedIn.
- Lead multiple high-impact, cross-functional initiatives, influencing technical strategy and architectural decisions across AI Platforms and the broader engineering organization.
- Mentor and develop engineers, raise the technical bar, and help shape the engineering culture and practices of a growing AI platform organization.
- Build and Platformitize Recursive Self Improving Agents
Basic Qualifications
- Bachelor’s Degree in Computer Science or related technical discipline, or equivalent practical experience
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at LinkedIn
Senior Sales Performance Consultant
Chicago, USA
SeniorFull-time
This role will be based in San Francisco, Sunnyvale, Chicago, New York City. At LinkedIn, our approach to flexible work is centered on trust and optimized for culture, connection, clarity, and the evolving needs of our b
Discovered TodayView →
Senior Sales Performance Consultant
Sunnyvale, USA
SeniorFull-time
This role will be based in San Francisco, Sunnyvale, Chicago, New York City. At LinkedIn, our approach to flexible work is centered on trust and optimized for culture, connection, clarity, and the evolving needs of our b
Discovered TodayView →
Regional Account Manager, LinkedIn Talent Solutions
New York City, USA
SeniorFull-time
At LinkedIn, our approach to flexible work is centered on trust and optimized for culture, connection, clarity, and the evolving needs of our business. The work location of this role is hybrid, meaning it will be perform
Discovered TodayView →
Sr Product Manager, Hiring Pro
Sunnyvale, USA
SeniorFull-time
This role can be based in San Francisco, Sunnyvale, New York. At LinkedIn, our approach to flexible work is centered on trust and optimized for culture, connection, clarity, and the evolving needs of our business. The wo
Discovered TodayView →
Senior Software Engineer, Systems Infrastructure
Bengaluru, IND
SeniorFull-time
This role will be based in Bangalore, India. At LinkedIn, our approach to flexible work is centered on trust and optimized for culture, connection, clarity, and the evolving needs of our business. The work location of th
Discovered TodayView →
Senior Deal Desk Strategist [Fixed Term Contract]
Bengaluru, IND
JuniorFull-time
At LinkedIn, our approach to flexible work is centered on trust and optimized for culture, connection, clarity, and the evolving needs of our business. The work location of this role is hybrid, meaning it will be perform
Discovered TodayView →
Senior Software Engineer - Fullstack (Productivity Engineering)
Bengaluru, IND
SeniorFull-time
Team Description: Our team builds and maintains the core products that power Global Sales Compensation, ensuring accurate, scalable, and compliant compensation processing. We also own and operate all Finance integrations
Discovered TodayView →
Mid Market Account Director, Sales Solutions
Chicago, USA
JuniorFull-time
At LinkedIn, our approach to flexible work is centered on trust and optimized for culture, connection, clarity and the evolving needs of our business. The work location of this role is hybrid, meaning it will be performe
Discovered TodayView →
Senior Sales Performance Consultant
Chicago, USA
Discovered TodayFull-time
Senior Sales Performance Consultant
Sunnyvale, USA
Discovered TodayFull-time
Regional Account Manager, LinkedIn Talent Solutions
New York City, USA
Discovered TodayFull-time
Sr Product Manager, Hiring Pro
Sunnyvale, USA
Discovered TodayFull-time
Senior Software Engineer, Systems Infrastructure
Bengaluru, IND
Discovered TodayFull-time
Senior Deal Desk Strategist [Fixed Term Contract]
Bengaluru, IND
Discovered TodayFull-time
Senior Software Engineer - Fullstack (Productivity Engineering)
Bengaluru, IND
Discovered TodayFull-time
Mid Market Account Director, Sales Solutions
Chicago, USA
Discovered TodayFull-time