{bc}
linkedin

Senior Applied Research Scientist

ServiceNow
Montreal, CAN
Full Time
Senior
4 days ago
Large Language ModelsPrompt EngineeringEval EngineeringAgent OrchestrationDistributed SystemsAPI Design
Free

Job Fit Check

Base Career helps you apply smarter for this job.

?%
Ready to Scan

Key skills for this role

Large Language ModelsPrompt EngineeringEval Engineering
Smart Apply

Full Job Posting

Company Description

  • ServiceNow is the AI control tower for business reinvention, helping 85% of the Fortune 500 work smarter, faster, and better.
  • We are building an AI native culture where technology and talent are unstoppable together.

About the Team

  • The Agentic Engineering org builds a conversational AI experience that turns enterprise intent into completed work.
  • The Agent Orchestration team owns the execution core: agent harness, orchestration runtime, multi agent coordination, memory management, and evaluation frameworks.

What You'll Do

  • Harness engineering: Design and build the agent execution harness — the orchestration layer that routes inputs, manages context, invokes tools, handles retries, and surfaces execution state.
  • Reliability at scale: Own the runtime's fault tolerance, latency, and throughput; design for enterprise workflows that cannot fail silently.
  • Observability: Instrument the harness with tracing, cost attribution, and latency visibility.
  • Prompt infrastructure: Build prompt management systems — versioning, templating, and systematic evaluation.
  • Eval engineering: Design and own evaluation frameworks (unit evals, integration evals, production monitors).
  • LLM integration: Integrate with and abstract over frontier LLMs, managing model routing, fallback strategies, cost, and latency tradeoffs.
  • Technical leadership: Raise the technical bar through architecture decisions, code reviews, and coaching.
  • System boundary design: Define where agent logic lives — tool call, sub agent, hardcoded path, or human escalation.

Qualifications

  • 4+ years building production software systems with a strong track record on reliability, performance, and scalability.
  • Hands on experience shipping generative AI products — not just integrating LLM APIs or building prototypes.
  • Solid depth in how large language models work: failure modes, context constraints, and how prompt design shapes model behavior.
  • Practical prompt engineering experience: systematically designing, versioning, and evaluating prompts.
  • A real track record in eval engineering — a portfolio of evaluation suites designed, shipped, and used to drive quality decisions.
  • Cost and efficiency awareness at the system level: experience reasoning about model routing, inference cost, and latency tradeoffs.
  • Strong software engineering fundamentals: distributed systems, API design, and testing discipline.
  • Comfort operating in fast moving, ambiguous, startup like AI product environments.

Nice to Have

  • Experience with multi agent coordination patterns (A2A, MCP).
  • Familiarity with agent frameworks (LangChain, LlamaIndex, or similar).
  • Prior experience shipping AI systems in enterprise software.
  • Experience with AI observability tooling (tracing, cost tracking, LLM specific monitoring).
  • Familiarity with cloud native infrastructure, service observability, logging, monitoring, reliability engineering, and production troubleshooting.

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at ServiceNow