Senior Software Engineer - LLM Ops & Evals

Audit and finance are still massively manual and we are changing that. DataSnipper is a $1B, bootstrapped unicorn with 600,000+ users across 180+ countries, already embedded in the daily workflows of top audit and accounting firms. Now, we are taking things further with our Excel Agent, bringing AI directly into where the work actually happens. Unlike generic AI tools, we do not sit on the sidelines. Our AI operates inside Excel, with access to real documents and audit evidence, meaning it does not just generate answers, it does the work, with full traceability. We are not just applying AI, we are redefining how audit gets done. If you want to build something category-defining at scale, this is the place.

Own the LLM gateway: routing, provider failover, rate limits, retries, and cost attribution across multiple model providers

Deploy, version and deprecate models across clouds, regions and environments, including quota and capacity planning, managed as infrastructure as code

Build the shared evaluation platform: versioned datasets, experiment tracking, run and result schemas, reporting, and trace linkage back to the run

Own the infrastructure for async and long-running AI workloads

Own observability for AI traffic: latency, retries and fallbacks, token usage, cost and errors, per team and per use case

Take part in the on-call rotation, run incidents, and close the follow-ups

Implement the security and compliance controls the platform is held to: retention, access control, RBAC and SSO

Define and maintain clean integration contracts between the platform and the teams that consume it

Partner with product and ML engineers to turn their requirements into platform capabilities that are self-service rather than a request queue

Experience with LLM or agent evaluation

Temporal or another durable workflow engine

Self-hosted inference, capacity planning, load testing

Synthetic data or document anonymisation pipelines

Document AI: VLMs, OCR, structured extraction and the metrics that go with it

Domain experience in audit, accounting or fintech

Ownership: You own work end-to-end, anticipate issues, and ensure high-quality delivery without close supervision

Growth Mindset: You encourage open feedback exchange and provide clear, balanced feedback that helps others grow

Collaboration: You build strong cross-functional relationships and influence peers through expertise, data, and empathy

Adaptability: You navigate ambiguity calmly, model positive behavior, and help peers adjust through clear communication

Judgment: You exercise sound judgment in ambiguous situations, balance speed and accuracy, and adjust priorities proactively

Recruiter screen

Hiring Manager interview

Peer programming session

System design interview

Final interviews with Engineering leadership