ashby
Backend Engineer
LiteLLM
San Francisco, USA
Full-time
Senior · 1+ years experience
Onsite
USD 170000-230000 yearly / year
Discovered 1 weeks ago
PythonFastAPIFlaskDatadogSplunkPrometheus
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
PythonFastAPIFlask
Role Overview
You’ll focus on owning our guardrails and logging world-class. You will be in charge of the backend code that ensures all guardrail calls are consistently logged, errors are surfaced to users (not silently swallowed), and our observability instrumentation works for real-world, high-volume traffic. Your attention to detail in areas like latency metrics, logging traceability, and backend guardrail registration will directly impact user trust in our security and compliance features.
Key Skills for This Role
PythonFastAPIFlaskDatadogSplunkPrometheus
Full Job Posting
Backend Engineer
LiteLLM is the world's most popular AI Gateway, trusted by top companies like Adobe, Netflix, and NASA. Our platform empowers developers by providing secure, reliable access to LLMs and adjacent services, and we're looking for a Senior Backend Engineer to help us build rock-solid guardrails and observability tooling at scale.
About The Role
You’ll focus on owning our guardrails and logging world-class. You will be in charge of the backend code that ensures all guardrail calls are consistently logged, errors are surfaced to users (not silently swallowed), and our observability instrumentation works for real-world, high-volume traffic. Your attention to detail in areas like latency metrics, logging traceability, and backend guardrail registration will directly impact user trust in our security and compliance features.
Responsibilities
- Build and scale our product, ensuring performance, reliability, and continuous improvement.
- Ensure all guardrail and policy enforcement calls (e.g., applyguardrail ) are properly logged and traceable through our SpendLogs and relevant database tables
- Build and design CPU-level guardrails to cover common attacks on LLM API's / MCP servers / Agents
- Identify and fix areas where silent failures occur in guardrail creation, registration, and policy application—ensuring robust error handling and transparency to end users
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at LiteLLM
Business Manager
, USA
Discovered 2 days agoFull-time
Rust Engineer
San Francisco, USA
Discovered 1 weeks agoFull-time
Account Executive
, USA
Discovered 1 weeks agoFull-time
Staff Engineer
San Francisco, USA
Discovered 1 weeks agoFull-time
Founding Developer Relations
San Francisco, USA
Discovered 1 weeks agoFull-time
Forward Deployed Engineer
San Francisco, USA
Discovered 1 weeks agoFull-time
Technical Account Manager
San Francisco, USA
Discovered 1 weeks agoFull-time
Forward Deployed Engineer (New Grad)
, USA
Discovered 3 weeks agoFull-time