Senior Software Developer, AI Agents

Role overview

Responsibilities

• Build the Python agent harness with orchestration, tool execution, memory, context management, structured outputs, and failure handling.
• Create evaluation datasets, automated evaluators, and LLM-as-judge pipelines using LangSmith.
• Diagnose agent failures, tune prompts and tools, and run model comparisons and benchmarks.
• Build Python and Node.js microservices for agent hosting and LLM interaction.
• Develop Laravel PHP APIs that integrate agents into the platform.
• Deploy secure, scalable infrastructure using AWS Lambda, Bedrock, and CDN delivery.
• Optimize MySQL queries and indexing, participate in code reviews, collaborate cross-functionally, and mentor engineers.

Required qualifications

• Have 6+ years of backend software development experience.
• Have production experience building LLM-powered agents with tool calling, orchestration, context management, and prompt engineering.
• Have experience evaluating LLM systems through dataset curation, automated evaluators, LLM-as-judge, and regression tracking.