AI Systems Engineer – LLM Evaluation & Accuracy

BE / MTech in Computer Science, AI/ML, Computational Data Science (CDS), Computer Science & Automation (CSA), or related discipline.

Range 4–6 years total; minimum 2 years building production LLM or AI systems; minimum 2 years in evaluation, quality measurement, or accuracy improvement work.

"Must-have" Skills

• Production-grade Python — clean, tested, maintainable systems; not just scripts (pytest, FastAPI or Flask)
• Hands-on LLM API experience (OpenAI, Anthropic, Gemini, AWS Bedrock or equivalent) with systematic, measurement-driven prompt engineering — methodology over instinct
• Agentic pipeline design — multi-step reasoning, tool use, orchestration frameworks (LangChain, LlamaIndex or equivalent), automated evaluation and feedback loops
• Evaluation framework design for LLM systems — precision/recall/F1, confusion matrices, A/B testing, per-class error analysis
• Analytical depth sufficient to design meaningful accuracy metrics and interpret why a model fails on a specific document or field type
• MongoDB or equivalent NoSQL — queries, aggregations, indexing pandas / numpy for data processing and batch analysis
• Git, code reviews, CI/CD basics (GitHub Actions or Jenkins)
• Clear written communication — able to explain model behaviour and accuracy findings to non-technical stakeholders

"Would-be-nice" Skills

• "Document AI: PDF parsing, layout-aware extraction, OCR, structured form extraction
• RAG pipeline design and vector search (Pinecone, Weaviate, or similar)
• Classification systems with large label spaces (50+ classes)
• Async Python (asyncio, aiohttp) for pipeline throughput
• Embedding models and semantic similarity for document matching
• Prior experience working alongside a Research or Applied Science team as the engineering counterpart"

Working Knowledge (Tools) Python, FastAPI / Flask, MongoDB, Git, GitHub Actions / Jenkins, LLM APIs (OpenAI / Anthropic / Gemini or equivalent), LangChain / LlamaIndex, pandas / numpy, pytest, Docker

NLP concepts, LLM prompt engineering patterns, REST APIs, RAG pipelines, vector databases, JSON data structures

Agentic workflow design and orchestration, LLM evaluation metrics (F1 / Precision / Recall, per-class analysis, confusion matrices), production Python systems (error handling, retries, logging, monitoring), NoSQL aggregations, systematic A/B testing for model changes, prompt optimization methodology

By submitting your application, you agree that your personal information and resume may be collected, processed, and stored by us for recruitment purposes, including consideration for future roles.