Testing Lead-QA

Design tests for:

Prompt templates and prompt changes

RAG pipelines (retrieval quality, grounding, hallucination control)

Multi-turn conversations and long-context behaviour

Maintain golden datasets , regression test suites, and test result summaries.

Document prompt behaviour, edge cases, and known model quirks.

Test VLMs for image-text alignment, OCR, captioning, and reasoning.

Document model performance across different image types, quality levels, and domains.

Track and publish model behaviour changes between versions.

Build Python-based automation for evaluation and regression testing.

Integrate tests into CI/CD and MLOps pipelines .

Produce readable quality reports and dashboards for engineers and leadership.

Monitor and document production issues such as model/data drift and degradation .

Establish QA and documentation standards that scale with a startup.

Mentor engineers on writing testable code and meaningful documentation.

Act as the single source of truth for AI quality, testing, and known risks.

Experience with VLMs, multimodal models, or computer vision .

Exposure to RAG architectures , vector databases, and embeddings.

Familiarity with tools like LangChain, LlamaIndex, MLflow, or similar.

Experience documenting AI risks, limitations, or compliance requirements.