Prompt Systems Engineer

Role overview

Accountabilities

• Design and maintain prompts for agents supporting enrollment, learning, retention, and student services.
• Translate educational and business requirements into precise agent behavior, personas, constraints, fallback handling, and multi-turn logic.
• Design chained agent workflows and RAG experiences using program-specific knowledge bases.
• Configure and deploy agents with engineering teams through an AI orchestration platform.
• Build evaluation frameworks for accuracy, tone, hallucination rates, task completion, and learning-objective alignment.
• Use observability platforms to monitor production performance and identify regressions.
• Develop adversarial testing, prompt versioning, reusable components, and prompt engineering standards.
• Collaborate with product, learning, enrollment, student success, engineering, and university stakeholders.
• Monitor model-provider capabilities and contribute to prompt automation, evaluation, and testing workflows.

Requirements

• 2+ years designing and iterating on prompts for production LLM-powered applications.
• Familiarity with few-shot prompting, system and user roles, output constraints, tool-use prompting, and RAG integration.
• Experience creating prompt evaluations, metrics, test cases, result analysis, and data-informed improvements.
• Experience with Langfuse, Weights & Biases, or similar LLM observability tools.
• Familiarity with LTI and AI integrations with learning management systems such as Moodle, Canvas, or Instructure.
• Ability to read and write basic JavaScript or Python for prompt automation and testing.
• Working knowledge of OpenAI, Anthropic, or comparable model-provider APIs.
• Strong written communication, analytical, problem-solving, organizational, and collaboration skills.
• Experience with voice agents or multimodal prompting is a plus.

Compensation

• Annual salary is $100,000–$120,000.

Workplace

• Fully remote work is available, with an option to work from a New York City office.

Benefits

• Benefits include company equipment, retirement matching, equity opportunities, medical, dental, vision, life and disability insurance, HSA and FSA options, and employee assistance resources.
• Additional offerings include paid parental leave, paid vacation, paid holidays, sick leave, wellness resources, commuter benefits, mobile connectivity support, and an annual education stipend.