Ai Evaluation Engineer

Firstignite

Full-time Other-General
Apply Now
Location
valle de chalco solidaridad, estado de mΓ©xico, Mexico
Posted
June 07, 2026

Job Description

OverviewAbout FirstIgnite: FirstIgnite is the AI-powered business development platform for university technology transfer offices (TTOs).
We help research institutions turn breakthroughs into partnerships, licenses, and companies by combining deep LLM-driven workflows with the relationships that actually move deals forward.
Our product suite spans expert discovery, grants search, and AI-driven outreach β€” all built on a modern, agentic stack.
We ship fast, we measure everything, and we believe evaluations are the difference between AI features that demo well and AI features that work in production.ResponsibilitiesBuild evaluation infrastructure: Design and maintain eval suites using Promptfoo, LLM-as-judge methodologies, and custom harnesses for features like our expert search system, natural language grants search, and AI SDR agents.Define what good means: Partner with product and domain experts to translate fuzzy customer outcomes (for example, +does this surface the ri...