Senior Software Engineer, AI Benchmarking

Full-time IT & Technology

Responsibilities

• Perform pre-release and post-release assessments of frontier and open-source AI models using SecureBio's suite of biosecurity evaluations
• Build adapters for models, identify and troubleshoot unexpected agent behaviors, and ensure methodological rigor
• Build, scale, maintain, and continuously improve SecureBio's suite of biosecurity evaluations
• Improve evaluations to use state-of-the‑art agent frameworks, jailbreaking methods, and elicitation strategies
• Develop and improve internal tools that enable researchers to run capability assessments at scale
• Contribute to and maintain cloud infrastructure to run evaluations and analyses
• Contribute analyses to the public dashboard of trends in AI biology capabilities

Requirements

• 5+ years of experience in software engineering (industry or open‑source)
• Fluency in Python
• Proficiency with Docker and AW...