Member of Technical Staff, Agents

About the Role

Join a small, senior team building autonomous AI agents that handle real executive workflows. You will design new capabilities, measure their reliability, and improve them using evidence from real conversations before changes reach users.

What You'll Do

• Build agents that complete delegated work end to end with minimal user oversight.
• Define trustworthy measures of agent quality and evaluate performance across representative tasks.
• Develop new agent capabilities for tools people already use, and verify they are ready before launch.
• Investigate production failures and deliver durable, measurable fixes.
• Build systems to detect known failure patterns and less predictable issues at scale.
• Improve agent recovery when tools or websites fail.
• Create repeatable evaluation practices so engineers can assess whether agent changes improve outcomes.

What We're Looking For

• At least 3 years in data science or machine learning, building or delivering evaluation systems, metrics, or product quality measurement.
• Strong production proficiency in Python and SQL, including analysis of large datasets and automated pipelines.
• Experience designing evaluation frameworks with measurable criteria, representative datasets, and appropriate metrics.
• Experience building or operating agents for real users and improving their reliability.
• Familiarity with statistics, experiment design, and realistic end-to-end simulations.
• Experience owning systems through production launch and measuring their impact.
• Experience with post-training, fine-tuning, or reinforcement learning is relevant.
• Working knowledge of Django, React, and TypeScript is useful.
• Typically suited to candidates with fewer than 5 years of engineering experience and a track record of progression or startup ownership.