Senior Quality Assurance Engineering Lead

About Us

We are an innovative AI startup with a team of 100+ people focused on transforming core enterprise functions in Fortune 500 companies. Our platform combines a rich understanding of a company’s current state with deep domain expertise to execute these functions with accuracy and consistency through deterministic code, statistical methods, and guided ML and LLM execution. It handles processes end-to-end, engaging humans when needed to maximize efficiency and quality.

Join us at the forefront of AI innovation, where your expertise will directly shape the future of enterprise operations.

Role Overview

We are looking for a hands-on Senior Quality Assurance Engineering Lead to lead our QA team and ensure the quality of processes and outputs across our enterprise platform. You will set the team’s priorities, define our quality strategy, build scalable test automation, and establish standards for validating end-to-end processes and the accuracy, completeness, and consistency of their outputs.

Working closely with engineering, product managers, and domain experts, you will validate how the platform uses company state and domain knowledge across deterministic code and guided LLM execution. This role combines team leadership, practical test engineering, and mentorship to ensure our solutions are reliable, secure, and ready for production.

Key Responsibilities

• Process and Output Quality: Own the quality strategy across the development lifecycle, working with product managers and domain experts to define test plans, acceptance criteria, and release standards. Validate that enterprise processes use the correct company state and domain rules to produce accurate, complete, and consistent outputs.
• Develop evaluation frameworks and datasets to validate business logic, accuracy, grounding, retrieval relevance, and consistency. Test the handoffs between deterministic code and guided LLM execution, including state updates, failure recovery, and human review paths.
• QA Team Leadership: Lead and mentor the QA team, set priorities, coordinate testing activities, and develop the team’s capabilities to ensure the quality of processes and outputs.
• Test Automation and Continuous Delivery: Build and maintain Python-based automation for APIs, services, user interfaces, and end-to-end workflows, integrating tests and quality gates into CI/CD pipelines.
• Reliability and Continuous Improvement: Lead validation of Azure-based applications for scalability, performance, resilience, data integrity, and security. Partner with engineering to resolve defects, prevent regressions, and use quality metrics to improve processes and outputs.

Qualifications

• Experience
• 6+ years of experience in quality assurance or quality engineering, including substantial hands-on test automation experience.
• 2+ years of experience leading QA teams, setting testing priorities, establishing quality standards, and mentoring engineers.
• Experience validating end-to-end business processes and assessing the accuracy, completeness, and consistency of their outputs.
• Hands-on experience evaluating AI/ML or Generative AI applications, preferably platforms that combine deterministic business logic, company data, and guided LLM execution.
• Experience testing cloud-native applications, preferably on Azure.
• Education
• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
• Technical Skills
• Experience validating business rules, company-state representations, state transitions, and data consistency across deterministic and LLM-supported processing.
• Experience with API testing, integration testing, end-to-end automation, and browser testing tools such as Playwright or Selenium.
• Strong proficiency in Python and automated testing frameworks such as pytest or equivalent tools.
• Experience creating evaluation datasets, scoring rubrics, regression benchmarks, and repeatable evaluation workflows.
• Experience with performance testing, observability, and debugging distributed applications.
• Familiarity with Azure services, including Azure Functions, Azure Databases, and Azure AI Foundry.
• Soft Skills
• Strong team leadership skills, with the ability to set clear expectations, coach engineers, and build shared accountability for quality.
• Excellent communication skills, including the ability to explain quality risks and release tradeoffs clearly.
• Strong analytical judgment and a practical approach to problem-solving.
• Ability to balance release speed with reliability and prioritize testing according to business impact.
• Comfort working in a dynamic startup environment with evolving requirements.