Job Description:

Role & Responsibilities:

Responsibilities:

- Lead the architecture and delivery of certification, regression, and validation systems for Windows AI platform capabilities.

- Define the end-to-end engineering strategy, including test taxonomy, certification criteria, coverage expectations, baseline management, release gates, and evidence required for ship decisions.

- Create scalable patterns for onboarding models, APIs, runtimes, devices, and application scenarios into common regression and validation infrastructure.

- Drive automation across CI/CD, pre-merge, scheduled, release, and servicing workflows while preserving reproducibility, traceability, and actionable diagnostics.

- Define representative validation strategies across device classes, silicon configurations, release channels, languages, workloads, and customer scenarios while maintaining consistent certification standards.

- Establish approaches for model-output regression, functional correctness, performance, power, memory, reliability, compatibility, security, privacy, accessibility, and responsible AI validation.

- Lead complex investigations across model, runtime, API, OS, driver, hardware, and application boundaries and turn findings into durable engineering improvements.

- Improve regression-suite health through coverage analysis, regression-signal measurement, flaky-test reduction, threshold governance, and disciplined ownership models.

- Partner across organizational boundaries to align release criteria, dependencies, validation responsibilities, and risk decisions.

- Raise engineering standards through design reviews, code reviews, mentoring, operational readiness, and clear communication of release risks and outcomes.

Required Qualifications :

- 8+ years of professional software development, test-platform development, or validation-infrastructure engineering experience.

- Strong proficiency in one or more programming languages such as C++, C#, Python, or equivalent.

- Demonstrated experience designing and shipping complex test platforms, CI/CD systems, developer infrastructure, distributed systems, or production software.

- Strong understanding of software architecture, automated testing, debugging, reliability engineering, data analysis, and source-control practices.

- Proven ability to lead technical execution across multiple contributors or partner teams and deliver through ambiguity.

- Strong communication skills with the ability to explain release confidence, release risks, trade-offs, and recommendations to engineering and leadership audiences.

Preferred Qualifications :

- Deep experience with AI/ML evaluation systems, model-performance measurement, regression infrastructure, benchmark datasets, or certification frameworks.

- Experience defining evaluation metrics, acceptance thresholds, validation matrices, regression gates, release criteria, and risk-based test strategies.

- Experience with Azure, cloud-native systems, distributed test execution, CI/CD, telemetry, observability, security, and production operations.

- Experience with Windows development, Visual Studio, Windows SDK, WinAppSDK, MSIX, systems programming, Windows servicing, OS integration, compatibility, or multi-device validation.

- Experience building platforms that support self-service onboarding, reusable test contracts, result analysis, and release-readiness dashboards.

- Working knowledge of responsible AI, privacy, security, accessibility, compliance, and data-governance requirements.

What Success Looks Like :

- Establishes shared certification and regression foundations adopted across multiple AI platform capabilities.

- Creates clear, evidence-based release gates that identify regressions early and support consistent ship decisions.

- Improves regression signal, coverage, reliability, and execution efficiency across devices and scenarios.

- Identifies systemic risks and validation gaps early and drives pragmatic resolution across teams.

- Builds durable ownership, operational, and engineering practices that allow the regression suite to scale.

- Grows team capability through technical leadership, mentoring, and a clear validation and release-engineering strategy.

Total Experience : 8 - 10 years