Job description :

Role : QA Architect, Data Products

About Alegeus :

Alegeus powers the most configurable platform in consumer-directed healthcare benefits, serving employers, TPAs, and health plans that collectively deliver HSA, FSA, HRA, commuter, lifestyle, and emerging account-based benefits to tens of millions of Americans. Our partners depend on Alegeus for accuracy, uptime, and a consumer experience that performs flawlessly during the highest-stakes moments of someones financial and medical life.

The Role :

The QA Architect, Data Products is a senior individual contributor and technical lead who owns the quality of our data products end-to-end and shapes the testing strategy across application and AI products in partnership with the QA Manager.

What makes this role unusual is the combination it requires : the product intuition to understand what working correctly actually means for a benefits platform customer, the data engineering literacy to test complex pipelines, the AI-native fluency to use generative tools as a daily accelerator, and the hands-on automation depth to implement what you design. You will also mentor a small team of QA engineers - not as a manager, but as the most technically advanced person in the room.

If you have spent time thinking about quality from both a product outcome lens and a data systems lens, and you want to help define what AI-native testing looks like for a company that processes healthcare benefits at scale, this role was written for you.

What You Will Own :

Test Architecture & Strategy :

- Own the overall test strategy for data products from end to end - what gets tested, at which layer, with what data, and how results are validated against business expectations.

- Define and evolve the testing strategy for application and AI products in partnership with the other testers - covering test pyramid structure, tool selection, automation priorities, and release quality gates.

- Design the test architecture for system and component layers : what services, contracts, APIs, and data flows are verified where, and how test execution integrates into CI/CD.

- Bring a product thinking lens to all of it : validate outcomes, not just implementation. Ask whether the system is doing the right thing, not only whether it is doing the thing right.

Test Data, Synthetic Data & Configuration Variation :

- Architect the test data strategy : what data is needed at each test layer, how it is sourced, governed, refreshed, and compliant with HIPAA and PII constraints.

- Design and build synthetic data generation capabilities using AI tooling - creating realistic, edge-case-rich, compliant datasets that cover the breadth of customer scenarios without relying on production data.

- Own testing across the full range of customer configuration variations. Alegeuss platform is highly configurable across plan types, benefit rules, contribution limits, and employer and TPA setups. You will ensure test coverage spans the meaningful configuration space, not just the happy path.

- Develop parameterized and data-driven test frameworks that make configuration variation testing maintainable as the product and customer base grows.

Data Product Testing Ownership :

- Own the quality of all data products - data pipelines, ETL/ELT workflows, reporting outputs, data APIs, and data platform services - from requirements through production.

- Define and implement data quality validation : completeness, accuracy, consistency, timeliness, and schema integrity at rest and in transit.

- Build contract and schema tests that catch breaking changes at the source before they propagate downstream.

- Partner with data engineers on pipeline observability and production data monitoring so quality doesnt stop at the release gate.

AI & Application Product Testing :

- Oversee the testing strategy for AI/ML products : evaluation harnesses, offline and online evaluation, prompt regression, model bias and drift detection, and output quality scoring.

- Partner with ML engineers and product managers to define what good looks like for AI generated outputs in a benefits context - and build the tests that measure it.

- Contribute to system-level and integration-level test design for application products, ensuring the strategies you define are actually implemented and maintained.

AI-Native Testing Practice :

- Use AI tools daily : generate test cases from requirements and user stories, produce synthetic datasets on demand, run exploratory test agents, and use LLMs to accelerate test design and defect analysis.

- Be the practitioner the rest of the QA team learns from. Document your AI-assisted workflows, run internal demos, and build repeatable playbooks that others can adopt.

- Evaluate emerging AI testing tools actively; bring recommendations to the QA Manager and Senior Director with evidence, not opinion.

Hands-On Automation :

- Write automation at the system and component test levels yourself - this is not a role that delegates all implementation to others.

- Build reliable, maintainable test code that the team can own over time : clean interfaces, documented patterns, and automation that engineers trust.

- Contribute to the shared automation platform and CI/CD integration that the COE maintains across teams.

What You Bring :

- Experience. 7-10 years in software quality engineering with clear progression into technical leadership or architecture. You have designed test strategies, not just executed them.

- Product background. You have spent meaningful time working from a product outcome lens - validating business behavior, partnering with product managers, and thinking about what the customer actually experiences. Prior experience as a QA lead embedded in a product team, or a background that includes product testing, QA ownership of a product area, or time working alongside PMs on UAT.

- Data product testing. Hands-on experience testing data pipelines, ETL/ELT workflows, data APIs, or reporting products. You understand schema validation, data quality frameworks, and what it means to test data correctness at each stage of a pipeline.

- Synthetic data and test data architecture. Experience designing test data strategies and building or using synthetic data generation - ideally with AI tooling. You understand the constraints that HIPAA and PII place on test data in a healthcare benefits context.

- Configuration-variation testing. Experience testing highly configurable or multi-tenant platforms where customer setups vary significantly. You have built frameworks that handle this systematically.

- Automation depth. Strong hands-on automation skills at system and component levels. You write clean, maintainable test code in at least one major language and framework (Python, Java, TypeScript, Playwright, Pytest, TestNG, or similar).

- AI-native, not AI-adjacent. You are using AI tools actively in your testing practice today : generating test cases, producing synthetic data, using LLMs in exploratory and analytical tasks. You have opinions formed from daily use, not vendor demos.

- AI and ML product testing. Exposure to evaluating AI or ML model outputs : evaluation harnesses, prompt regression, drift or bias detection. You do not need to be an ML engineer, but you need to work productively with them.

- Education. Bachelors in Computer Science, Engineering, Information Systems, or equivalent practical experience.

Nice to Have :

- Experience with benefits administration, HSA/FSA/HRA, payments, or similar multi tenant financial or healthcare SaaS.

- Familiarity with data quality tools such as Great Expectations, dbt tests, Monte Carlo, or similar.

- Hands-on with ML evaluation frameworks such as DeepChecks, Giskard, Ragas (for RAG pipelines), or custom eval harnesses.

- Prior experience in a formal QA Center of Excellence or platform quality role.

- Contributions to open-source testing tools, conference talks, or written work on AI-native testing practices.

Why This Role Matters :

Data products are increasingly central to how Alegeus delivers value - to employers making benefits decisions, to TPAs managing complex plan configurations, and to members accessing the right information at the right moment. At the same time, we are introducing AI capabilities that have to be held to a quality standard that is still being defined industry-wide. This role sits at the intersection of those two hard problems, with the product context to ask the right questions, the technical depth to engineer the right answers, and the team around you to make it count.

Role : Automation Architect

Industry Type : Software Product

Department : Engineering - Software & QA

Employment Type : Full Time, Permanent

Role Category : Software Development

Education :

UG : Any Graduate