They are both Data Engineers and quality assurance engineers. They write code to test software, focusing on creating automated tests, frameworks, and tools that help ensure the quality of data pipelines and data itself.

The SDET works closely with Scrum teams and stakeholders to ensure the quality and reliability of complex data pipelines, ETL processes, and large-scale data systems.

The role requires automating testing at various stages of the development lifecycle, from unit testing frameworks to system, integration, UAT and monitoring production data.

The core responsibilities for the job include the following :

Collaborating with Scrum Teams :

- Actively participate in daily Scrum meetings, planning, and refinement sessions to develop an overall test strategy for the Epic/Project.

- Work closely with Product Owners, Business Analysts and Data Engineers to ensure that quality is baked into the design and development of data pipelines by identifying functional and non-functional test cases.

- Engage with cross-functional teams like release/deployment, Infrastructure to ensure testing is integrated into the entire data engineering delivery process.

Test Strategy and Framework Development :

- Design, develop, and implement unit testing frameworks for data pipelines, ETL processes, Data Visualization dashboards and distributed data systems.

- Develop automated testing frameworks for system, integration, and UAT.

- Define test strategies and best practices for ensuring accuracy, reliability, scalability, and performance of the data engineering solutions.

Automation of Testing Processes:

- Automate tests for data validation, schema verification, data transformation, and data quality checks across different stages of the pipeline.

- Write scripts and programs in languages such as Python, Java, or Scala to automate test execution.

Requirements :

- Relevant Experience - 7+ years of experience in the fields of QA Testing, Warehousing, Data Engineering, Big Data, Cloud and Analytics, and Data Visualization.