Key Responsibilities :

- ETL Testing & Validation : Lead the quality assurance process for our ETL (Extract, Transform, Load) pipelines.
You will be responsible for validating data accuracy and integrity between various sources and targets.

- SQL & Data Warehouse Testing : Use your expertise in SQL Server Management Studio (SSMS) to write complex queries and validate data in relational databases and data warehouses.

- Azure Data Stack Testing : Actively test and validate data processes built on the Azure Data Factory, Azure Databricks, and Azure Synapse platforms.

- Pyspark Scripting & Review : Use Pyspark notebooks and scripts to analyze and validate data within Databricks, ensuring data transformations are working as expected.

- Collaboration & Agile Methodology : Work closely with data engineers and business stakeholders in an Agile environment.
Use Jira for managing test cases, tracking defects, and reporting on progress.

- Process Understanding : Leverage a strong understanding of ETL processes and transformations to design comprehensive test cases that cover all data flow scenarios.

- Source & Target Expertise : Demonstrate experience working with a variety of data sources and targets, ensuring seamless and accurate data movement.

Required Skills & Experience :

- Strong knowledge of and experience with SSIS (SQL Server Integration Services).

- Proficiency in SQL and hands-on experience with SQL Server Management Studio (SSMS).

- Hands-on testing experience with the Azure Data Factory and Azure Synapse.

- Experience with Azure Databricks and working with Pyspark notebooks.

- Strong understanding of Pyspark for data validation.

- Practical experience working in an Agile environment.

- Proficiency with Jira or similar project management tools.

- A deep understanding of ETL processes and transformations.

- Proven experience in validating data between various sources and targets.

- Experience working with multiple data sources and targets