EverestDX - Data Engineer - ETL/Spark

Skills and Development :

- Build Azure data factory pipeline, Azure Data Lake, Datawarehouse

- Design, develop, and maintain data pipelines and ETL processes to extract, transform, and load data from various sources into data warehouse

- Optimize and fine-tune data processes to ensure high performance and reliability

- Implement data validation, testing, and quality assurance processes to maintain data accuracy and consistency

- Proficient in Azure Data Factory, Azure Data Catalog, Data Architecture, Apache Spark, ETL

tools and techniques

- Develop and maintain documentation for data pipelines, processes, and data dictionaries

- Monitor and troubleshoot data pipeline issues, resolving them

- Collaborate with multiple teams and ensure data integrity, accuracy, and availability

Qualification :

- Azure Data Factory, Azure Data Lake, Azure Devops , Azure Data warehouse

- Experience with Repository and version control, GitHub