Role Overview :

As an Azure Data Engineer, you will be at the forefront of building and maintaining robust data pipelines and infrastructure within the Azure cloud environment. Your daily tasks will involve designing, developing, and deploying ETL processes, ensuring data quality, and optimizing data storage and retrieval.

You will collaborate closely with data scientists, business analysts, and other engineering teams to understand their data needs and deliver solutions that drive business insights. Your work will directly impact the organization's ability to make data-driven decisions, improve operational efficiency, and enhance customer experiences.

Key Responsibilities :

- Design and implement scalable and reliable ETL pipelines using Azure Data Factory to ingest, transform, and load data from various sources into Azure data storage solutions.

- Develop and maintain data models and schemas in Azure Synapse Analytics or other Azure data warehouses to support analytical and reporting requirements.

- Write and optimize Python scripts for data processing, automation, and data quality checks to ensure data accuracy and consistency.

- Monitor and troubleshoot data pipelines and infrastructure to identify and resolve performance bottlenecks and data quality issues.

- Collaborate with data scientists and business analysts to understand their data requirements and provide them with the necessary data infrastructure and tools.

- Implement data governance and security policies to ensure data privacy and compliance with regulatory requirements.

- Automate data engineering tasks and processes using scripting and orchestration tools to improve efficiency and reduce manual effort.

- Contribute to the development of data engineering best practices and standards to ensure consistency and quality across all data projects.

Required Skillset :

- Demonstrated expertise in designing, developing, and deploying ETL solutions using Azure Data Factory.

- Proven ability to write efficient and maintainable Python code for data processing and automation.

- Strong understanding of data warehousing concepts, data modeling techniques, and database design principles.

- Experience with Azure data storage solutions, such as Azure Synapse Analytics, Azure Data Lake Storage, and Azure SQL Database.

- Excellent problem-solving and analytical skills with the ability to troubleshoot complex data issues.

- Strong communication and collaboration skills with the ability to work effectively in a team environment.

- Ability to adapt to a fast-paced and dynamic environment and learn new technologies quickly.

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.