Position : Azure Data Engineer

Experience : 3-5 Years

Job Type : Full-time

Job Summary :

We are seeking a skilled Azure Data Engineer with 3-5 years of experience to design, develop, and maintain scalable data pipelines within the Microsoft Azure ecosystem.

The ideal candidate will have strong expertise in Azure Data Factory, Azure Databricks, PySpark/Scala Spark, and Python.

This role is critical for building efficient ETL processes, optimizing data workflows, and ensuring high-quality, reusable code to support business analytics and reporting needs.

Key Responsibilities :

Data Pipeline Development :

- Design, develop, and maintain robust and scalable data pipelines, primarily utilizing PySpark / Scala Spark, Python, Azure Databricks, and Azure Data Factory.

Code Quality & Optimization :

- Demonstrate strong programming skills by writing optimized, reusable, and high-quality code.

- Optimize SQL queries for efficient data extraction and manipulation across various data sources.

Azure Platform Expertise :

- Apply expertise in Azure Databricks, including effective workflow management, collaborative data exploration, and leveraging the Delta Lakehouse architecture.

ETL/ELT Best Practices :

- Implement and enforce best practices for data pipeline optimization, ETL (Extract, Transform, Load) processes, and query performance tuning.

Data Storage & Services :

- Utilize and manage core Azure data services, including Azure SQL Database, Azure Blob Storage, Azure Synapse Analytics, and Azure Data Lake Storage.

Collaboration & Documentation :

- Collaborate effectively with cross-functional teams to accurately understand and translate business requirements into technical solutions.

- Create and maintain comprehensive documentation for all data pipelines and workflows.

Maintenance & Troubleshooting :

- Proactively troubleshoot and resolve critical issues in data pipelines, ensuring minimal downtime and high data reliability.

Qualifications :

Experience : 3-5 years of professional experience as a Data Engineer, with a focus on Azure Cloud.

Mandatory Technical Stack : Strong proficiency in PySpark / Scala Spark, Python, Azure Databricks, and Azure Data Factory.

Data Management : Experience with Azure data storage solutions, specifically Azure Blob Storage and Data Lake Storage.

Query Optimization : Expertise in writing and optimizing complex SQL queries for performance tuning.

Process : Solid understanding of ETL/ELT methodologies and best practices for building production-grade data solutions.

Preferred Skills :

- Experience with Azure Synapse Analytics and relevant Azure Data Engineer certifications