Job Description

Data Engineer - Microsoft Fabric, Databricks & Azure

Overview :

We are looking for a highly skilled Data Engineer with expertise in Microsoft Fabric, Databricks, Azure Data Factory, SQL, and PySpark to join our data engineering team. This role involves designing and building scalable data pipelines, optimizing data workflows, and enabling advanced analytics across cloud platforms, with a strong focus on Microsoft technologies.

Responsibilities :

- Design, develop, and maintain data pipelines using Azure Data Factory, Microsoft Fabric, and Databricks.

- Implement data transformation logic using PySpark and SQL for both batch and streaming workloads.

- Collaborate with data architects, analysts, and business stakeholders to understand data requirements and deliver solutions.

- Manage structured and unstructured data across various sources and formats.

- Optimize performance of Spark workloads and troubleshoot data pipeline issues.

- Configure and manage environments, clusters, and user access in Databricks and Microsoft Fabric.

- Ensure data governance, quality, and security across all data platforms.

Must Have :

- Hands-on experience with Microsoft Fabric (mandatory).

- Strong proficiency in SQL and PySpark.

- Experience with Azure Data Factory for orchestrating data workflows.

- Solid understanding of Databricks including Spark Core, Spark SQL, and Spark Streaming.

- Experience with real-time and batch data processing.

- Familiarity with data modelling and ETL methodologies.

Good to Have :

- Experience with Azure Synapse Analytics, Azure Data Lake Storage, and Logic Apps.

- Exposure to Delta Lake, Databricks ML, and Databricks migration projects.

- Certifications in Microsoft Fabric, Azure Data Engineering, or Databricks.

- Experience with cloud platforms like AWS or GCP.

Key Skills :

- SQL, Python, PySpark

- Microsoft Fabric, Databricks, Azure Data Factory

- Azure Synapse, Azure Data Lake, Logic Apps

- Data Modelling, ETL Design, Big Data Technologies