Job Title : Senior Data Engineer - Microsoft Fabric & Azure/AWS

Location : Rajajinagar, Bengaluru - 4 days Onsite

Experience Level : 5+ Years

Job Overview :

We are seeking a highly skilled Senior Data Engineer with strong expertise in Azure Data Services, Microsoft Fabric, OneLake, PySpark (Notebooks), and Databricks to join our growing data engineering team. The ideal candidate will be responsible for building scalable data pipelines, designing robust data lake/warehouse solutions, and driving high-impact data integrations and analytics across the enterprise data ecosystem.

Key Responsibilities :

- Design, build, and maintain data warehouses, data lakes, and lakehouses with a focus on scalability, reliability, and performance.

- Develop and manage end-to-end ETL/ELT workflows to seamlessly ingest, transform, and load

structured and unstructured data into the data platform.

- Build and optimize pipelines using Azure Synapse, Microsoft Fabric, OneLake, and Databricks.

- Create and maintain PySpark Notebooks for advanced transformation and automation use

cases.

- Translate business and data requirements into technical designs, data models, and ETL

strategies on cloud-native data platforms.

- Ingest data from diverse source systems, ensuring completeness, accuracy, and performance

through effective pipeline design.

- Maintain and enhance complex data environments by integrating multiple data sources and

managing real-time and batch data processing.

- Implement robust backup, recovery, and archiving strategies for data platforms and databases.

- Collaborate with cross-functional teams to support BI tools (e.g., Power BI, Tableau), and downstream applications through clean, enriched data.

- Drive improvements in DevOps practices within the data platform team to ensure agility and continuous integration/deployment.

- Monitor data quality and operational health of all pipelines; troubleshoot and resolve integration failures or data inconsistencies.

Required Skills & Experience :

- Proven experience in data engineering with a focus on cloud-native platforms.

- Expertise in Azure Data Services, Microsoft Fabric, OneLake, Databricks, and PySpark (Notebooks).

- Solid understanding of data modeling, database design, data lake architecture, and ETL frameworks.

- Strong SQL skills and working knowledge of both relational and NoSQL databases.

- Experience with building enterprise-grade, distributed data systems with high availability and fault tolerance.

- Hands-on experience in building data pipelines using a combination of batch and streaming technologies.

- Strong understanding of data validation, quality assurance, and data governance practices.

- Working knowledge of data visualization tools like Power BI and Tableau.

- Experience integrating with CI/CD pipelines and monitoring tools in a DevOps environment.

- Excellent communication skills to articulate complex data engineering concepts to both technical and non-technical stakeholders.

Preferred Qualifications :

- Azure Data Engineer Certification (DP-203) or equivalent.

- Prior experience in implementing solutions using Microsoft Fabric and OneLake in production environments.

- Familiarity with data cataloging, metadata management, and data lineage tracking tools.