Location : Hyderabad, Telangana, India

Industry : IT Services & Consulting

Role Overview :

We are hiring a Senior Data Engineer to design, develop, and optimize scalable cloud data platforms and modern ETL/ELT pipelines using Microsoft Fabric, Databricks, or Snowflake. In this role, you will build and maintain data lakehouse and data warehouse solutions, develop high-performance data pipelines, enable analytics and reporting, and drive data engineering best practices.

Key Responsibilities :

1. Pipeline & Architecture Development :

- Design, build, and maintain scalable batch and real-time data pipelines, data models, and ETL/ELT processes using Delta Lake, Apache Spark, Microsoft Fabric, and Snowflake.

2. Data Processing & Optimization :

- Develop high-performance solutions using PySpark, Python, Scala, and SQL to handle large, complex datasets efficiently.

3. Lakehouse & Analytics Support :

- Build data lake, lakehouse, and data warehouse architectures, and support business analytics and self-service reporting through Power BI and DAX modeling.

4. Governance & Reliability :

- Implement data quality, governance, security, and monitoring practices across pipelines while troubleshooting and resolving performance and production issues.

5. DevOps & Collaboration :

- Collaborate with data scientists, analysts, and stakeholders while implementing CI/CD workflows, Git, code reviews, and reusable frameworks.

Required Qualifications & Skills :

1. Education & Experience :

- Bachelor's/Master's degree in Computer Science, IT, Engineering, or a related field with 6 - 10 years of core data engineering experience.

2. Primary Platforms (at least one) :

- Microsoft Fabric : Data Factory, Lakehouse, Data Warehouse

- Databricks : Spark, Delta Lake, PySpark, MLflow

- Snowflake : Data Warehousing, Snowpipe, Performance Optimization

- Power BI : Data Modeling, DAX, Report Development

3. Core Technical Expertise :

- Strong proficiency in SQL, Python or Scala, PySpark/Apache Spark, and cloud data platforms (Azure, AWS, or GCP).

4. Architecture & Delivery :

- Deep understanding of data modeling, data governance, security, and CI/CD/DevOps practices.

5. Key Skills :

- Data Engineering, PySpark, Python, SQL, Microsoft Fabric, Databricks, Snowflake, Power BI, Delta Lake, ETL/ELT, Cloud Data Platforms (AWS/Azure/GCP), CI/CD