Job Description :

We have received an urgent requirement for Azure Databricks Data Engineers. We currently have approximately 12 open positions and require strong candidates with extensive experience in Azure Data Engineering, Databricks, Streaming, and Lakehouse architecture.

Requirement Snapshot :

Role : Azure Databricks Data Engineer

Location : Bangalore, Chennai, Pune, Jaipur, Gurugram, Hyderabad, Bhopal/ Abu Dhabi (Onsite)

Work mode : 5 days Office

Notice Period Immediate only

UAN no. is mandatory for all profiles

Experience : 8+ Years

Additional Benefits (Abu Dhabi) : Employee Visa, Medical Insurance, Annual Airfare & 15 Days Accommodation

About the Role :

This role is responsible for building reliable, scalable, and governed data pipelines that power analytics, operational reporting, and the Data Intelligence Layer on Databricks. The ideal candidate should have strong expertise in Azure Data Engineering, Databricks Lakehouse architecture, streaming data pipelines, and cloud-native data processing frameworks.

Key Responsibilities :

- Build batch pipelines using Delta Lake with optimized table design, partitioning, Z-ORDER, OPTIMIZE, and VACUUM strategies.

- Implement incremental ingestion using Databricks Auto Loader with schema evolution and checkpointing.

- Develop Structured Streaming pipelines with watermarking, state management, and restart safety.

- Implement Lakeflow pipelines for governed and declarative data processing.

- Design replayable and idempotent pipelines supporting safe backfills.

- Optimize Spark workloads using Adaptive Query Execution, shuffle tuning, skew mitigation, and join optimization.

- Build analytics-ready datasets consumed by Databricks SQL, dashboards, Genie, and downstream applications.

- Develop DBSQL models and semantic data layers aligned with Data Intelligence requirements.

- Package and deploy solutions using Databricks Repos and Asset Bundles.

- Support Synapse-to-Databricks migration initiatives and performance benchmarking.

Mandatory Technical Skills :

- Azure Databricks (Mandatory)

- Azure Data Factory (ADF)

- Unity Catalog

- Databricks Asset Bundles

- Medallion Architecture

- PySpark

- SQL & Databricks SQL (DBSQL)

- Azure Synapse Analytics

- Real-Time Streaming & Structured Streaming

- Lakeflow Pipelines

- Delta Lake

- Data Intelligence Layer Implementation

- CI/CD & Databricks Repos

- Performance Optimization & Query Tuning

Important Screening Note :

- Passport with minimum 1 year validity is mandatory.

- Candidates should be open for Abu Dhabi onsite opportunities if required.

- Strong hands-on experience in Azure Databricks is mandatory.

- Experience with Lakeflow, Streaming, DBSQL, Delta Lake, and Unity Catalog is required.

- Share only candidates with relevant Azure Data Engineering project experience.

- Kindly focus on quality profiles aligned to the mandatory skill stack.