Databricks Engineer - Python/PySpark

Description :

We are hiring an experienced Databricks Engineer for a leading opportunity based in Bengaluru (Hybrid). The ideal candidate should have strong expertise in building scalable data solutions using Databricks and hands-on experience in the Life Science or Pharma domain.

Role Details :

- Position : Databricks Engineer

Location : Bengaluru (Hybrid)

Experience : 57 Years

Notice Period : Immediate Joiners/Early Joiners Preferred

Required Skills :

- Strong hands-on experience in Python/PySpark and SQL

Expertise in Databricks including :

- Workspace Management

- Notebooks

- Clusters & Autoscaling

- DBFS

- Delta Lake

- Workflows

- Delta Live Tables (DLT)

- Jobs Scheduling

- Unity Catalog

- Experience in designing and optimizing ETL/ELT pipelines

- Good understanding of data warehousing concepts and performance tuning

- Exposure to cloud platforms such as AWS / Azure / GCP is preferred

Domain Requirement (Mandatory) :

- Prior experience in Life Science or Pharma domain is mandatory

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines on Databricks

- Build robust ETL workflows using PySpark and SQL

- Optimize Databricks jobs, clusters, and performance for large datasets

- Implement data governance and security using Unity Catalog

- Work closely with business and analytics teams to deliver data solutions

- Ensure data quality, monitoring, and troubleshooting of pipelines

Preferred Candidate Profile :

- Strong problem-solving and analytical skills

- Excellent communication and collaboration abilities

- Ability to work in a hybrid environment and deliver independently