Job Title: Databricks Engineer

Noon Shift: 2 PM - 11 PM

Location: Pune

Expected Experience: 7+ years

Must Have Skills:

- Databricks (minimum 2-3 years hands-on)

- Apache Spark / PySpark

- Delta Lake

- Unity Catalog

- Python

- SQL

- Cloud (Azure preferred, AWS/GCP also acceptable)

- ETL/ELT Pipeline Development

- Data Warehousing & Data Modeling

- Airflow / ADF

- Git & CI/CD

- 7+ years overall experience

Good to Have:

- Scala

- Streaming (Structured Streaming, Kafka)

- Medallion Architecture (Bronze-Silver-Gold)

- Data Governance & Data Quality

- Migration Projects to Databricks

- Performance Tuning & Optimization

Key Responsibilities:

- Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.

- Build batch and streaming data processing solutions for large-scale datasets.

- Optimize ETL/ELT workflows for performance, reliability, and cost efficiency.

- Work with data from multiple sources and integrate it into curated data models.

- Implement data quality checks, monitoring, and troubleshooting for production pipelines.

- Collaborate with stakeholders to understand data requirements and translate them into technical solutions.

- Develop reusable frameworks, notebooks, and libraries for data engineering standards.

- Ensure adherence to data governance, security, and compliance best practices.

- Support migration of legacy data workloads to Databricks/cloud platforms.

- Mentor junior engineers and contribute to engineering best practices.

- Apply best practices in schema management, data observability, data governance, and performance optimization.

Required Qualifications:

- 7+ years of experience in data engineering or related roles.

- Strong hands-on experience with Databricks, Apache Spark, Delta Lake, and Unity Catalog.

- Proficiency in Python, Scala, and SQL.

- Experience with cloud platforms such as Azure, AWS, or GCP.

- Strong understanding of ETL/ELT, data modeling, and data warehousing concepts.

- Experience with orchestration tools such as Airflow, ADF, or similar.

- Familiarity with CI/CD, Git, and deployment automation for data solutions.

- Excellent problem-solving, communication, and collaboration skills.

- Databricks Platform Expertise: Strong proficiency in using the Databricks Lakehouse Platform for data engineering tasks.

Application Details:

- Share your resume at mansi.gupta@appzime.in OR 8810359325

- Subject: Databricks Engineer | Location | Pune