Databricks Developer - ETL/ELT Pipelines

Description :

We are seeking a talented Databricks Developer to join our team and help build scalable, high-performance data pipelines and analytics solutions on the Databricks Lakehouse platform. If you have a passion for turning complex data into actionable insights and enjoy working in a collaborative, fast-paced environment, we want to hear from you.

Roles & Responsibilities :

- Design, develop, and maintain ETL/ELT pipelines using PySpark, SQL, and Databricks workflows for structured and semi-structured data.

- Tune Spark jobs, manage partitioning, caching, and cluster configurations to reduce cost and runtime.

- Implement Delta Lake tables, manage schema evolution, and ensure data quality leveraging Unity Catalog.

- Partner with data analysts, data scientists, and engineering teams to translate business requirements into production-ready solutions.

- Monitor data pipelines, troubleshoot issues, and continuously improve existing workflows.

Required Skills & Experience :

- Strong proficiency in PySpark, SQL, and Python.

- Experience with Delta Lake and Unity Catalog is a plus.

- Familiarity with cloud platforms (AWS, Azure, or GCP) and CI/CD for data workflows.

- Solid understanding of data warehousing concepts, data modeling, and performance tuning.

- Experience with orchestration tools such as Airflow, ADF, or Databricks Workflows.

Nice to Have :

- Databricks Certified Data Engineer (Associate or Professional).

- Experience with MLflow for model deployment.