- Lead the design, development, and delivery of enterprise-scale data solutions on the Databricks Lakehouse Platform.

- Provide technical leadership while remaining hands-on with development, troubleshooting, and performance optimization.

- Architect and implement ETL/ELT pipelines using PySpark, Spark SQL, and Databricks.

- Design and build batch and real-time/streaming data pipelines using Structured Streaming, Kafka, or Event Hubs.

- Implement Delta Lake and Medallion Architecture (Bronze, Silver, Gold) data platforms.

- Drive Spark performance tuning, query optimization, cluster sizing, and cost optimization initiatives.

- Lead code reviews, establish development standards, and mentor Data Engineers.

- Integrate data from APIs, cloud storage, databases, and enterprise applications.

- Implement CI/CD, DevOps, monitoring, and data governance best practices.

- Collaborate with Architects, Product Owners, and Business stakeholders to define scalable data solutions

Required Skillset :

- Demonstrated expertise in building scalable data platforms using Azure Databricks, PySpark, and Delta Lake, with a deep understanding of distributed computing principles.

- Proven ability to design and manage real-time streaming architectures using Kafka and Azure-native services to solve complex data integration challenges.

- Strong proficiency in performance tuning and troubleshooting large-scale ETL processes to ensure high availability and data integrity.

- Exceptional communication skills with the ability to translate technical complexities into actionable insights for non-technical stakeholders and leadership.

- A collaborative mindset with a track record of mentoring team members and fostering a high-performance culture in a hybrid work environment.