Description :

Key Skills & Experience :

- 2+ years of hands-on experience with Databricks, including job optimization and performance tuning.

- 4+ years in Data Engineering, with a solid foundation in data pipeline design, development, and implementation.

- Expertise in PySpark for large-scale data processing and transformations.

- Proven experience in ETL design for scalable and efficient data workflows.

- Strong understanding of Data Quality, Data Governance, and Data Control principles to ensure data accuracy, consistency, and reliability.

- Proficiency with Delta Tables for data storage, versioning, and ACID transactions in Databricks.

Responsibilities :

- Design and implement efficient ETL pipelines to process large datasets.

- Optimize Spark jobs in Databricks for better performance and cost efficiency.

- Implement best practices for data quality and data governance to ensure reliable and accurate datasets.

- Work with Delta Tables for transactional integrity and managing data lakes.

- Collaborate with cross-functional teams to ensure alignment on data management and reporting requirements