Description :

Key Responsibilities :

Data Source Identification & Quality Assessment:

- Identify, analyze, and document available data sources, formats, and access methods.

- Assess data quality across completeness, accuracy, consistency, and reliability.

Data Mapping & Integration:

- Define and maintain comprehensive data mapping between source systems and Databricks tables.

- Design and implement scalable ETL/ELT pipelines using Databricks and Apache Spark.

Databricks & Data Modeling:

- Develop and optimize Databricks workloads using Spark and Delta Lake.

- Design efficient data models optimized for performance, analytics, and API consumption.

Required Skills & Qualifications :

Technical Skills :

- Strong hands-on experience with Databricks and Apache Spark.

- Proficiency in Python and SQL.

- Proven experience in data mapping, transformation, and data modeling.

- Experience integrating data from APIs, databases, and cloud storage.

- Solid understanding of ETL/ELT concepts and data warehousing principles.