Description :

- Experience Level.

- 10+ years of experience in data engineering, with at least 35 years providing architectural guidance, leading teams, and standardizing enterprise data solutions.

- Must have deep expertise in Databricks, GCP, and modern data architecture patterns.

Key Responsibilities :

- Provide architectural guidance and define standards for data engineering implementations.

- Lead and mentor a team of data engineers, fostering best practices in design, development, and operations.

- Own and drive improvements in performance, scalability, and reliability of data pipelines and platforms.

- Standardize data architecture patterns and reusable frameworks across multiple projects.

- Collaborate with cross-functional stakeholders (Product, Analytics, Business) to align data solutions with organizational goals.

- Design data models, schemas, and dataflows for efficient storage, querying, and analytics.

- Establish and enforce strong data governance practices, ensuring security, compliance, and data quality.

- Work closely with governance teams to implement lineage, cataloging, and access control in compliance with standards.

- Design and optimize ETL pipelines using Databricks, PySpark, and SQL.

- Ensure robust CI/CD practices are implemented for data workflows, leveraging Terraform and modern DevOps practices.

- Leverage GCP services such as Cloud Functions, Cloud Run, BigQuery, Pub/Sub, and Dataflow for building scalable solutions.

- Evaluate and adopt emerging technologies, with exposure to Gen AI and advanced analytics capabilities.

Qualifications & Skills :

- Bachelors or Masters degree in Computer Science, Data Engineering, or related field.

- Extensive hands-on experience with Databricks (Autoloader, DLT, Delta Lake, CDF) and PySpark.

- Expertise in SQL and advanced query optimization.

- Proficiency in Python for data engineering and automation tasks.

- Strong expertise with GCP services : Cloud Functions, Cloud Run, BigQuery, Pub/Sub, Dataflow, GCS.

- Deep understanding of CI/CD pipelines, infrastructure-as-code (Terraform), and DevOps practices.

- Proven ability to provide architectural guidance and lead technical teams.

- Experience designing data models, schemas, and governance frameworks.

- Knowledge of Gen AI concepts and ability to evaluate practical applications.

- Excellent communication, leadership, and stakeholder management skills.

Skills :

- Google Cloud Platform (GCP), databricks, Architecture, bigquery, Google Cloud Storage, Generative AI and Dataflow architecture.