Job Description :

We are looking for a data engineer with 8+ years of experience responsible for designing and building reliable data pipelines and transforming raw data into clean, consumable datasets. The role focuses on developing and maintaining ETL and ELT workflows, data modeling, and ensuring data quality and lineage for downstream users. You will collaborate with product and analytics teams to translate business requirements into scalable data products. This position is hands-on and involves coding, optimizing queries, troubleshooting pipelines, and implementing best practices for data governance and reliability.

Role & Responsibilities :

- Design, develop, and maintain scalable and reliable data pipelines and platforms on Google Cloud Platform to support enterprise analytics and reporting.

- Collaborate with business and technical teams to understand data requirements and translate them into efficient data engineering solutions.

- Build data ingestion frameworks using Kafka Confluent and integrate data from multiple cloud and on-premises sources.

- Implement ELT and ETL pipelines using PySpark, SQL, Dataform, DBT, and Dataproc for efficient data transformation and processing.

- Design and manage data models, schemas, and data warehousing structures optimized for BigQuery performance and scalability.

- Ensure data quality, consistency, and integrity through effective data validation and monitoring mechanisms.

- Implement data governance practices including data lineage, metadata management, privacy, and compliance.

- Develop automated monitoring and alerting mechanisms to ensure reliability, availability, and performance of data pipelines.

- Collaborate with cross-functional teams to enable downstream reporting, analytics, and AI/ML model consumption.

- Follow best practices for version control, documentation, and CI/CD workflows using Git and infrastructure-as-code tools.

- Provide operational support for data pipelines and proactively troubleshoot performance issues.

- Continuously optimize data processing, query execution, and resource utilization in GCP environments.

Required Skills :

- Strong programming skills in Python and PySpark for large-scale data processing.

- Proficiency in SQL for data manipulation, analysis, and performance tuning.

- Experience with Dataform, Dataproc, and BigQuery for data pipeline development and orchestration.

- Hands-on experience with Kafka and Confluent for real-time data streaming.

- Knowledge of Cloud Scheduler and Dataflow for automation and workflow management.

- Familiarity with DBT, Machine Learning, and AI concepts is a good advantage.

- Understanding of Data Governance principles and implementation practices.

- Experience using Git for version control and CI/CD pipelines for automated deployments.

- Working knowledge of Infrastructure as Code (IaC) for cloud resource management and automation.