Key Responsibilities :

- Design and develop batch and real-time data pipelines on GCP.

- Build scalable ETL/ELT pipelines using Dataflow, BigQuery, and Cloud Composer.

- Ingest data from databases, APIs, files, SaaS applications, and streaming sources.

- Design and optimize BigQuery datasets, tables, partitioning, and clustering.

- Develop data transformations using SQL, Python, Apache Beam, Spark, dbt, or Dataform.

- Implement workflow orchestration and dependency management using Apache Airflow / Cloud Composer.

- Implement data quality checks, reconciliation, lineage, monitoring, and validation processes.

- Configure and manage GCP IAM, service accounts, and least-privilege access.

- Optimize BigQuery and Dataflow workloads for performance, scalability, and cloud cost efficiency.

- Build and manage infrastructure using Terraform.

- Participate in CI/CD implementation and deployment processes.

- Troubleshoot and resolve production data-pipeline failures and performance issues.

- Collaborate with Data Architects, Data Analysts, Data Scientists, and application engineering teams.

- Ensure data pipelines are reliable, secure, scalable, and maintainable.

Required Skills:

- 6+ years of experience in Data Engineering.

- Strong hands-on experience with Google Cloud Platform (GCP).

- Expertise in BigQuery, Dataflow, and Cloud Composer/Airflow.

- Strong SQL and Python programming skills.

- Experience with Apache Beam / Spark.

- Experience with ETL/ELT pipeline development and data integration.

- Knowledge of dbt / Dataform.

- Strong understanding of data warehousing, data modelling, partitioning, and clustering.

- Experience with data quality, reconciliation, lineage, monitoring, and troubleshooting.

- Hands-on experience with GCP IAM and service accounts.

- Experience with Terraform and Infrastructure as Code (IaC).

- Understanding of CI/CD and DevOps practices.

Preferred Candidate Profile:

- Strong experience designing production-grade, scalable data platforms on GCP.

- Experience handling both batch and streaming data workloads.

- Strong understanding of cloud cost optimization and performance tuning.

- Excellent analytical and problem-solving skills.

- Ability to work effectively with technical and business stakeholders.

- Strong communication and collaboration skills.