Data Engineer - Google Cloud Platform

Job Description :

Key Responsibilities :

- Implement ETL (Extract, Transform, Load) processes and ensure data quality and consistency across all data sources.

- Work with GCP data services such as BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub, and Composer to manage and process large volumes of data.

- Develop and optimize SQL queries and data models to support business analytics and reporting needs.

- Integrate data from various sources, including internal databases, third-party data, and APIs.

- Collaborate with data scientists and business stakeholders to identify data needs and deliver actionable insights.

- Monitor and troubleshoot data pipeline performance, ensuring data is processed and delivered on time.

- Apply best practices for data security, including data encryption, access control, and audit logging in GCP.

- Continuously evaluate and improve data infrastructure, ensuring high availability, scalability, and reliability.

- Automate data operations and reporting using tools like Cloud Functions, Cloud Run, and others.

- Stay up to date with emerging GCP technologies, data engineering trends, and best practices.

Required Qualifications :

- Proven experience in data engineering or a similar role with expertise in Google Cloud Platform (GCP).

- Strong proficiency in SQL and experience with relational databases (BigQuery, Cloud SQL, etc.).

- Experience working with GCP data services : BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, and Cloud Composer.

- Knowledge of data modeling, data warehousing concepts, and ETL processes.

- Familiarity with programming languages like Python, Java, or Scala.

- Experience with version control systems (e.g., Git) and agile development methodologies.

- Understanding of data security best practices in cloud environments.

- Strong problem-solving and analytical skills.

- Excellent communication and collaboration skills.