GCP Data Engineer

Role Overview :

We are seeking a highly skilled and motivated GCP Data Engineer to join our dynamic team. In this role, you will be responsible for designing, building, and maintaining robust and scalable data pipelines on Google Cloud Platform (GCP).

You will collaborate closely with data scientists, analysts, and other engineers to deliver high-quality data solutions that drive business insights and improve decision-making. Your work will directly impact our ability to leverage data effectively, enabling us to better serve our customers and achieve our business goals.

Key Responsibilities :

- Design and implement scalable and reliable data pipelines using GCP services like Dataflow, Dataproc, and BigQuery to ingest, process, and store large datasets.

- Develop and maintain data warehousing solutions using BigQuery, ensuring data quality, consistency, and performance for analytical workloads.

- Automate data workflows and orchestration using Airflow or Cloud Composer to streamline data processing and ensure timely data availability.

- Write and optimize SQL queries for data extraction, transformation, and loading (ETL) processes to support data analysis and reporting needs.

- Collaborate with data scientists and analysts to understand their data requirements and provide them with the necessary data infrastructure and tools to perform their analyses.

- Monitor and troubleshoot data pipelines, identifying and resolving issues to ensure data integrity and availability.

- Implement data governance and security best practices to protect sensitive data and comply with regulatory requirements.

Required Skillset :

- Demonstrated ability to design, develop, and deploy data pipelines on Google Cloud Platform (GCP) using services such as Dataflow, Dataproc, and BigQuery.

- Proven expertise in data warehousing concepts and experience building and maintaining data warehouses using BigQuery.

- Strong proficiency in SQL and experience with data modeling, schema design, and query optimization.

- Hands-on experience with Python and PySpark for data processing and analysis.

- Experience with workflow orchestration tools such as Airflow or Cloud Composer.

- Excellent problem-solving and analytical skills, with the ability to identify and resolve data-related issues.

- Strong communication and collaboration skills, with the ability to work effectively in a team environment.

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

- Ability to work effectively in a hybrid work environment, collaborating with team members both in-person and remotely.