Role Overview :

As a Senior GCP Data Engineer, you will be instrumental in designing, building, and optimizing advanced data pipelines and architectures on the Google Cloud Platform. This role involves working closely with data scientists, analysts, and business stakeholders to translate complex data requirements into scalable and efficient data solutions. You will play a critical part in shaping our data strategy, ensuring data quality, accessibility, and reliability, ultimately driving impactful business outcomes through robust data engineering practices. This position offers the flexibility of working from anywhere in India, collaborating with distributed teams and clients.

Key Responsibilities :

- Design and implement robust, scalable, and fault-tolerant data pipelines on Google Cloud Platform (GCP) using services like Dataflow, Dataproc, BigQuery, and Cloud Storage, to ingest, transform, and load large datasets for analytical and operational use cases.

- Develop and optimize data processing jobs using Python and PySpark, ensuring efficient execution and resource utilization for high-volume, real-time, and batch data processing requirements.

- Architect and manage data warehousing solutions within BigQuery, including schema design, partitioning, clustering, and performance tuning, to support complex analytical queries and reporting needs for business intelligence teams.

- Collaborate with data scientists and machine learning engineers to build and deploy data-driven applications and models, providing reliable data infrastructure and feature engineering support.

- Establish and enforce data governance, security, and compliance standards across all data assets on GCP, safeguarding sensitive information and ensuring regulatory adherence for our clients.

- Troubleshoot and resolve complex data-related issues, optimizing existing data pipelines and processes for improved performance, reliability, and cost-efficiency, to maintain uninterrupted data flow for critical business operations.

- Mentor junior engineers and contribute to best practices, code reviews, and architectural discussions, fostering a culture of technical excellence and continuous improvement within the data engineering team.

Required Skillset :

- Demonstrated expertise in designing, developing, and deploying large-scale data solutions on Google Cloud Platform (GCP), showcasing a deep understanding of its core data services.

- Advanced proficiency in Python for data manipulation, scripting, and API integration, coupled with extensive experience in building and optimizing PySpark applications for distributed data processing.

- Strong command over SQL for complex data querying, analysis, and database management, particularly within BigQuery and other relational/NoSQL databases.

- Proven ability to architect and implement data warehousing, data lakes, and ETL/ELT processes, ensuring data quality, integrity, and accessibility for diverse business needs.

- Experience with CI/CD pipelines, version control (Git), and automated testing frameworks to ensure robust and maintainable data engineering solutions.

- Exceptional problem-solving capabilities and a proactive approach to identifying and resolving technical challenges in complex data environments.

- Excellent communication and collaboration skills, with the ability to articulate technical concepts to both technical and non-technical stakeholders, fostering effective teamwork in a remote or hybrid setting.

- A Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field, providing a strong foundation in computational principles and data structures.