Sunware Technologies - Data Engineer

Job Title : GCP Data Engineer

Relevant Experience : 5+ Years

Work Location : Pune, Hyderabad, Chennai, Bangalore, Delhi

Role Overview :

We are looking for a hands-on GCP Data Engineer to work on implementing and managing data ingestion using our in-house Origin Data Product (ODP) framework. The role involves pipeline configuration, data loading, debugging issues, and ensuring smooth ingestion operations. Candidates who understand the framework well are encouraged to suggest improvements and contribute to enhancements in collaboration with the core development team.

Key Responsibilities :

- Configure and execute data ingestion pipelines using a reusable GCP-based framework

- Work with GCS, BigQuery, Composer, Data Fusion, Dataproc for ETL/ELT operations

- Manage parameters, job configurations, and ingestion metadata

- Debug and resolve data, parameter, and job execution issues

- Monitor daily job runs and ensure SLAs are met

- Escalate framework-related defects to the core development team when required

- Collaborate with cross-functional teams for delivery

- Use Git for version control

- Maintain deployment and infrastructure configurations using Terraform

(Nice-to-have) :

- Work with TWSd or similar scheduling/monitoring tools

- Suggest process and framework improvements

- Contribute to minor framework enhancements where applicable

Required Skills & Experience :

- 5+ years of experience as a Data Engineer or similar role

- Strong hands-on experience with GCS, BigQuery, Composer, Data Fusion, Dataproc

- Proficiency in Python for scripting, debugging, and automation

- Experience with Terraform (Infrastructure as Code)

- Working knowledge of Git

- Strong understanding of data ingestion and ETL workflows

- Ability to independently debug runtime data issues

- Strong analytical and problem-solving skills

Good to Have :

- Exposure to TWSd or other enterprise schedulers

- Basic understanding of BigQuery SQL optimization