HamburgerMenu
hirist

Data Engineer - Scala/GCP

Geektrust
7 - 12 Years
Bangalore

Posted on: 16/11/2026

Job Description

Mandatory Skill Set:

- Strong hands-on expertise in Apache Spark and Scala for building and optimizing large-scale data pipelines.

- Experience working with high-volume batch and streaming workloads across multiple data sources.

- Strong experience with Hive and distributed data processing.

- Hands-on experience with Kafka and event/streaming-based architectures.

- Strong programming experience in Scala and/or Python.

- Experience designing, developing and troubleshooting large-scale ETL/data pipelines.

- Strong understanding of data modelling, data quality, performance tuning and distributed systems.

- Experience working with at least one major cloud platform - GCP, AWS or Azure.

- Experience with BigQuery and GCP services is preferred.

- Experience with ETL workflows and orchestration tools such as Oozie or equivalent.

- Hands-on experience with Git, Jenkins and CI/CD practices, including automated build, testing and deployment pipelines.

- Ability to independently troubleshoot complex data, performance and production issues.

Preferred / Added Advantage:

- Strong Google Cloud Platform (GCP) experience.

- Exposure to BigQuery, Google Cloud Storage (GCS), Dataproc and Pub/Sub.

- Experience migrating on-premise Spark/Hive workloads to cloud platforms.

- Experience with large-scale data migration and reconciliation.

- Knowledge of data quality and observability frameworks.

- Experience working with multiple upstream and downstream integrations.

- Exposure to multiple cloud environments.

What would you do here:

- Design, develop and optimize large-scale data pipelines using Apache Spark and Scala.

- Work with high-volume batch and streaming workloads across multiple data sources.

- Develop and support data processing solutions using Hive, Kafka and GCP services.

- Troubleshoot complex data, performance and production issues with minimal supervision.

- Contribute to cloud migration and platform modernization initiatives.

- Build and maintain ETL workflows and data orchestration pipelines.

- Perform performance tuning and optimization of Spark jobs and large datasets.

- Develop reusable, maintainable and production-ready data solutions.

- Implement and maintain CI/CD pipelines and engineering automation.

- Collaborate with engineering, product, analytics and platform teams to deliver end-to-end solutions.

- Participate in code reviews, testing, production deployments and operational support.

- Quickly understand existing systems and take ownership of assigned deliverables.

info-icon

Did you find something suspicious?

Similar jobs that you might be interested in

Loading chat...