Big Data Engineer - AWS Platform

Job Title : Big Data Engineer

Loc : Hyderabad (WFO)

Exp : 8+y Mandatory

Job Summary :

We are seeking a highly skilled Big Data Engineer to join a long-term, mission-critical project on a Contract-to-Hire basis.

The ideal candidate will have strong hands-on experience in building and maintaining large-scale data pipelines within modern big data ecosystems.

This role requires deep expertise in Spark, Scala, Python, and AWS, along with exposure to real-time streaming frameworks such as Kafka. The position is based in Hyderabad and requires full-time Work From Office.

Key Responsibilities :

- Design, develop, and maintain scalable and high-performance data pipelines in a big data environment

- Build and optimize batch and real-time data processing workflows using Apache Spark

- Develop data engineering solutions using Scala and Python

- Work extensively on AWS cloud services to support data ingestion, processing, and storage

- Implement and support streaming data pipelines using Kafka or similar frameworks

- Ensure data quality, reliability, and performance across pipelines

- Collaborate with data architects, analysts, and downstream consumers to understand data requirements

- Monitor and troubleshoot production data pipelines and resolve performance or data issues

- Follow best practices related to data security, governance, and compliance

- Participate in code reviews and contribute to continuous improvement initiatives

Required Skills & Qualifications :

- 8+ years of experience as a Big Data Engineer or in a similar data engineering role

- Strong hands-on experience with :

1. Apache Spark

2. Scala

3. Python

- Solid data engineering experience on the AWS Cloud Platform (e.g., S3, EMR, Glue, Redshift, Lambda)

- Good understanding of streaming frameworks, especially Apache Kafka

- Experience designing and maintaining large-scale data pipelines in big data environments

- Strong problem-solving and debugging skills

- Ability to work independently in a fast-paced, on-site environment

Nice to Have :

- Experience with other big data tools (Hive, HDFS, Airflow, etc.)

- Exposure to real-time analytics or event-driven architectures

- Prior experience in long-term Contract-to-Hire engagements

- Strong communication and stakeholder collaboration skills