Big Data Engineer

Big Data Engineer

Key Responsibilities :

- Build and manage highly scalable Big Data pipelines for large structured datasets.

- Collaborate with Product Management & Engineering leadership to design and implement optimal solutions.

- Participate in design discussions and contribute to selecting, integrating, and maintaining Big Data frameworks.

- Develop distributed data processing systems using Spark, Akka, or similar frameworks.

- Review, optimize, and enhance existing data pipelines to resolve bottlenecks.

- Act as a Senior Individual Contributor by taking ownership of features/modules.

- Troubleshoot production issues and ensure seamless data flow.

- Work with US & India engineering teams to build and scale Java/Scala-based data pipelines.

- Lead technical excellence in the India engineering team and ensure delivery quality.

- Follow Agile practices and use tools such as JIRA, Git, Gradle/Maven/SBT for development & issue tracking.

Must-Have Skills :

- Coding expertise Strong programming & database skills.

- SQL Hands-on experience.

- Scala, PySpark, Cloud (AWS/Azure).

- Data Lake, Snowflake.

- Proven experience in building products from scratch.

- Strong background in Java or Scala, OOP, data structures, algorithms, profiling, and optimization.

- Experience with ETL/Data pipeline tools (Apache NiFi, Airflow, etc.).

- Hands-on with version control (Git) and build tools (Gradle, Maven, SBT).