Role : Senior Data Engineer - L3 Production Support

Experience : 5+ years of relevant experience

Location : Pan India

Role Summary :

We are looking for an experienced Senior Data Engineer (L3 - Production Support) with strong expertise in Big Data technologies and Spark-based data processing. The role will focus on building, supporting, and optimising scalable data pipelines, handling complex production issues, and ensuring high availability and performance of data platforms.

Technical Skills - Primary (Must Have) :

- 5+ years of experience as a Data Engineer working on Big Data technologies

- Strong expertise in building scalable data pipelines using Spark and PySpark

- Good hands-on experience with Python

- Hands-on experience with :

1. Databricks

2. MongoDB

3. Docker

4. Kubernetes

- Good understanding of DevOps technologies and practices

Key Responsibilities :

- Develop, maintain, and provide production support for scalable data pipelines

- Analyse business requirements and data models to design and develop ETL solutions

- Work independently on Spark-based data pipelines and resolve L3 production issues

- Write and optimise complex business logic using PySpark and Spark SQL

- Demonstrate strong understanding of Spark clusters, parallel processing, and performance tuning

- Develop unit test cases to validate ETL code and data accuracy

- Create reusable functions and follow a modular, clean coding approach

- Work extensively with Athena :

1. Create and manage tables and indexes

2. Write complex SQL queries

- Fetch and load data from :

1. Relational databases (Oracle, SQL Server, etc.)

2. NoSQL databases (MongoDB)

- Ensure data reliability, stability, and performance in production environments

Technical Skills - Secondary (Nice to Have) :

- Experience with AWS IAM :

1. IAM role creation

2. Policy creation and management

- Experience in Docker image creation and modifications

- Exposure to Camunda and workflow orchestration using Camunda

Other Expectations / Soft Skills :

- Strong verbal and written communication skills

- Excellent troubleshooting and problem solving abilities

- Continuous learning mindset and willingness to upskill

- Proactive approach towards automation, reducing manual efforts through code and scripts

- Ability to work effectively in high pressure production support environments