Senior Data Engineer - ETL/Apache Spark

Role : Lead AWS Data Engineer

Experience Level : 8 to 12 Years

Notice Period : Immediate to 15 Days

Location : [Specify if it's Remote/WFO/Hybrid can be added if applicable]

Job Description :

We are seeking a highly experienced Lead AWS Data Engineer to architect and manage scalable data solutions. In this role, you will lead a team of data engineers, collaborate with cross-functional teams, and establish best practices in data engineering to enable data-driven decision-making. A strong background in SQL, cloud platforms (AWS), and the insurance domain (P&C) is essential.

Key Responsibilities:

- Design, build, and manage scalable data pipelines and ETL/ELT processes.

- Architect secure, scalable, and high-performance data platforms across cloud (AWS, GCP, Azure) and on-prem environments.

- Provide technical leadership, mentoring, and guidance to the data engineering team.

- Collaborate with data scientists, analysts, and business stakeholders to gather requirements and deliver robust solutions.

- Optimize data workflows for performance, cost-efficiency, and reliability.

- Implement and enforce data governance, quality controls, and security standards.

- Monitor, troubleshoot, and maintain data infrastructure for high availability and reliability.

- Stay current with emerging data technologies and provide recommendations for adoption.

Required Qualifications:

- 812 years of hands-on experience in data engineering, with strong expertise in:

- SQL, Python, Apache Spark.

- ETL, data warehousing, and big data platforms.

- Cloud data services: AWS Glue, Redshift, S3, Athena (GCP/BigQuery, Snowflake is a plus).

- Proven experience leading and mentoring data engineering teams.

- Strong knowledge of data modeling, pipeline orchestration tools (e.g., Airflow, Control-M), and data governance practices.

- Excellent problem-solving, communication, and stakeholder management skills.

Domain Expertise (Must Have) :

- Strong background in the Insurance domain, specifically P&C (Property & Casualty).

- Proven ability to align data solutions with business requirements in insurance/finance sectors.

Education :

Bachelors or Masters degree in Computer Science, Information Technology, or a related field.

Other Requirements :

- Excellent verbal and written English communication skills.

- Must be available to join within 15 days.