Job Overview :

We are looking for an experienced Senior Data Engineer with strong hands-on expertise in Python, Big Data technologies, SQL, Apache Spark, and AWS. The role involves building scalable data pipelines, managing large-scale data processing, and supporting analytics and business intelligence initiatives. This is not a Snowflake Developer role; basic exposure to Snowflake is sufficient.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines using Python and Apache Spark

- Process and manage large volumes of structured and unstructured data

- Optimize batch and streaming data processing workflows

- Develop and manage cloud-based data platforms on AWS (S3, EMR, Glue, Redshift, Lambda, etc.)

- Write and optimize complex SQL queries for analytics and reporting

- Ensure data quality, performance, reliability, and security

- Collaborate with analytics, data science, and business teams

- Troubleshoot production issues and perform performance tuning

- Mentor junior engineers and contribute to architectural decisions

Required Skills & Qualifications :

- 7 - 12 years of experience in Data Engineering or Big Data roles

- Strong proficiency in Python

- Hands-on experience with Apache Spark (PySpark preferred)

- Strong SQL skills with performance tuning experience

- Experience with Big Data ecosystems (HDFS, Hive, Kafka as applicable)

- Extensive hands-on experience with AWS cloud services

- Good understanding of data modeling and data warehousing concepts

- Experience working in Linux environments

- Strong analytical and communication skills

Good to Have :

- Basic exposure to Snowflake (not a Snowflake developer role)

- Experience with real-time or streaming data pipelines

- Knowledge of CI/CD pipelines and DevOps practices

- Experience working in Agile/Scrum teams