Senior Software Engineer - Big Data Platform

Description :

About the Role :

We are seeking an experienced Senior Software Engineer (Big Data Platform Engineer) to design, develop, and optimize large-scale data processing systems.

The ideal candidate will have strong expertise in distributed data frameworks, data pipelines, and cloud-based big data platforms, with the ability to translate complex business requirements into efficient, scalable solutions.

This role will start as an Individual Contributor (IC) with potential to move into a Team Lead position after 56 months based on performance.

Key Responsibilities :

- Design, build, and maintain Big Data pipelines and data lake architectures supporting high-volume data ingestion, processing, and analytics.

- Develop scalable data solutions using Spark, Hadoop, Hive, Kafka, and Airflow (or equivalent).

- Implement ETL/ELT workflows to manage data extraction, transformation, and loading from multiple sources.

- Collaborate with data scientists, analysts, and engineering teams to optimize data availability and reliability.

- Ensure data security, performance tuning, and compliance with governance policies.

- Deploy and manage data workloads on Cloud environments (AWS, Azure, or GCP).

- Troubleshoot production issues, perform root cause analysis, and continuously enhance system performance.

Requirements :

- Bachelors or Masters degree in Computer Science, Engineering, or a related field.

- 5- 10 years of experience in Big Data Engineering / Data Platform Development.

- Strong programming skills in Python, Scala, or Java.

- Hands-on experience with Spark, Hadoop, Hive, Kafka, Airflow, HBase, or similar tools.

- Solid understanding of data lake architecture, data warehousing concepts, and ETL/ELT design.

- Experience with Cloud-based data platforms (AWS Glue, EMR, Redshift, GCP BigQuery, or Azure Synapse).

- Strong problem-solving, debugging, and performance optimization skills.

- Excellent communication and collaboration abilities.

- Preference for stable career track (no frequent job changes)