Data Software Engineer - Big Data Tech Stack

Job Overview :

We are looking for a seasoned Data Software Engineer with deep experience in building and scaling distributed data solutions. The ideal candidate is well-versed in Apache Spark, Python, and Azure Databricks, with a strong foundation in data engineering principles and modern cloud architectures.

Key Responsibilities :

- Write efficient, scalable, and reusable Python code for data transformation and ETL

- Integrate data from various sources including SQL/NoSQL databases, ERP systems, and flat files

- Optimize Spark jobs and tune performance for enterprise-scale workloads

- Implement and manage data flows on Azure Databricks

- Work with real-time data streaming platforms (Kafka, Spark Streaming, etc.)

- Collaborate with cross-functional teams in Agile environments

- Mentor junior team members and participate in code reviews

Must-Have Skills :

- Expertise in Apache Spark (core & streaming)

- Strong programming skills in Python

- Hands-on with Azure Databricks or AWS equivalents

- Solid knowledge of Hadoop ecosystem : HDFS, MapReduce, Sqoop

- Familiarity with Hive, Impala, SQL (queries, joins, procedures)

- Experience with Kafka / RabbitMQ

- Exposure to NoSQL databases : MongoDB, HBase, Cassandra

- Strong understanding of ETL frameworks and data modeling principles

- Experience in cloud-native data services and DevOps/data ops practices

Good to Have :

- Exposure to Apache Storm or other real-time frameworks

- Understanding of CI/CD for data pipelines

- Prior experience leading or mentoring a data team

Other Details :

Joining Time : [ Immediateto90 days preferred]