Big Data Developer - Hadoop/Apache Spark

Description :

Job Title : Big Data Developer

Location : Hyderabad

Work Mode : 5 Days Work From Office

Experience : 4+ Years

Job Summary :

We are seeking a skilled Big Data / Hadoop Developer with hands-on experience in designing and building scalable data processing solutions. The role involves working extensively with the Hadoop ecosystem, Spark, and Hive to develop high-performance ETL pipelines and optimize large-scale data platforms.

Key Responsibilities :

- Design, develop, and maintain scalable big data solutions using Hadoop technologies

- Build and optimize data processing workflows using Apache Spark and Hive

- Develop and manage robust ETL pipelines to ensure data accuracy and reliability

- Collaborate with data architects, analysts, and business stakeholders to convert requirements into technical solutions

- Perform data validation, quality checks, and troubleshoot pipeline issues

- Optimize query performance, data storage, and cluster resource utilization

- Ensure adherence to data security, governance, and compliance standards

Required Skills & Experience :

- 4+ years of hands-on experience in Big Data development

- Strong expertise in the Hadoop ecosystem (HDFS, MapReduce, YARN)

- Proficiency in Apache Spark (Batch & Streaming) and Hive

- Experience in designing and maintaining large-scale ETL pipelines

- Strong understanding of data partitioning, performance tuning, and optimization techniques

- Working knowledge of NoSQL databases (HBase, Cassandra, MongoDB preferred)

- Proficiency in at least one programming or scripting language : Java, Scala, Python, or Shell

- Excellent analytical and problem-solving skills

Nice to Have :

- Experience with real-time data processing

- Exposure to data governance and security frameworks

- Experience working in large-scale enterprise data environments