Senior Data Engineer

Job Title : Senior Data Engineer

Location : Remote

Experience : 6+ years

Shift Timings : 11 : 00 AM - 8 : 00 PM IST

Work Days : 5 days

What You'll Be Doing :

- Design and build parts of our data pipeline architecture for extraction, transformation, and loading (ETL) of data from a wide variety of data sources using the latest Big Data technologies

- Identify, design, and implement internal process improvements : automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability

- Work with stakeholders including Executive, Product, Data, and Design teams to assist with data-related technical issues and support their data infrastructure needs

- Collaborate with machine learning, data, and analytics experts to drive innovation, accuracy, and greater functionality in our data systems

Qualifications :

- Bachelors degree in Engineering, Computer Science, or a relevant field

- 3+ years of relevant and recent experience in a Data Engineer role

- 3+ years of recent experience with Apache Spark and solid understanding of its fundamentals

- Deep understanding of Big Data concepts and distributed systems

- Strong coding skills in Scala, Python, Java, and/or other languages with the ability to switch between them efficiently

- Advanced working knowledge of SQL

- Cloud experience with Databricks

- Strong understanding of Delta Lake architecture and working with data formats such as Parquet, JSON, and CSV

- Comfortable working in a Linux shell environment and writing scripts as needed

- Experience working in an Agile environment

- Machine Learning knowledge is a plus

- Ability to work independently and deliver stable, efficient, and reliable software

- Excellent written and verbal communication skills in English

- Experience supporting and collaborating with cross-functional teams in a dynamic environment