As a Senior Data Engineer, you'll be part of a team of smart, highly skilled technologists who are passionate about learning and supporting cutting-edge technologies such as Spark, Scala, Pyspark, Databricks, Airflow, SQL, Docker, Kubernetes, and other Data engineering tools. These technologies are deployed using DevOps pipelines leveraging Azure, Kubernetes, Jenkins and Bitbucket/GIT Hub.

WHAT YOU'LL DO :

- Develop, test, troubleshoot, debug, and make application enhancements leveraging, Spark , Pyspark, Scala, Pandas, Databricks, Airflow, SQL as the core development technologies.

- Deploy application components using CI/CD pipelines.

- Build utilities for monitoring and automating repetitive functions.

- Collaborate with Agile cross-functional teams - internal and external clients including Operations, Infrastructure, Tech Ops

- Collaborate with Data Science team and productionize the ML Models.

- Participate in a rotational support schedule to provide responses to customer queries and deploy bug fixes in a timely and accurate manner

Qualifications :

WE'RE LOOKING FOR PEOPLE WHO HAVE :

- 6 to 9 Years of years of applicable software engineering experience

- Strong fundamentals with experience in Bigdata technologies, Spark, Pyspark, Scala, Pandas, Databricks, Airflow, SQL,

- Must have experience in cloud technologies, preferably Microsoft Azure.

- Must have experience in performance optimization of Spark workloads.

- Good to have experience with DevOps Technologies as GIT Hub, Kubernetes, Jenkins, Docker.

- Good to have knowledge of relational databases, preferably PostgreSQL.

- Excellent English communication skills, with the ability to effectively interface across cross-functional technology teams and the business

- Minimum B.S. degree in Computer Science, Computer Engineering or related field