Role Description :

This is a full-time on-site role for a Data Engineer based in Navi Mumbai or Pune.

The Data Engineer will focus on designing, developing, and maintaining efficient data engineering systems.

Responsibilities include creating and managing data models, building ETL pipelines, optimizing data workflows, and ensuring data quality for advanced analytics and reporting purposes.

Collaboration with cross-functional teams to understand project requirements and support organizational data-driven initiatives will also be a key part of this role.

Key Responsibilities :

- Design, develop, and maintain data pipelines using Databricks and Apache Spark.

- Extract, transform, and load (ETL) data from various sources(structured and unstructured) into the Databricks platform.

- Optimize data pipelines for performance and scalability.

- Collaborate with data analysts and data scientists to understand data requirements and provide data solutions.

- Implement data quality checks and monitoring mechanisms.

- Troubleshoot and resolve data pipeline issues.

- Stay up-to-date with the latest data engineering technologies and best practices.

- Participate in code reviews and contribute to documentation to ensure best practices are followed.

Required Skills and Experience :

- Bachelor's degree in Computer Science, Data Science, Engineering, or a related field.

- 8+ years of experience in data engineering or related fields.

- Strong proficiency in Python or Scala programming languages.

- Hands-on experience with Databricks and Apache Spark.

- Knowledge of SQL and data warehousing concepts.

- Experience with cloud platforms (AWS, Azure, GCP).

- Understanding of data modeling and data warehousing principles.

- Strong problem-solving and analytical skills, with attention to detail.

- Excellent communication and collaboration skills.

- Exposure to CI/CD practices in Databricks environments.

- Basic knowledge of machine learning lifecycle and model deployment on Databricks.

- Experience with data orchestration tools such as Apache Airflow.

Bonus Skills :

- Exposure to CI/CD practices in Databricks environments.

- Basic knowledge of machine learning lifecycle and model deployment on Databricks.

- Experience with data orchestration tools such as Apache Airflow.