Big Data Engineer - Hadoop/Spark/Scala

Key Responsibilities

- Design, develop, and optimize robust big data pipelines for data ingestion, processing, and transformation using tools like Hadoop, Spark, Scala, Hive etc.

- Manage and maintain large-scale distributed data storage and processing systems.

- Collaborate with data scientists and analysts to implement and scale data models and machine learning workflows.

- Ensure data quality, data governance, and security standards are adhered to.

- Monitor system performance and troubleshoot data-related issues.

- Write efficient, reusable, and scalable code in languages such as Python, Scala, or Java.

- Automate data workflows and build monitoring dashboards.

- Work with cloud platforms such as AWS, Azure, or Google Cloud to manage data infrastructure.

- Stay current with emerging big data technologies and best practices.