Role Overview :

As a Big Data Engineer at Sysmind Tech, you will be responsible for designing, developing, and maintaining our clients' big data infrastructure.

You will work closely with data scientists, analysts, and other engineers to build and optimize data pipelines, ensuring data quality and accessibility for various analytical and reporting needs.

Your contributions will directly impact the ability of our clients to extract valuable insights from their data and make data-driven decisions.

Key Responsibilities :

- Design and implement scalable and reliable data pipelines using Hadoop, Spark, and other big data technologies to process large volumes of data for various business applications.

- Develop and maintain data models and schemas to ensure data consistency and integrity across different data sources for accurate reporting and analysis.

- Optimize data processing and storage solutions to improve performance and reduce costs for efficient data management.

- Collaborate with data scientists and analysts to understand their data requirements and provide them with the necessary data infrastructure and tools for their analytical projects to enable data-driven decision making.

- Implement data quality checks and monitoring systems to ensure data accuracy and reliability for trustworthy insights.

- Troubleshoot and resolve data-related issues to minimize downtime and ensure data availability for critical business operations.

- Deploy and manage big data infrastructure on cloud platforms such as AWS, Azure, or GCP to provide scalable and cost-effective solutions.

Required Skillset :

- Demonstrated ability to design and implement large-scale data processing pipelines using Hadoop, Spark, and related technologies.

- Proven expertise in programming languages such as Scala and Python for data manipulation and analysis.

- Solid understanding of SQL and relational database concepts for data querying and management.

- Experience with cloud platforms (AWS, Azure, GCP) for deploying and managing big data infrastructure.

- Strong Linux/Unix skills for system administration and scripting.

- Excellent problem-solving and communication skills to effectively collaborate with cross-functional teams.

- Ability to adapt to new technologies and learn quickly in a fast-paced environment.

- Bachelor's or Master's degree in Computer Science or a related field.