Data Engineer Specialist

Role Overview :

As a Data Engineer Specialist, you will be at the forefront of building and maintaining our data infrastructure, ensuring seamless data flow and accessibility for our analytics and business intelligence teams. Your day-to-day will involve designing, developing, and deploying robust data pipelines, optimizing data storage solutions, and collaborating with data scientists and analysts to understand their data needs. You will work closely with cross-functional teams, including product managers and software engineers, to deliver data-driven insights that directly impact our business outcomes and enhance the user experience.

Key Responsibilities :

- Design and implement scalable and reliable data pipelines using AWS services to ingest, process, and transform large datasets for analytics and reporting purposes.

- Develop and maintain data models and database schemas to ensure data integrity, consistency, and performance across various data platforms.

- Collaborate with data scientists and analysts to understand their data requirements and provide them with the necessary data infrastructure and tools to perform their analyses effectively.

- Optimize data storage and retrieval processes to improve query performance and reduce data processing costs.

- Implement data quality checks and monitoring systems to identify and resolve data inconsistencies and errors.

- Automate data pipeline deployment and monitoring processes to ensure continuous data availability and reliability.

- Contribute to the development of data governance policies and procedures to ensure compliance with data privacy regulations.

- Troubleshoot and resolve data-related issues in a timely manner to minimize disruption to business operations.

Required Skillset :

- Demonstrated expertise in building and maintaining data pipelines using AWS services such as S3, EC2, Lambda, Glue, and Redshift.

- Proficient in Python programming and experience with PySpark for large-scale data processing.

- Strong SQL skills and experience with relational databases such as MySQL, PostgreSQL, or Oracle.

- Ability to design and implement data models and database schemas that meet business requirements.

- Excellent problem-solving and analytical skills with the ability to identify and resolve data-related issues.

- Strong communication and collaboration skills with the ability to work effectively in a team environment.

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

- Ability to work on-site in Hyderabad.