- Design, develop, and maintain scalable data pipelines and ETL processes leveraging AWS services such as S3, Glue, EMR, Lambda, and Redshift.
- Collaborate with data scientists and analysts to understand data requirements and implement solutions that support analytics and machine learning initiatives.
- Optimize data storage and retrieval mechanisms to ensure performance, reliability, and cost-effectiveness.
- Implement data governance and security best practices to ensure compliance and data integrity.
- Troubleshoot and debug data pipeline issues, providing timely resolution and proactive monitoring.
- Stay abreast of emerging technologies and industry trends, recommending innovative solutions to enhance data engineering capabilities.
Requirements :
- Must have minimum 6+ years of experience in Data Engineering, with strong experience designing, developing, and maintaining scalable data pipelines and ETL processes.
- Must have 3+ years of hands-on experience with AWS data engineering services, particularly S3, AWS Glue, EMR, Lambda, and Redshift.
- Must have strong programming experience in Python, Java, or Scala, with hands-on development of data processing and pipeline solutions.
- Must have strong expertise in SQL, data warehousing, and database technologies, including experience with both SQL and NoSQL databases.
- Must have hands-on experience working with Big Data technologies and distributed data processing, including designing and optimizing large-scale data pipelines.
- Must have experience with data pipeline troubleshooting, monitoring, data quality, governance, and security best practices.
Mandatory Requirements :
- Mandatory (Education) : B.Tech / B.E./ M.Tech
- Mandatory (Notice Period) : Immediate joiners or candidates who can join within 2 weeks.