Posted on: 05/12/2025
Description :
- Develop and manage robust ETL pipelines using Apache Spark (Scala).
- Understand Spark concepts, performance optimization techniques, and governance tools.
- Develop a highly scalable, reliable, and high-performance data processing pipeline to extract, transform, and load data from various systems to the Enterprise Data Warehouse/Data Lake/Data Mesh hosted on AWS or Azure.
- Collaborate cross-functionally to design effective data solutions.
- Implement data workflows utilizing AWS Step Functions or Azure Logic Apps for efficient orchestration.
- Leverage AWS Glue and Crawler or Azure Data Factory and Data Catalog for seamless data cataloging and automation.
- Monitor, troubleshoot, and optimize pipeline performance and data quality.
- Maintain high coding standards and produce thorough documentation.
- Contribute to high-level (HLD) and low-level (LLD) design discussions.
Technical Skills :
Did you find something suspicious?
Posted By
Posted in
Data Engineering
Functional Area
Data Engineering
Job Code
1585472
Interview Questions for you
View All