Data Engineer - ETL/Data Warehousing

Description :

Job Title : Fullstack Data Engineer

Experience Required :

- Design, develop, and maintain end-to-end data pipelines on AWS, including data preparation, extraction, transformation, loading, normalization, aggregation, warehousing, data lakes, and governance

- Develop and maintain Data Warehouses and Lakehouse architectures

- Work with modern data architectures such as Data Lake, Data Warehouse, Lakehouse, and Data Mesh

- Design and implement data models and ensure high-quality, scalable data architecture

- Ensure cost-effective management of data pipelines and resources

- Implement CI/CD processes for data pipelines and infrastructure

- Work with Agile methodologies (Scrum/Kanban) for iterative delivery

- Scope, estimate, and deliver committed work independently and as part of the agile team

- Collaborate with teams to ensure data quality, security, and governance across platforms

Required Skills & Technologies :

- Programming : Python, PySpark, Bash/Shell Scripting

- Databases : SQL, NoSQL (MongoDB), Snowflake

- Data Orchestration & ETL : Airflow, Glue, DBT

- Streaming / Messaging : Kafka, Kinesis

- Cloud / Infrastructure : AWS, Terraform

- Version Control : GitHub

- Other : APIs, Lakehouse architecture knowledge