Data Engineer - AWS

Job Description :

Key Responsibilities :

- Lead a technical team or squad to deliver data engineering projects in a timely and efficient manner.

- Ensure high-performance and fault-tolerant solutions for handling large-scale data sets and real-time data streams.

- Collaborate with data scientists, analysts, and other engineers to gather requirements and build data

solutions for analytics and machine learning models.

- Optimize existing data processes and pipelines for speed, scalability, and cost-efficiency.

- Write and optimize complex queries in SQL, PySpark, and Python to process and transform large data sets.

- Design and implement data architectures that support data lakes, data warehouses, and analytics platforms.

- Monitor, troubleshoot, and resolve any issues related to data pipelines, ensuring high data quality and system reliability.

- Ensure compliance with data governance, privacy, and security standards.

Qualifications & Experience :

- 3+ years of experience leading technical teams or squads in an engineering capacity.

Skills & Expertise :

- Advanced programming experience with PySpark, Python, and SQL.

- Proven experience building, deploying, and managing scalable data pipelines on cloud platforms (preferably AWS).

- Strong understanding of data architecture, big data platforms, and real-time data processing.

- Experience with version control, CI/CD processes, and automation tools.

- Strong problem-solving and troubleshooting skills.

- Excellent communication and leadership abilities.

Preferred Skills :

- Experience with data warehousing solutions such as Amazon Redshift, Snowflake, or similar platforms.

- Familiarity with containerization and orchestration tools like Docker and Kubernetes.

- Knowledge of data governance, security best practices, and data privacy regulations