AppSierra Solutions - AWS Data Engineer - Python/PySpark

Description :

We are seeking a talented and motivated AWS Data Engineer with expertise in Python, CI/CD practices, DataBricks, and PySpark to join our innovative team. In this role, you will be responsible for developing and maintaining robust data pipelines that transform raw data into valuable information for our stakeholders. You will work with cloud technologies and big data solutions to support our data-driven decision-making processes. Your primary focus will be to ensure the integrity, efficiency, and accessibility of data across various environments while leveraging the power of AWS services.

This position offers an exciting opportunity to collaborate with cross-functional teams, implement best practices in data engineering, and contribute to the overall data strategy of the organization. The ideal candidate will possess a strong foundation in data architecture and cloud computing, combined with practical experience in CI/CD methodologies to facilitate seamless deployment of data solutions. If you are passionate about data and driven by results, we invite you to apply and be part of a dynamic team that values innovation and growth.

Responsibilities :

- Design and implement scalable data pipelines using AWS services and DataBricks.

- Develop data transformation scripts using Python and PySpark to ensure data processing efficiency.

- Collaborate with data scientists and analysts to understand their data needs and implement solutions accordingly.

- Set up and maintain CI/CD pipelines for data integration and deployment processes.

- Monitor data workflows and troubleshoot issues to ensure high data quality and reliability.

- Optimize data storage solutions for cost and performance using AWS technologies.

- Document data architecture, processes, and workflows to facilitate knowledge sharing within the team.

Requirements :

- Bachelor's degree in Computer Science, Information Technology, or a related field.

- Proven experience as a Data Engineer working with AWS cloud services.

- Strong proficiency in Python for data manipulation and analysis.

- Hands-on experience with DataBricks and PySpark for large-scale data processing.

- Familiarity with CI/CD principles and tools to automate deployment processes.

- Solid understanding of data warehousing concepts and ETL processes.

- Ability to work collaboratively in a team and communicate effectively with technical and non-technical stakeholders.