Data Engineer - ETL/Data Pipeline

We are seeking a skilled Data Engineer with 3-4 years of experience in building and maintaining robust data pipelines. The ideal candidate will have a strong command of AWS services and be proficient in both SQL and NoSQL databases. Attention to detail, a commitment to data integrity, and the ability to manage large-scale data workflows are essential for this role.

Responsibilities :

- Design, develop, and maintain scalable data pipelines and ETL processes to handle large volumes of data efficiently.

- Ensure data integrity and reliability by implementing data validation, cleansing, and transformation processes.

- Collaborate with data scientists, analysts, and other stakeholders to define data requirements and support data-driven decision-making.

- Optimise data workflows and storage solutions to improve performance and reduce costs.

- Develop and maintain database schemas, queries, and stored procedures using SQL and NoSQL databases.

- Deploy and manage data infrastructure on cloud platforms like AWS or Azure, ensuring optimal performance and security.

- Implement best practices for data governance, security, and compliance.

- Monitor and maintain data systems in production, troubleshoot issues, and provide timely solutions.

- Stay up-to-date with the latest technologies and trends in data engineering, and apply them to the project when necessary.

- Write and maintain documentation related to data pipelines, data models, and other technical aspects of the project.

Requirements :

- Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.

- At least 3-4 years of experience in data engineering, with a focus on building and maintaining data pipelines and ETL processes.

- Strong proficiency in SQL and experience with database systems such as MySQL, PostgreSQL, or MongoDB.

- Experience with big data technologies like Apache Spark, Hadoop, or Kafka.

- Familiarity with cloud platforms like AWS, Azure, or Google Cloud, and experience with services like S3 Redshift, Dataflow, or

BigQuery.

- Strong problem-solving and analytical skills, with the ability to work independently and in a team environment.

- Good communication and interpersonal skills, with the ability to collaborate effectively with other team members and

stakeholders.

- Experience with data warehousing and data modeling is a plus.