Qentelli - AWS Data Engineer

JOB SUMMARY :

We are looking for an experienced AWS Data Engineer with hands-on experience in building and maintaining data pipelines, data platforms, and cloud-based data solutions on AWS.

The candidate should have strong experience in Python, SQL, AWS data services, ETL/ELT pipelines, and data warehousing, along with practical exposure to Generative AI applications.

ROLES AND RESPONSIBILITIES :

- Design, develop, and maintain scalable data pipelines on AWS.

- Build ETL/ELT processes to extract data from multiple sources, transform it, and load it into data platforms.

- Develop data solutions using AWS services such as S3, Glue, Lambda, Redshift, Athena, EMR, and Step Functions.

- Develop and optimize SQL queries for data processing and reporting.

- Write Python scripts and applications for data processing and automation.

- Build data models and prepare datasets for analytics and AI applications.

- Implement data validation and quality checks across data pipelines.

- Monitor data pipelines and troubleshoot failures, performance issues, and data-related problems.

- Implement appropriate security, access controls, encryption, and data governance practices on AWS.

- Collaborate with data scientists, software engineers, analysts, and business teams to understand requirements.

- Support data preparation for Generative AI applications, including collecting, cleaning, transforming, and organizing relevant data.

- Build data pipelines that support AI-powered search, question-answering, and document-based applications.

MANDATORY SKILLS :

- 4+ years of experience in Data Engineering.

- Strong hands-on experience with AWS data services (S3, Glue, Redshift, Athena, Lambda, IAM).

- Strong programming experience in Python and SQL.

- Experience with relational databases (PostgreSQL, MySQL, Oracle, or SQL Server).

- Practical experience working on applications that use Generative AI, including embeddings and vector-based search.

- Experience with Git, CI/CD, and production support.

PREFERRED SKILLS :

- Experience with AWS AI services, Amazon OpenSearch, Spark/PySpark, Apache Airflow, Terraform/CloudFormation, and Docker.

- Experience with document processing pipelines (PDF, Word, HTML, XML).