Job Title : AWS Data Engineer

Location : Bangalore - Whitefield (Hybrid)

Experience : 6-10 Years

Employment Type : Full-Time, Permanent

Job Summary :

We are looking for an experienced AWS Data Engineer to design, build, and maintain scalable cloud-native data platforms on AWS. The ideal candidate will have strong expertise in developing enterprise-grade data pipelines, implementing Infrastructure-as-Code (IaC), automating deployments, and working with modern cloud data services. You will be responsible for building reliable, secure, and high-performance data solutions that enable analytics and business intelligence across the organization. This role requires strong hands-on experience with AWS, Python, Apache Spark, Terraform/CloudFormation, and modern DevOps practices.

Key Responsibilities :

1. Data Engineering :

- Design, develop, and optimize scalable ETL/ELT pipelines using Python and Apache Spark.

- Build and maintain reliable data ingestion, transformation, and processing workflows.

- Develop cloud-native data solutions capable of handling large-scale structured and unstructured datasets.

- Optimize data processing jobs for performance, scalability, and cost efficiency.

- Implement reusable frameworks and automation for data engineering processes.

- Ensure data quality, integrity, and consistency across enterprise data platforms.

2. AWS Cloud Development :

- Design and implement enterprise data solutions using AWS services.

- Develop and maintain data lakes using Amazon S3.

- Build serverless data processing workflows using AWS Lambda.

- Develop and optimize ETL jobs using AWS Glue.

- Query and analyze large datasets using Amazon Athena.

- Manage cloud compute resources using Amazon EC2.

- Collaborate with cloud engineering teams to improve scalability, reliability, and security.

3. Infrastructure Automation :

- Provision and manage cloud infrastructure using Infrastructure-as-Code.

- Automate infrastructure deployment using Terraform, AWS CloudFormation, or AWS CDK.

- Maintain cloud environments following security, governance, and compliance standards.

- Optimize cloud infrastructure utilization and cost.

4. DevOps & CI/CD :

- Build and maintain CI/CD pipelines for data engineering projects.

- Automate application deployment, testing, and release processes.

- Manage source code repositories using Git.

- Work closely with DevOps teams to improve deployment efficiency and operational stability.

5. Monitoring & Data Operations :

- Monitor data pipelines and cloud infrastructure using DataDog and AWS monitoring tools.

- Perform root cause analysis for production incidents.

- Ensure high availability and reliability of production data platforms.

- Implement proactive monitoring, logging, and alerting mechanisms.

- Support production releases and operational activities.

6. Data Modeling :

- Design scalable data models supporting analytics and reporting.

- Optimize database performance and query execution.

- Implement data governance and lifecycle management best practices.

- Work with business teams to understand reporting and analytical requirements.

7. Collaboration :

- Collaborate with Data Scientists, Data Analysts, Software Engineers, Product Managers, and Business Stakeholders.

- Participate in Agile ceremonies including Sprint Planning, Daily Stand-ups, Reviews, and Retrospectives.

- Mentor junior engineers and contribute to engineering best practices.

- Prepare technical documentation and architecture recommendations.

Must-Have Skills :

- 6+ years of hands-on experience in Data Engineering.

- Strong experience building enterprise data pipelines on AWS Cloud.

- Excellent programming skills in Python.

- Strong expertise in Apache Spark/PySpark.

- Experience working with AWS services such as S3, Glue, Lambda, EC2, Athena, and CloudWatch.

- Hands-on experience with Infrastructure-as-Code using Terraform, AWS CloudFormation, or AWS CDK.

- Experience implementing CI/CD pipelines and DevOps best practices.

- Strong understanding of Data Modeling, Data Warehousing, and Data Operations.

- Experience with monitoring and observability tools such as DataDog.

- Strong scripting and automation skills using Python, Bash, or Shell scripting.

- Experience working with Git and version control systems.

- Strong analytical, troubleshooting, and problem-solving skills.

- Excellent verbal and written communication skills.

- Ability to work independently and take ownership of deliverables.

Candidate Profile :

- 6-10 years of professional experience in Data Engineering.

- Proven experience developing cloud-native data platforms on AWS.

- Strong understanding of distributed data processing and big data technologies.

- Experience working on enterprise-scale data engineering projects.

- Familiarity with Infrastructure-as-Code and cloud automation.

- Strong understanding of DevOps principles and CI/CD implementation.

- Ability to work in a fast-paced Agile development environment.

- Proactive mindset with excellent ownership and stakeholder management skills.

Education :

- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.

Additional Advantage :

- Experience working in multi-cloud environments (AWS, Azure, or GCP).

- Exposure to the Volkswagen CAP Platform.

- Experience with Apache Airflow, Kafka, Snowflake, or Databricks.

- AWS Professional Certifications are an added advantage.