We are seeking a highly skilled Senior Data Engineer with strong hands-on experience in Python, AWS, ETL processes, and data streaming/integration frameworks.

The ideal candidate will be responsible for designing, architecting, and maintaining scalable data pipelines and infrastructure that supports analytics, applications, and business intelligence requirements.

The role requires expertise in large-scale data processing, cloud-based solutions, and integration technologies, with the ability to mentor junior engineers and contribute to architectural decision-making.

Key Responsibilities :

Data Pipeline & Architecture Development :

- Design, develop, and maintain scalable ETL/ELT pipelines for large datasets.

- Build real-time and batch processing systems to support analytics and data-driven applications.

- Implement data modelling, transformation logic, and workflow orchestration.

Cloud Engineering AWS :

- Work extensively on AWS cloud services for storage, compute, and data orchestration.

- Develop and manage solutions using Lambda, S3, Glue, RDS, Redshift, EMR, Step Functions, etc.

- Ensure high availability, fault tolerance, cost optimization, and security of cloud data systems.

Coding & System Integration (Python, Kafka, APIs) :

- Develop robust data processing scripts and services primarily using Python (4+ years).

- Build and optimize data streaming and ingestion pipelines using Kafka (23 years).

- Integrate external data sources via API-based ingestion frameworks (2+ years).

Testing, Quality & Optimization :

- Implement automated testing frameworks for data pipelines & workflows.

- Conduct peer code reviews, performance benchmarking, and code optimization.

- Ensure data quality, lineage, auditing, and reliability across systems.

Soft Skills & Competencies :

- Problem-solving mindset with strong analytical ability.

- Effective communication and collaboration with cross-functional teams.

- Ability to work independently and lead feature-level architecture.

- Strong documentation, design thinking, and code ownership