Role: Senior Data Engineer

Experience: 7+ Years

Location: Pune

Joining: Immediate to 30 Days' Notice Period

Role Summary:

We are looking for a skilled Sr. Data Engineer with strong expertise in Python and cloud-native technologies to design, develop, and maintain scalable data pipelines and distributed systems. The ideal candidate should have hands-on experience with ETL/ELT pipelines, workflow orchestration, AWS cloud services, data visualization, distributed architectures, and modern data engineering practices.

Key Responsibilities:

- Design, develop, and maintain scalable ETL/ELT data pipelines.

- Build and manage distributed and event-driven architectures for large-scale data processing.

- Develop backend services and data infrastructure using Python.

- Implement DAG-based workflow orchestration using Airflow, Prefect, or Dagster.

- Work with AWS services including Lambda, S3, ECS/EKS, SQS, and CloudWatch.

- Design and optimize data models and schema structures.

- Develop analytics dashboards and reporting solutions using Amazon QuickSight.

- Implement CI/CD pipelines and Infrastructure as Code (IaC).

- Monitor and maintain data platforms using observability and monitoring tools.

- Collaborate with Data Scientists, ML Engineers, and Backend Developers.

- Support AWS cloud migration and modernization initiatives.

Mandatory Skills:

- Strong proficiency in Python.

- Experience with Docker and Kubernetes.

- Strong understanding of distributed and event-driven architectures.

- Hands-on experience building ETL/ELT pipelines.

- Strong knowledge of data modeling and schema design.

- Experience with Airflow, Prefect, or Dagster.

- Hands-on experience with AWS Lambda, S3, ECS/EKS, SQS, and CloudWatch.

- Experience with CI/CD and Infrastructure as Code (IaC).

- Hands-on experience with Amazon QuickSight for dashboards, reporting, and analytics.

Good to Have:

- AWS Application Migration Service (MGN/AMC) or similar migration technologies.

- Logstash or other data ingestion frameworks.

- OpenSearch / Elasticsearch.

- Kafka / Kinesis and streaming technologies.

- Prometheus, Grafana, or ELK Stack.

- MLOps lifecycle knowledge.

- Vector databases and embedding pipelines.

- Large-scale indexing and search optimization.

- RAG, Natural Language Search, GenAI, knowledge graphs, prompt engineering, and embedding models.

- Knowledge of Golang, Rust, Java, C/C++, or other compiled languages.