Python Developer - ETL & Data Pipeline Automation

SWAI TECHNOLOGIES PRIVATE LIMITED

Bangalore

5 - 14 Years

4.8

22+ Reviews

Python ETL Data Pipeline Pandas LLM Generative AI Flask Microservices Architecture System Design

Posted on: 23/09/2025

Job Description

Job Role/Title : Python Developer ETL & Data Pipeline Automation

Experience Range : 5-14

Location : Bangalore(Hybrid) -looking for Immediate to 20 days notice

About the Role :

We are looking for a Senior Python Developer (L6) to join our engineering team and lead the development of robust, scalable backend systems while integrating cutting-edge AI/LLM solutions.

This role sits at the intersection of backend engineering, data pipelines, and AI/GenAI integration, requiring strong expertise in Python, ETL workflows, distributed systems, and large-scale system design. Beyond writing high-quality code, you will lead by example, mentor peers, and collaborate cross-functionally to deliver high-impact, data-driven, and AI-powered solutions.

Key Responsibilities :

System Architecture & Backend Engineering :

- Architect, design, and implement secure, scalable backend systems using Python.

- Develop and optimize ETL workflows and data pipelines leveraging Pandas, Hive SQL, and distributed data systems.

- Design and maintain RESTful APIs using Flask/FastAPI, ensuring modularity, scalability, and reliability.

- Integrate structured/unstructured data pipelines into backend services and AI workflows.

- Ensure high performance, fault tolerance, and observability across all services.

AI/LLM & Data Integration :

- Integrate Large Language Models (LLMs) and GenAI-powered features into core applications.

- Build robust AI-driven systems with safeguards against hallucinations and reliability issues.

- Leverage agentic frameworks and AI automation for workflow enhancement.

- Practical experience with OpenAI developer ecosystem (Completions, Responses, Files) and SDKs (Agents SDK, Web Search).

- Familiarity with AI debugging/monitoring tools (Phoenix, LangSmith, etc.).

- Configure, monitor, and optimize Elasticsearch clusters for indexing, querying, and analytics performance.

Engineering Excellence & DevOps :

- Define and enforce best practices in coding, testing, code reviews, and CI/CD automation.

- Ensure strong observability through logging, monitoring, and alerting.

- Write unit, integration, and functional tests to ensure high-reliability systems.

- Build and maintain CI/CD pipelines to support automated testing, smooth deployments, and rollback strategies.

Collaboration & Leadership :

- Partner with Data Scientists, Analysts, and cross-functional teams to enable data-driven applications and reporting.

- Mentor junior engineers and champion a culture of engineering excellence.

- Collaborate with stakeholders to align solutions with business and product goals.

- Document pipeline architecture, automation logic, and API specifications clearly.

Required Skills & Experience :

- 5 to 12+ years of professional Python development experience.

- Proven expertise in backend system design, microservices, and APIs at scale.

- Strong experience with data pipelines, ETL frameworks, and distributed data systems.

- Hands-on experience with Pandas, Hive SQL, Elasticsearch, and workflow orchestration.

- Expertise in LLMs, GenAI integrations, and AI-enhanced/agentic workflows.

- Proficiency in Flask/FastAPI for backend/API development.

- Strong database experience (e.g., PostgreSQL) across real-time and batch workloads.

- Solid knowledge of DevOps practices including CI/CD, monitoring, and observability.

- Excellent debugging, problem-solving, and communication skills.

- Demonstrated leadership in mentoring, cross-team collaboration, and stakeholder management