Senior Data Engineer - ETL/Python

Description :

Role : Senior Data Engineer - AI & Analytics

Position Overview :

- We are seeking a skilled Senior Data Engineer with strong experience in data architecture, pipeline development, and AI-driven engineering.

- The role focuses on building scalable data systems, integrating AI/LLM components, and collaborating across teams to deliver high-impact solutions.

Key Responsibilities :

- Design, develop, and optimize end-to-end data pipelines including ingestion, transformation, and orchestration.

- Build and maintain high-performance data models, ETL processes, and storage layers for large datasets.

- Integrate AI capabilities by implementing LLM workflows, embeddings, vector stores, and inference services.

- Collaborate with engineering, data, and product teams to ensure data quality, reliability, lineage, and security.

- Develop APIs, microservices, and interfaces that support analytics, reporting, and AI-driven features.

- Implement monitoring, logging, and automation frameworks for robust data operations.

- Conduct performance optimization of pipelines, queries, and data infrastructure.

- Evaluate and recommend modern tools and technologies in cloud, big data, and AI domains.

Requirements :

Required Skills & Experience :

- 58+ years of hands-on experience as a Data Engineer or related role.

- Strong programming expertise in Python / TypeScript / SQL / Spark / Databricks.

- Experience with Jupyter Notebooks for data exploration, prototyping, and validation of AI/data workflows.

- Solid knowledge of cloud platforms (Azure / AWS / GCP), data lakes/lakehouses, and distributed computing.

- Practical experience integrating AI/LLM APIs, vector databases (Pinecone, Chroma, etc.), and RAG workflows.

- Strong understanding of data warehousing, schema design, metadata management, and large-scale processing.

- Proficiency in orchestration tools like Airflow / Prefect / Dagster.

- Familiarity with containerization and deployment using Docker / Kubernetes.

- Experience building CI/CD pipelines for data and AI systems.

- Strong analytical and problem-solving skills.

Preferred Qualifications :

- Exposure to AI agent frameworks, LLM prompt automation, or retrieval pipelines.

- Understanding of enterprise security, RBAC, and compliance requirements.

- Prior experience working in large-scale enterprise data environments.

Soft Skills :

- Excellent communication and cross-functional collaboration skills.

- Ability to simplify complex technical concepts and convert them into actionable work.

- High ownership mindset and focus on delivering reliable, scalable solutions.

- Comfortable working independently as a high-impact individual contributor.

Benefits :

Why This Role is Important :

- You will contribute to the development of next-generation data and AI capabilities, enabling intelligence, automation, and insights across key business initiatives.

- This role offers significant opportunities for innovation and technical impact.