DataOps Engineer - ETL/Python

Position Summary :

As a DataOps Engineer, you will be a critical part of our data engineering team, working at the intersection of software engineering, DevOps, and data analytics.

You'll be responsible for creating and maintaining secure, scalable, and production-grade data pipelines and infrastructure.

Your work will directly support advanced analytics, machine learning, and real-time decision-making capabilities for our clients.

Key Responsibilities :

- Design, develop, and manage robust, scalable, and efficient ETL/ELT pipelines using Python and modern DataOps methodologies.

- Implement data quality checks, pipeline monitoring, and error handling mechanisms.

- Build data solutions using cloud-native services on AWS (e. , S3, ECS, Lambda, CloudWatch).

- Containerize applications using Docker and orchestrate using Kubernetes for scalable deployments.

- Work with infrastructure-as-code tools and CI/CD pipelines to automate deployments.

- Design and optimize data models using PostgreSQL, Redis, and PGVector for high-performance storage and retrieval.

- Support feature stores and vector-based storage for AI/ML applications.

- Drive Agile ceremonies (daily stand-ups, sprint planning, retrospectives) and ensure successful sprint delivery.

- Review pull requests (PRs), conduct code reviews, and enforce security and performance standards.

- Collaborate closely with product owners, analysts, and architects to refine user stories and technical requirements.

Required Skills & Qualifications :

- 10+ years of experience in Data Engineering, DevOps, or Software Engineering roles with a focus on data products.

- Strong hands-on experience with Python, Docker, Kubernetes, and AWS (especially S3 and ECS).

- Proficient with relational and NoSQL databases such as PostgreSQL, Redis, and experience with PGVector is a strong plus.

- Deep understanding of CI/CD pipelines, GitHub workflows, and modern source control practices.

- Experience working in Agile/Scrum environments with strong collaboration and communication skills.

- Passion for building clean, well-documented, and scalable code in a collaborative environment.

- Familiarity with DataOps principles, including automation, testing, monitoring, and deployment of data pipelines