We are looking for an experienced Python Lead Data Engineer to design, develop, and optimize scalable data platforms and enterprise-grade data pipelines. The ideal candidate will have strong expertise in Python, FastAPI, Apache Airflow, Snowflake, SQL, ETL/ELT, and CI/CD practices while leading a team of engineers to build high-performance data solutions for modern analytics and business intelligence platforms.

Key Responsibilities :

- Lead the design and development of scalable, secure, and high-performance data engineering solutions.

- Build and maintain robust ETL/ELT pipelines to ingest, transform, and load data from multiple structured and unstructured sources.

- Develop RESTful APIs and backend services using Python and FastAPI.

- Design, schedule, and monitor workflows using Apache Airflow.

- Build and optimize enterprise data warehouses using Snowflake.

- Write complex SQL queries for data transformation, reporting, and analytics.

- Implement data quality, validation, and monitoring mechanisms.

- Collaborate with Data Scientists, BI teams, Product Managers, and Business stakeholders to understand data requirements.

- Optimize data processing workflows for performance, scalability, and cost efficiency.

- Automate deployment processes using GitLab CI/CD pipelines.

- Ensure adherence to coding standards, security policies, and data governance best practices.

- Mentor junior engineers and conduct code reviews.

- Participate in architecture discussions and recommend modern data engineering best practices.

- Troubleshoot production issues and provide timely resolutions.

Required Skills :

1. Programming :

- Strong expertise in Python

- Experience with FastAPI

- Object-Oriented Programming

- Exception handling and multithreading concepts

- API development and integration

2. Data Engineering :

- ETL/ELT pipeline development

- Data ingestion and transformation

- Batch and workflow orchestration

- Data validation and quality checks

- Data modeling fundamentals

3. Workflow Orchestration :

- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.

- 9 to 12 years of experience in Data Engineering.

- Strong hands-on expertise in Python-based backend development.

- Experience designing enterprise-scale ETL pipelines.

- Proven expertise with Snowflake and SQL optimization.

- Hands-on experience implementing Apache Airflow workflows.

- Experience working with GitLab repositories and CI/CD pipelines.

- Strong understanding of software engineering best practices.

- Excellent analytical and problem-solving skills.

- Strong communication and stakeholder management abilities.

- Experience with cloud platforms (AWS, Azure, or GCP).

- Docker and Kubernetes.

- Data Lake architecture.

- Spark or PySpark.

- Kafka or other streaming technologies.

- Infrastructure as Code.

- Agile/Scrum methodology.

- Data governance and security practices.

- Excellent leadership capabilities.

- Strong analytical thinking.

- Problem-solving mindset.

- Effective communication skills.

- Team collaboration.

- Ownership and accountability.

- Ability to work in a fast-paced environment.