Description : A fast growing AI startup is looking to hire talented Senior Data Engineer. This opportunity is for a client of Talentfil.

Location : Remote

About the Role :

Our client is looking for a Senior Data Engineer with strong expertise in Python, web scraping, and scalable ETL pipelines.

The role focuses on building reliable data systems, maintaining data quality, and enabling high-performance analytics and ML workflows.

Key Responsibilities :

- Build and maintain scalable web-scraping pipelines.

- Develop robust ETL processes for large-scale data ingestion and transformation.

- Implement strong data quality checks, validation, and monitoring.

- Establish data observability frameworks (freshness, lineage, drift detection).

- Collaborate with analytics and product teams to deliver clean, structured datasets.

- Troubleshoot pipeline issues and ensure high system reliability.

- Mentor junior engineers and support best-practice architecture.

Required Qualifications :

- 5+ years of data engineering experience.

- Advanced Python skills (Scrapy, Selenium, BeautifulSoup, etc.).

- Strong ETL/ELT experience (Airflow, dbt, Spark, etc.).

- Hands-on experience with web scraping and handling unstructured data.

- Strong SQL and experience with cloud data platforms (AWS/GCP/Azure).

- Knowledge of version control and CI/CD workflows.

Preferred :

- Experience with anti-bot systems, proxies, CAPTCHAs.

- Exposure to data lakes, streaming (Kafka/Spark).

- Familiarity with Docker/Kubernetes.