This role focuses on designing and deploying scalable web scraping solutions using Python, Playwright, and AWS serverless services. The engineer will build automated workflows that navigate complex, JavaScriptheavy, and authenticationprotected sites, ensuring reliable data extraction at scale.

Key Responsibilities :

- Assist with the design and implementation of advanced scraping solutions using Python, Playwright, and

AWS services.

- Automate interactions with JavaScript-heavy and authentication-secured websites, including handling MFA, CAPTCHAs, and session or token-based login flows.

- Architect scraping pipelines using serverless AWS components such as Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.

- Build systems that scale to support high volumes of data extraction with fault tolerance, retries, and intelligent logging.

- Integrate and manage complex workflows across multiple portals, APIs, and data sources.

- Contribute to architectural decisions, tooling, and best practices.

- Responsible for code reviews and deployments.

Required Skills :

- Proficiency in Python and Playwright for web automation.

- Experience with AWS serverless services: Lambda, Step Functions, S3, CloudWatch, Secrets Manager.

- Strong understanding of authentication flows, MFA, CAPTCHAs, and session/token management.

- Ability to design faulttolerant, scalable data pipelines with retries and logging.

- Familiarity with integrating multiple APIs and data sources.