Data Engineer - Web Scraping

Job Role : Data Engineer - Web Scraping

Work Experience : 3- 5 years

Roles and responsibilities :

- Maintain and improve the existing scrapers

- Solve bottlenecks such as captchas, anti-bots, blockers

- Solve bugs arising from ongoing scraping jobs

Mandatory Skills required :

- 2+ years of experience in building scrapers with Python based libraries such as requests, aiohttp, httpx, selenium, playwright

- In depth knowledge of how different types of websites work

- 2+ years of experience with SQL databases - MySQL/PSQL

- 2+ years of experience with NoSQL databases - MongoDB

- 1+ years of experience with technologies like Proxy, Captcha solvers

- 1+ years of experience with Git

- 1+ years of experience with Agile development cycle

Good to have :

- Understanding of ETL pipelines

- Contribution in any open-source projects

Behavioral expectations :

- Passion for results and excellence

- Team player

- Must be able to get the job done by working collaboratively with others

- Be inquisitive and an analytical mind; out-of-the-box thinking

- Prioritize among competing opportunities, balance consumer needs with business and product priorities, and clearly articulate the rationale behind product decisions

- Straightforward and professional

- Good communicator

- Maintain high energy and motivate

- A do-it-yourself orientation, consistent with the companys roll-up the- sleeves culture

- Proactive