Senior Web Scraper

Title : Senior Web Scraper

Key Responsibilities :

- Web Crawling & Reverse Engineering : Apply a strong understanding of web crawling strategies. Experience with reverse engineering web applications to bypass anti-scraping measures will be highly advantageous.

- REST API Development : Lead the design and development of robust REST APIs to serve the data collected by scrapers and facilitate internal/external integrations.

- Backend Development : Build and maintain scalable backend services using Node.js and its relevant frameworks (like Express).

- Database & Data Management : Work effectively with various databases and data management tools such as MySQL, MongoDB, and Redis to store, process, and manage scraped data.

- Technical Troubleshooting : Leverage excellent problem-solving and analytical skills to diagnose and resolve complex technical challenges related to web scraping, API performance, and data integrity.

- Cloud Deployment : Utilize familiarity with cloud-based services such as AWS or Azure for deploying and managing scraping and backend infrastructure.

- Team Leadership & Mentorship : Apply strong communication and leadership skills to potentially guide junior team members and collaborate effectively with cross-functional teams

.

Required Skills & Qualifications :

- Proven experience in building large-scale and complex web scrapers.

- Good understanding of web crawling strategies.

- Expertise in REST API design and development.

- Strong understanding of web technologies and protocols such as HTML, CSS, and HTTP.

- Strong proficiency with JavaScript.

- Strong knowledge of Node.js and frameworks available for it (like Express).

- Experience working with databases and data management tools such as MySQL, MongoDB, and Redis.

- Excellent problem-solving and analytical skills.

- Familiarity with cloud-based services such as AWS or Azure.

- Strong communication and leadership skills.