- Experienced in Python, with knowledge of Scrapy framework
- Maintaining the running web crawlers full stack application
- Design, build web crawlers to scrape data and URLs.
- Integrate the data crawled and scraped into our databases
- Create more/better ways to crawl relevant information
- Understanding of data privacy policies (esp. GDPR) and personally identifiable information
- 3+ years of experience in Python.
- Experience with web crawler projects.
- Experience in working with MySQL and SQL is plus.
- Primary Skills – Python, SQL / MySQL, Scrapy Framework via Python;
- Secondary Skills – AWS, XPath / CSS Selectors, JSon, XML, GIT, Proxy, Firwalls, Pandas, Jupyter, Redis
- Good Communication skills.
- A target and customer oriented way of working.