Oxycon 2021, the online web scraping conference, played host to major players in the European data industry.
Managing Director of Merit Data and Technology spoke on the ethics of data scraping, a matter of priority across all of Merit’s dealings.
The presentation by Con Conlon
, laid out an action guide for companies that strive to be ethical, covering legal aspects, the benefits of ethical data collection policies and the implications of such policies on customer expectations.
Con stressed the importance of data scraping in various departments of a company, including marketing and governance, and why ethical data collection policy should be a priority, including the following:
- Consistency across an organisation in data compliance and SOP
- Better code and maintenance outcomes
- Reduced costs
- Avoidance of reputational damage and legal costs
- It forces clients to be more conscious and specific about their requirements
When writing a data policy, Con Conlon suggested a few specific elements should be considered.
- What are the target sites and the weight of the policy on these areas 6
- The collection of only necessary data
- The adherence to Robots.txt files and tracking of changes
- The provision of user stringer whenever possible
Having an ethical data collection policy is integral to the management of any data within a business and should be revised every 6 months by the company’s tech team in adherence with any changes in relevant data policies.
One such opportunity for this will be the changes that will soon be made to the UK’s Data Protection policy which is going under consult in November 2021.
Related Case Studies
Automotive Data Aggregation Using Cutting Edge Tech Tools
An award-winning automotive client whose product allows the valuation of vehicles anywhere in the world and tracks millions of price points and specification details across a large range of vehicles.
High Speed Big Data Harvesting For The Oil, Gas & Energy Sector
Find out how we provided more than 515 scrapers that collects data 24/7, uninterrupted.