Robot Etiquette: web scraping and the law
Whether you call it web scraping, web crawling or spidering, programmatic data access and extraction belongs to the realm of technologies that have outpaced developments within the world of legislation. In the following piece, we provide an introduction to legal theories and concepts that have proven relevant to web scraping activities, as well as some widely accepted guidelines for good bot behaviour.