On 30 May 2024, the Italian data protection authority (the Garante) issued the Guidance on scraping and generative AI (in Italian, Guidance). The Guidance also reflects the contributions received by the Garante in the course of its fact-finding investigation on scraping (in Italian), which it launched in December 2023.

The Guidance briefly addresses data protection concerns around the mass collection of personal data from the Internet via scraping techniques and the subsequent processing of such data to train generative AI algorithms.

So far, the discourse on privacy has revolved around the lawfulness of scraping activities, focusing on the scraper’s data protection role and its challenges to comply with GDPR. Indeed, the massive – and often undetected – collection of personal data available on the Internet is facing increased scrutiny by EU data protection authorities.

These authorities are currently calling for adequate legal basis for such processing and for the protection of the data subjects’ rights whose data have been scraped. The Garante is certainly among such authorities: as said in the press release of the Guidance, it is still “waiting to decide – following the outcome of a number of investigations already under way, including the one against OpenAI - on the lawfulness of web scraping of personal data performed on the basis of legitimate interest”.

Yet, with the Guidance, the Garante takes a different approach, by delving into the measures to be taken by website owners and platform operators (for simplicity’s sake, the ‘website owners’), to protect personal data published online from scraping.

In this respect, the Guidance starts from defining scraping as “the massive and indiscriminate collection of data (including personal data) conducted via web crawling techniques and combined with the storage and retention of the data collected by bots for subsequent targeted analysis, processing and use”.

It then focuses on the recommendations for website owners – recommendations that are additional to the technical and organizational measures that website owners, as data controllers, must implement under art. 32 GDPR to protect data from malicious web scraping.

To protect websites against scraping, the Garante suggests: 

• Creating reserved areas (accessible only by logging in with user credentials), to make data no longer publicly available.
• Prohibiting scraping activities in the website’s terms and conditions. The Garante notes that, even if this is an ex-post measure, it may serve in any case as a deterrent.
• Monitoring HTTP requests and webpage traffic, to detect any anomalies in the flows of incoming and outgoing data. Such monitoring can be also coupled with a rate limiting to limit network traffic.
• Implementing technical measures against bots, including: 
1. Inserting CAPTCHAs, to require human intervention and block bots
2. Periodically updating the HTML, to obstruct scraping
3. Embedding certain content within multimedia objects (e.g. images), which are more difficult to scrape
4. Monitoring log files, to detect and block unwanted user agents
5. Including instructions in the robot.txt file, to disallow scraping

As stated by the Garante, these are not one-size-fits-all recommendations and require a case-by-case analysis. For instance, the Garante points out that creating reserved areas may conflict with the principle of data minimization—as the user registration may result in the collection of unnecessary data. Yet, the Garante does not offer further insights on how to solve this apparent paradox, leaving it to controllers to assess which options are the most suitable to protect the personal data processed.

The Guidance represents an important tool for controllers, especially for small and medium enterprises, to reduce their datasets exposure to scraping. This is obviously only one side of the coin: as said above, once the Garante will take position on the current OpenAI proceedings, there will be a clearer position on the dilemmas posed by scraping, including the one about the lawfulness of the processing. 

Stay tuned with more insights on AI and scraping here.