Matthew Prince, Cloudflare’s chief executive, told the New York Times that it was “changing the rules of the internet across all of Cloudflare”. The move was made from a concern that AI companies scraping the web and accessing content freely would stop people being incentivised from publishing new writing or other forms of content, he said.

With the new policy, all new websites onboarded to Cloudflare will automatically block known AI bots — even those that don’t respect robots.txt protocols.