The rise and fall of robots.txt
www.theverge.com
As unscrupulous AI companies crawl for more and more data, the basic social contract of the web is falling apart.
  • @Ascend910@lemmy.mlEnglish
    10•9 months ago

    This is a very interesting read. It is very rarely people on the internet agree to follow 1 thing without being forced