The rise and fall of robots.txt
www.theverge.com
As unscrupulous AI companies crawl for more and more data, the basic social contract of the web is falling apart.
  • @mark@programming.devEnglish
    2•9 months ago

    This seems to interestingly prove the point made by the person this is in reply to. Breaking laws come with consequences. Not caring about a robots.txt file doesn’t. But maybe it should.

    • Tlaloc_TemporalEnglish
      3•9 months ago

      My angle was more about all rules being social contructs, and said rules being important for the continued operation of society, but that’s a good angle too.

      Lots of laws don’t come with real punishments either, especially if you have money. We can change this too.