The rise and fall of robots.txt
www.theverge.com
As unscrupulous AI companies crawl for more and more data, the basic social contract of the web is falling apart.
    • @mark@programming.devEnglish
      2•9 months ago

      This seems to interestingly prove the point made by the person this is in reply to. Breaking laws come with consequences. Not caring about a robots.txt file doesn’t. But maybe it should.

      • Tlaloc_TemporalEnglish
        3•9 months ago

        My angle was more about all rules being social contructs, and said rules being important for the continued operation of society, but that’s a good angle too.

        Lots of laws don’t come with real punishments either, especially if you have money. We can change this too.