• General_Effort@lemmy.world
      link
      fedilink
      English
      arrow-up
      6
      arrow-down
      4
      ·
      3 months ago

      Shocked? You’d think all the people outraged at having their websites scraped would be delighted. That’s probably the real reason for this.

      • subignition@fedia.io
        link
        fedilink
        arrow-up
        3
        ·
        3 months ago

        It’s not the scraping itself, but the purpose of the scraping, that can be problematic. There are good reasons for public sites to allow scraping.

        • General_Effort@lemmy.world
          link
          fedilink
          English
          arrow-up
          2
          ·
          3 months ago

          I have the distinct impression that a number of people would object to the purpose of re-hosting their content as part of a commercial service, especially one run by Google.

          Anyway, now no one has to worry about Google helping people bypass their robots.txt or IP-blocks or whatever counter-measures they take. And Google doesn’t have to worry about being sued. Next stop: The Wayback Machine.