I just noticed a new way in which AI scraper bots work. A page is requested from a bot at a given IP address, then an instant later a swarm of other bots from a plethora of different addresses download every link that's in the page: images, media, connected pages, etc... then there seems to be a timeout and the cycle restarts with the linked pages.

Why do I know these are specifically AI scraper bots? Because as you know they've been written by the most brilliant minds in the tech sector, and are super-optimized and not wasteful at all. Which is why they re-scrape the same article posted 2 years ago 130 times a day, every day of the week, for months on end.