U Albany Archive's llms.txt file: https://archives.albany.edu/llms.txt
How many GLAM organizations are doing something like this? Do you have any way of telling if it's effective? I wonder if would be worth including an "easter egg" type phrase in there you could look for in AI-generated requests.
Ironically, I kept trying to access this llms.txt file but it wouldn't load, then I realized I was on our VPN. VPNs are apparently a red flag now which is...not great for internet privacy writ large.
@phette23 I've seen discussion of it in the context of ai harvesters ddosing repositories. My instinct is that if your aim is to provide data to the bots more thoroughly it might be effective. But if your aim is to beg the bots to stop making the same requests over and over and taking your server down, I think they'll honour it exactly as well as they currently honour robots.txt files.