2025-03-24 03:26:34.850469+01 by Dan Lyke 0 comments
I really need to write or install a bot fucker-upper... AI6YR Ben @ai6yr@m.ai6yr.org
List of AI bots to add to robots.txt (although they may not obey -- may need to throw them in the bitbucket and 404 or 444 them). In addition to these, you may have to block specific random browser versions for the most aggressive bots who ignore robots.txt.
https://github.com/ai-robots-t....robots.txt/blob/main/robots.txt
[ related topics: Robotics Artificial Intelligence ]
comments in ascending chronological order (reverse):
We will not edit your comments. However, we may delete your comments, or cause them to be hidden behind another link, if we feel they detract from the conversation. Commercial plugs are fine, if they are relevant to the conversation, and if you don't try to pretend to be a consumer. Annoying endorsements will be deleted if you're lucky, if you're not a whole bunch of people smarter and more articulate than you will ridicule you, and we will leave such ridicule in place.
Flutterby™ is a trademark claimed by
Dan Lyke for the web publications at www.flutterby.com and www.flutterby.net.