AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |
Back to Blog
Dotbot disallow4/15/2023 ![]() SetEnvIfNoCase User-Agent "^FrontPage" bad_bot SetEnvIfNoCase User-Agent "^flunky" bad_bot SetEnvIfNoCase User-Agent "^Foobot" bad_bot SetEnvIfNoCase User-Agent "^EyeNetIE" bad_bot SetEnvIfNoCase User-Agent "Extractor" bad_bot SetEnvIfNoCase User-Agent "^Express\ WebPictures" bad_bot SetEnvIfNoCase User-Agent "^Exabot" bad_bot SetEnvIfNoCase User-Agent "^EroCrawler" bad_bot SetEnvIfNoCase User-Agent "^EmailWolf" bad_bot SetEnvIfNoCase User-Agent "^EmailSiphon" bad_bot SetEnvIfNoCase User-Agent "^EmailCollector" bad_bot SetEnvIfNoCase User-Agent "^EirGrabber" bad_bot SetEnvIfNoCase User-Agent "^ebingbong" bad_bot SetEnvIfNoCase User-Agent "^EasyDL" bad_bot SetEnvIfNoCase User-Agent "^eCatch" bad_bot SetEnvIfNoCase User-Agent "^Drip" bad_bot SetEnvIfNoCase User-Agent "^dragonfly" bad_bot SetEnvIfNoCase User-Agent "^Download\ Wonder" bad_bot SetEnvIfNoCase User-Agent "^Download\ Devil" bad_bot SetEnvIfNoCase User-Agent "^Download\ Demon" bad_bot SetEnvIfNoCase User-Agent "^DittoSpyder" bad_bot SetEnvIfNoCase User-Agent "^DIIbot" bad_bot SetEnvIfNoCase User-Agent "^DISCo" bad_bot SetEnvIfNoCase User-Agent "^AIBOT" bad_bot SetEnvIfNoCase User-Agent "^Custo" bad_bot SetEnvIfNoCase User-Agent "^Crescent" bad_bot SetEnvIfNoCase User-Agent "^cosmos" bad_bot SetEnvIfNoCase User-Agent "^CopyRightCheck" bad_bot SetEnvIfNoCase User-Agent "Copier" bad_bot SetEnvIfNoCase User-Agent "Collector" bad_bot SetEnvIfNoCase User-Agent "^ChinaClaw" bad_bot SetEnvIfNoCase User-Agent "^CherryPicker" bad_bot SetEnvIfNoCase User-Agent "^CheeseBot" bad_bot SetEnvIfNoCase User-Agent "^Cegbfeieh" bad_bot SetEnvIfNoCase User-Agent "^BunnySlippers" bad_bot SetEnvIfNoCase User-Agent "^Bullseye" bad_bot SetEnvIfNoCase User-Agent "^BuiltBotTough" bad_bot SetEnvIfNoCase User-Agent "Buddy" bad_bot SetEnvIfNoCase User-Agent "^BotALot" bad_bot SetEnvIfNoCase User-Agent "^BlowFish" bad_bot SetEnvIfNoCase User-Agent "^BlackWidow" bad_bot SetEnvIfNoCase User-Agent "^Black.Hole" bad_bot SetEnvIfNoCase User-Agent "^Bigfoot" bad_bot SetEnvIfNoCase User-Agent "^BatchFTP" bad_bot SetEnvIfNoCase User-Agent "Bandit" bad_bot SetEnvIfNoCase User-Agent "^BackWeb" bad_bot SetEnvIfNoCase User-Agent "^BackDoorBot" bad_bot SetEnvIfNoCase User-Agent "^attach" bad_bot SetEnvIfNoCase User-Agent "^asterias" bad_bot SetEnvIfNoCase User-Agent "^Alexibot" bad_bot SetEnvIfNoCase User-Agent "^Cogentbot" bad_bot SetEnvIfNoCase User-Agent "^Java" bad_bot SetEnvIfNoCase User-Agent "360Spider" bad_bot SetEnvIfNoCase User-Agent "80legs" bad_bot ![]() SetEnvIfNoCase User-Agent "Aboundex" bad_bot SetEnvIfNoCase User-Agent "^AspiegelBot" bad_bot RewriteCond % ^.(Mb2345Browser|AspiegelBot|LieBaoFast|MicroMessenger|zh-CN|Kinza|Mb2345Browser).$ ĭisallow: /privacy-policy-cookie-restriction-mode/Īm I missing something or writing something incorrectly? Ï'm kinda at a loss here. ![]() I've been trying to block the bot in both in the sites htaccess with no sucess. I have issues with AspiegelBot crawling one of the sites on a server, this results in a lot of cores getting used up.
0 Comments
Read More
Leave a Reply. |