"[a-z0-9\\-_]*((?<!cu|power[ _]|m[ _])bot(?![ _]TAB|[ _]?5[0-9])|crawler|crawl|checker|archiver|transcoder|spider)([^a-z]|$)|RyowlEngine\/(\\d+)|Hatena-Favicon|HuaweiWebCatBot|scaninfo@expanseinc.com|httpx|WhatCMS|datagnionbot|Choosito|ArchiveBox|PageThing.com|MuscatFerret|LinkpadBot|veryhip|Cocolyzebot|DataXu\/\\d|Project-Resonance|HeartRails_Capture\/\\d|Nimbostratus-Bot|parse.ly scraper|WebDataStats|notifyninja.+monitoring|PingAdmin.Ru|MicroAdBot|https:\/\/whatis.contentkingapp.com|BW\/(?:(\\d+[\\.\\d]+))|adbeat|BDCbot|Barkrowler|VelenPublicWebCrawler|SemanticScholarBot|ias-va.*admantx.*service-fetcher|SerendeputyBot|PetalBot|Domains Project|Robozilla|Grammarly|Vercelbot|Seobility|Nutch|GTmetrix|Startpagina-Linkchecker|LCC|SMTBot|oBot|AwarioRssBot|AwarioSmartBot|WikiDo|Bytespider|IDG\/IT|BoardReader Favicon Fetcher|weborama-fetcher|zelist.ro feed parser|tracemyfile|Buck|CATExplorador|(Match|LinkCheck) by Siteimprove.com|woorankreview|eZ Publish Link Validator|Kaspersky Lab CFR link resolver|BrandVerity|Embedly|The Knowledge AI|^Spotify|^sentry|(A6-Indexer|nuhk|TsolCrawler|Yammybot|Openbot|Gulper Web Bot|grub-client|Download Demon|SearchExpress|Microsoft URL Control|borg|altavista|dataminr.com|tweetedtimes.com|TrendsmapResolver|teoma|blitzbot|oegp|furlbot|http%20client|polybot|htdig|mogimogi|larbin|scrubby|searchsight|seekbot|semanticdiscovery|snappy|vortex(?! Build)|zeal|fast-webcrawler|converacrawler|dataparksearch|findlinks|BrowserMob|HttpMonitor|ThumbShotsBot|URL2PNG|ZooShot|GomezA|Google SketchUp|Read%20Later|RackspaceBot|robots|SeopultContentAnalyzer|7Siters|centuryb.o.t9|InterNaetBoten|EasyBib AutoCite|Bidtellect|tomnomnom\/meg)|RSSRadio \\(Push Notification Scanner;support@dorada\\.co\\.uk\\)|Server Density Service Monitoring.*|AdMantX.*admantx\\.com|NetLyzer FastProbe|EmailWolf|Willow Internet Crawler|WebThumbnail|PagePeeker|ichiro|Pompos|charlotte|gigabot|scooter|nrsbot|netresearch|ScoutJet|Speedy Spider|Slurp|lycos|Zao\/|bitlybot|Catchpoint( bot)?|www\\.monitor\\.us|GrapeshotCrawler|Let's Encrypt validation server|s~snapchat-proxy|Site24x7|Pinterest(bot)?\/\\d\\.\\d.*www\\.pinterest\\.com.*|HubPages.*crawlingpolicy|.*Java.*outbrain|Yahoo Ad monitoring.*yahoo-ad-monitoring-SLN24857.*|YottaaMonitor|ZumBot|Zookabot|zgrab|YRSpider|YYSpider|YOURLS v[0-9]|YoudaoBot|Yeti|NaverJapan|Yandex(SpravBot|ScreenshotBot|MobileBot|AccessibilityBot|ForDomain|Vertis|Market|Catalog|Calendar|Sitelinks|AdNet|Pagechecker|Webmaster|Media|Video|Bot|Images|Antivirus|Direct|Blogs|Favicons|ImageResizer|Verticals|News(links)?|Metrika|\\.Gazeta Bot)|YaDirectFetcher|YandexTurbo|YandexTracker|YandexSearchShop|YandexRCA|YandexPartner|YandexOntoDBAPI|YandexOntoDB|YandexMobileScreenShotBot|Y!J-BRW|YahooCacheSystem|Yahoo Link Preview|Yahoo:LinkExpander:Slingstone|Yahoo! Slurp|Yahoo!-AdCrawler|yacybot|XenForo|Wotbox|WordPress|websitepulse[+ ]checker|WebbCrawler|WeSEE(:Search)?|PTST\/|Wappalyzer|W3C_Unicorn|W3C-mobileOK|W3C_Validator|Validator.nu|W3C-checklink|W3C_I18n-Checker|Jigsaw|VSMCrawler|vkShare; |Vagabondo|URLAppendBot|UptimeRobot|Uptimebot|via secureurl\\.fwdcdn\\.com|UniversalFeedParser|Twitterbot|Twingly Recon|TweetmemeBot|TweetedTimes Bot|TurnitinBot|trendictionbot|theoldreader.com|Tiny Tiny RSS|TinEye-bot|TLSProbe|TelegramBot|TarmotGezgin|SurveyBot|SputnikImageBot|SputnikFaviconBot|SputnikBot|Spinn3r|Sparkler\/[0-9]|Superfeedr bot|StatusCake|SSL Labs|sqlmap\/|Sprinklr|Sosospider|Sosoimagespider|(Sogou (web|inst|Pic) spider)|New-Sogou-Spider|Slackbot|Slack-ImgProxy|sixy.ch|SiteSucker|compatible; (?:SISTRIX )?Optimizer|SISTRIX Crawler|SimplePie|SilverReader|ShopWiki|shopify-partner-homepage-scraper|SeznamBot|SklikBot|Seznam screenshot-generator|SkypeUriPreview|seoscanners\\.net|SEOkicks-Robot|SEOENG(World)?Bot|SensikaBot|SemrushBot|ScreenerBot|Screaming Frog SEO Spider|Scrapy|SafeDNSBot|ROI Hunter|rogerbot|Riddler|redditbot|Rainmeter|Qwantify|QuerySeekerSpider|RamblerMail|Quora-Bot|Quora Link Preview|Pingdom(?:\\.com|TMS)|psbot(-page)?|PRTG Network Monitor|PritTorrent|PocketParser|phpservermon|phantomas\/|PaperLiBot|OrangeBot|VoilaBot|OpenWebSpider|spbot|OpenindexSpider|omgili(?:bot)?|Octopus [0-9]|Nuzzel|Nmap Scripting Engine|nlcrawler|NewsGatorOnline|NewsBlur .*(Fetcher|Finder)|Netvibes|netEstate NE Crawler|Netcraft( Web Server Survey| SSL Server Survey|SurveyAgent)|nbertaupete95\\(at\\)gmail.com|check_http\/v|NalezenCzBot|munin|MojeekBot|Mnogosearch|MJ12bot|MixrankBot|MetaInspector|MetaJobBot|meanpathbot|Mastodon\/|masscan|MagpieRSS|magpie-crawler|Mail\\.RU(_Bot)?|ltx71|LinkedInBot|linkdexbot(-mobile)?|linkdex\\.com|([A-z0-9]*)-Lighthouse|larbin|kouio|IP-Guide\\.com|ips-agent|iisbot|inoreader.com|ICC-Crawler|HTTPMon|HubSpot |heritrix|AdsBot-Google(-Mobile)?|Adwords-(DisplayAds|Express|Instant)|Google Web Preview|Google[ -]Publisher[ -]Plugin|Google-(Ads-Qualify|Adwords|AMPHTML|Assess|HotelAdsVerifier|Read-Aloud|Shopping-Quality|Site-Verification|speakr|Test|Youtube-Links)|(APIs|DuplexWeb|Feedfetcher|Mediapartners)-Google|Googlebot(-Mobile|-Image|-Video|-News)?|GoogleProducer|Google.*\/\\+\/web\/snippet|ShopAlike|Heurekabot-Feed|Seznam-Zbozi-robot|SeznamEmailProxy|via ggpht\\.com GoogleImageProxy|GoogleStackdriverMonitoring|Google-Structured-Data-Testing-Tool|Google-Cloud-Scheduler|google_partner_monitoring|Google Page Speed Insights|Google Search Console|Google Favicon|ichiro\/mobile goo|Gluten Free Crawler|GigablastOpenSource|Genieo|FreshRSS|Findxbot|FlipboardProxy|FlipboardRSS|Fever\/[0-9]|Feedspot|(Meta)?Feedly(Bot|App)?|Feed Wrangler|FeedBurner|Feedbin|facebookexternalhit|facebookplatform|facebookexternalua|Ezooms|ExactSeek Crawler|Exabot(-Thumbnails|-Images)?|ExaleadCloudview|evc-batch|EMail Exractor|eCairn-Grabber|EasouSpider|DuckDuck(?:Go-Favicons-)?Bot|DotBot|Domain Re-Animator Bot|support@domainreanimator.com|discobot(-news)?|Dazoobot|Daum(oa)?[ \/][0-9]|Dataprovider|Datanyze|Datadog Agent|CSS Certificate Spider|CommaFeed|collectd|coccoc.com|CloudFlare-AlwaysOnline|CloudflareDiagnostics|Cloudflare-AMP|Cliqzbot|CCBot|CareerBot|(?<!HTC)[ _]Butterfly\/|BUbiNG|Browsershots|BountiiBot|BoardReader Blog Indexer|Blogtrottr|Bloglovin|BLEXBot(Test)?|Blekkobot|MSNBot|msrbot|bingbot|BingPreview|msnbot-(UDiscovery|NewsBlogs)|adidxbot|BazQux|baiduspider(-image)?|baidu Transcoder|baidu.*spider|BacklinkCrawler|Backlink-Check\\.de|Ask Jeeves\/Teoma|archive\\.org_bot|special_archiver|Curious George|Castro 2, Episode Duration Lookup|AspiegelBot|Arachni|Applebot|ApacheBench|AmorankSpider|Amazon[ -]Route ?53[ -]Health[ -]Check[ -]Service|alexa site audit|ia_archiver|alexabot|verifybot|AhrefsBot|AddThis\\.com|AcoonBot|Aboundex|360Spider(-Image|-Video)?"