Bots
A bot is a software application programmed to do certain tasks.
Bots can be used for good (chatbots, search engine crawlers) or for evil (inventory hoarding, credential stuffing).
Cloudflare maintains an internal list of verified bots that are associated with search engine optimization (SEO), website monitoring, and more.
You can use this list to prevent any bot protection measures from impacting otherwise helpful bots, such as search crawlers.
For a partial list of verified bots, refer to Cloudflare Radar ↗.
You can block artificial intelligence (AI) bots, crawlers, and scrapers from scraping your website content and training large language models (LLM) to recreate it without your permission. When you enable this feature via a pre-configured managed rule, Cloudflare can detect and block verified AI bots that respect robots.txt and crawl rate, and do not hide their behavior from your website. The rule has also been expanded to include more signatures of AI bots that do not follow the rules.
This feature is available on all Cloudflare plans.