The door at
wikipedia.org
Open door.wikipedia.org has a robots.txt and no rule against any AI crawler. The bots are inside already.
0 of 22 AI crawlers banned.
Trains models
- GPTBotOpenAI · keeps out /w/ /api/ /trap/Walked in
- ClaudeBotAnthropic · keeps out /w/ /api/ /trap/Walked in
- anthropic-aiAnthropic · keeps out /w/ /api/ /trap/Walked in
- Google-ExtendedGoogle · keeps out /w/ /api/ /trap/Walked in
- GoogleOtherGoogle · keeps out /w/ /api/ /trap/Walked in
- CCBotCommon Crawl · keeps out /w/ /api/ /trap/Walked in
- Applebot-ExtendedApple · keeps out /w/ /api/ /trap/Walked in
- BytespiderByteDance · keeps out /w/ /api/ /trap/Walked in
- meta-externalagentMeta · keeps out /w/ /api/ /trap/Walked in
- FacebookBotMeta · keeps out /w/ /api/ /trap/Walked in
- cohere-aiCohere · keeps out /w/ /api/ /trap/Walked in
- DiffbotDiffbot · keeps out /w/ /api/ /trap/Walked in
AI search
- OAI-SearchBotOpenAI · keeps out /w/ /api/ /trap/Walked in
- Claude-SearchBotAnthropic · keeps out /w/ /api/ /trap/Walked in
- AmazonbotAmazon · keeps out /w/ /api/ /trap/Walked in
- PerplexityBotPerplexity · keeps out /w/ /api/ /trap/Walked in
- YouBotYou.com · keeps out /w/ /api/ /trap/Walked in
Fetches for a user
- ChatGPT-UserOpenAI · keeps out /w/ /api/ /trap/Walked in
- Claude-UserAnthropic · keeps out /w/ /api/ /trap/Walked in
- Perplexity-UserPerplexity · keeps out /w/ /api/ /trap/Walked in
- DuckAssistBotDuckDuckGo · keeps out /w/ /api/ /trap/Walked in
- MistralAI-UserMistral · keeps out /w/ /api/ /trap/Walked in
Search engine
- GooglebotGoogle · keeps out /w/ /api/ /trap/Walked in
- BingbotMicrosoft · keeps out /w/ /api/ /trap/Walked in
