User-agent: * Allow: / # AI assistants and answer engines, listed explicitly so the intent is # unambiguous and any future disallow has to be a deliberate edit. # OpenAI: training, user-initiated fetches, ChatGPT search User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: OAI-SearchBot Allow: / # Anthropic User-agent: ClaudeBot Allow: / User-agent: Claude-User Allow: / User-agent: Claude-SearchBot Allow: / # Perplexity User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / # Google AI Overviews / Gemini grounding User-agent: Google-Extended Allow: / # Common Crawl, a corpus many models train on User-agent: CCBot Allow: / # Apple Intelligence User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / # ByteDance, feeding Doubao and TikTok's assistant. Allowed for consistency # with the rest of this list rather than because that audience matters here. # Worth knowing that Bytespider is widely reported to ignore robots.txt, so # this records intent; it does not enforce anything. A real block would have # to be a firewall rule. User-agent: Bytespider Allow: / Sitemap: https://meiseki.co/sitemap.xml