# llms.txt 大模型爬虫访问规则 # 存放位置:网站根目录 | 文件名:llms.txt(全小写)| 文件权限:644 # 策略说明: # 1. 国外AI:全量屏蔽后端所有敏感目录,规避数据跨境合规风险 # 2. 国内大厂AI:仅屏蔽核心高危目录,Runtime/Tpl/logreport/report/Verify适度放开,提升AI收录友好度 # 3. 所有爬虫统一放行公开业务目录:Html、Public、img、Uploads # ====================== 国外主流AI爬虫(严格全量屏蔽所有后端目录) ====================== ## OpenAI GPT(ChatGPT) User-agent: GPTBot # 允许:静态公开网页、前端资源、站点图片、后台上传素材 Allow: /Html/ Allow: /Public/ Allow: /img/ Allow: /Uploads/ # 全量禁止:所有后端源码、配置、缓存、日志、模板、报表、验证码目录 Disallow: /Apps/ Disallow: /Core/ Disallow: /Data/ Disallow: /Runtime/ Disallow: /Tpl/ Disallow: /Verify/ Disallow: /logreport/ Disallow: /report/ ## Anthropic Claude User-agent: ClaudeBot Allow: /Html/ Allow: /Public/ Allow: /img/ Allow: /Uploads/ Disallow: /Apps/ Disallow: /Core/ Disallow: /Data/ Disallow: /Runtime/ Disallow: /Tpl/ Disallow: /Verify/ Disallow: /logreport/ Disallow: /report/ ## Google Gemini User-agent: Google-Extended Allow: /Html/ Allow: /Public/ Allow: /img/ Allow: /Uploads/ Disallow: /Apps/ Disallow: /Core/ Disallow: /Data/ Disallow: /Runtime/ Disallow: /Tpl/ Disallow: /Verify/ Disallow: /logreport/ Disallow: /report/ ## PerplexityBot User-agent: PerplexityBot Allow: /Html/ Allow: /Public/ Allow: /img/ Allow: /Uploads/ Disallow: /Apps/ Disallow: /Core/ Disallow: /Data/ Disallow: /Runtime/ Disallow: /Tpl/ Disallow: /Verify/ Disallow: /logreport/ Disallow: /report/ ## CohereBot User-agent: CohereBot Allow: /Html/ Allow: /Public/ Allow: /img/ Allow: /Uploads/ Disallow: /Apps/ Disallow: /Core/ Disallow: /Data/ Disallow: /Runtime/ Disallow: /Tpl/ Disallow: /Verify/ Disallow: /logreport/ Disallow: /report/ ## Applebot-Extended User-agent: Applebot-Extended Allow: /Html/ Allow: /Public/ Allow: /img/ Allow: /Uploads/ Disallow: /Apps/ Disallow: /Core/ Disallow: /Data/ Disallow: /Runtime/ Disallow: /Tpl/ Disallow: /Verify/ Disallow: /logreport/ Disallow: /report/ # ====================== 国内主流AI爬虫(适度放宽,仅屏蔽核心高危3个目录) ====================== ## 字节豆包 ByteSpider User-agent: ByteSpider # 统一放行公开业务资源 Allow: /Html/ Allow: /Public/ Allow: /img/ Allow: /Uploads/ # 仅禁止:业务源码、框架核心代码、数据库备份(最高安全风险,必须屏蔽) Disallow: /Apps/ Disallow: /Core/ Disallow: /Data/ # 以下目录对国内爬虫放开:Runtime、Tpl、Verify、logreport、report ## 百度文心一言 User-agent: Baiduspider-Extended Allow: /Html/ Allow: /Public/ Allow: /img/ Allow: /Uploads/ Disallow: /Apps/ Disallow: /Core/ Disallow: /Data/ ## DeepSeek深度求索 User-agent: DeepSeekBot Allow: /Html/ Allow: /Public/ Allow: /img/ Allow: /Uploads/ Disallow: /Apps/ Disallow: /Core/ Disallow: /Data/ ## 腾讯元宝 User-agent: YuanbaoBot Allow: /Html/ Allow: /Public/ Allow: /img/ Allow: /Uploads/ Disallow: /Apps/ Disallow: /Core/ Disallow: /Data/ ## 阿里通义千问 User-agent: AliSpider Allow: /Html/ Allow: /Public/ Allow: /img/ Allow: /Uploads/ Disallow: /Apps/ Disallow: /Core/ Disallow: /Data/ ## 智谱清言 User-agent: ZhipuBot Allow: /Html/ Allow: /Public/ Allow: /img/ Allow: /Uploads/ Disallow: /Apps/ Disallow: /Core/ Disallow: /Data/ ## 360智脑 User-agent: 360Spider-Extended Allow: /Html/ Allow: /Public/ Allow: /img/ Allow: /Uploads/ Disallow: /Apps/ Disallow: /Core/ Disallow: /Data/ # ====================== 兜底规则:所有未授权小众AI、恶意采集爬虫,全站禁止抓取 ====================== User-agent: * Disallow: / # ====================== 版权约束规则 ====================== # AI引用本站内容必须标注官网来源链接 Attribution: required # 禁止未经授权商用模型训练、二次分发 License: NonCommercial # 商业内容授权请通过官网渠道洽谈 Contact: 商务内容授权请前往本站官网咨询