At a glance
| robots.txt name | OAI-SearchBot |
|---|---|
| Company | OpenAI |
| Kind | AI search. Finds pages for ChatGPT search answers |
| Visits sites | Yes |
| User agent | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot |
| Official page | platform.openai.com |
OAI-SearchBot visits sites to find pages for ChatGPT search answers. A search crawler finds and reads pages so the assistant can quote, cite and link to them in answers that search the web. Blocking it usually means the assistant cannot cite or link to the site. This matters to a business because if the crawler is blocked, ChatGPT cannot reference the site in its answers, which may reduce visibility in AI search results.
Whether to block or allow OAI-SearchBot depends on your goals. If you want ChatGPT to be able to cite and link to your site, allow it. If you prefer not to be included in AI search answers, you can block it. Remember that robots.txt is a request, not a lock, and reputable crawlers follow it. A firewall or bot-protection service can block a crawler even when robots.txt allows it.
robots.txt lines
User-agent: OAI-SearchBot Disallow: /
User-agent: OAI-SearchBot Allow: /
Questions
What is OAI-SearchBot?
OAI-SearchBot is OpenAI's AI search crawler. It finds and reads pages so ChatGPT can quote, cite and link to them in answers that search the web.
Should I block OAI-SearchBot?
It depends on whether you want ChatGPT to be able to cite and link to your site. Blocking it usually means the assistant cannot cite or link to the site. If you want that visibility, allow it.
How do I block OAI-SearchBot in robots.txt?
To block it for the whole site, add these two lines to robots.txt: User-agent: OAI-SearchBot Disallow: /
Does blocking OAI-SearchBot affect AI search or Google?
Blocking OAI-SearchBot affects whether ChatGPT can cite and link to your site in its search answers. It does not affect Google search. Being readable by AI does not mean being recommended.