Glossary

GPTBot

GPTBot is OpenAI's web crawler, used to gather content for training and retrieval. Full definition and related terms.

Diploria

GPTBot is OpenAI's web crawler. It fetches publicly available web content that can be used to train and improve OpenAI's models, and OpenAI also operates related agents such as OAI-SearchBot and ChatGPT-User for search and live browsing. Site owners can allow or disallow it in robots.txt.

For brands, the key fact is that GPTBot, like most AI crawlers, does not execute JavaScript. If your pages render their content only after JavaScript runs in the browser, GPTBot may see an empty shell, which is one of the most common reasons a site is invisible to AI. Serving server-rendered HTML and allowing the bot in robots.txt are the two prerequisites.

Learn more

Frequently asked questions

Know exactly where
AI mentions your brand

Track ChatGPT, Gemini, Perplexity, Claude and 11 more AI engines - weekly. See how your competitors rank, spot gaps, and fix them fast.