More and more people ask an AI assistant instead of scanning a list of links. The answer they get often names its sources. This guide explains what you control so your site can be one of those sources, and what nobody can guarantee.
Short answer
AI answers in Google Search, ChatGPT, Claude and Perplexity rely on sites their search bots can read. Let those bots in through robots.txt and your hosting, stay in Google’s index with snippets allowed, write a clear answer as text at the top of the page, and track citations in Search Console and Bing Webmaster Tools. Google says AI features need no special files and no special optimization.
Three kinds of bots
Companies that build AI tools send different bots for different jobs. It pays to tell them apart, because each one is controlled separately.
- Search bots. They read sites so they can show and cite them in answers: Googlebot, Bingbot, OAI-SearchBot (ChatGPT search), Claude-SearchBot and PerplexityBot.
- Training bots. They collect text to train models: GPTBot (OpenAI) and ClaudeBot (Anthropic). At Google the Google-Extended token plays this role. It covers Gemini models and does not affect Google Search.
- User-requested fetches. ChatGPT-User, Claude-User and Perplexity-User open a page when a user asks for it in a conversation. This is not automatic crawling, so robots.txt rules do not always apply to them.
1. Let search bots in
If a search bot is blocked, your site cannot appear in that tool. OpenAI says sites that opt out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links. Anthropic says blocking Claude-SearchBot may reduce your visibility in its search results.
Check two things. First robots.txt: does any rule, old or added out of caution, block these bots? Then your hosting and CDN: bot protection sometimes serves a challenge page instead of your content. Google’s guidance specifically says crawling must be allowed at the CDN and hosting level too.
2. Separate search from training
Whether your text may be used to train models is a business decision, not a technical one. At OpenAI, Anthropic and Google, search and training are set independently. If you want citations but not training, block only the training bots in robots.txt, for example User-agent: GPTBot with Disallow: /, and the same for ClaudeBot and Google-Extended. Keep the search bots allowed. OpenAI and Perplexity say their systems can take up to 24 hours to pick up the change.
3. Stay indexed and allow snippets
Google has no additional requirements for AI Overviews and AI Mode. A page must be indexed and eligible to appear in Search with a snippet. Pages you want cited must not carry noindex. The nosnippet, data-nosnippet and max-snippet controls also limit how a page appears in AI features, so use them only where you really do not want the text shown.
4. Write the answer as text, at the top
Google recommends making important content available as text. Text inside images, or content that only appears after a click, is harder to read. It helps when a page answers right away: one question in the heading, a short and complete answer in the first paragraph, then the details.
Be specific. Say who the service is for, what the process looks like and what the client gets. A sentence that still makes sense when lifted out of the page is easier to cite.
5. Structured data that matches the text
There is no special schema.org markup for AI features. Google asks that structured data match the visible text on the page. Data about your organisation, services, articles and page path helps search engines understand who is behind the content, but only if it agrees with what visitors see.
6. Link your pages together
Google’s recommendations include internal links and a good page experience. Every article should link to the service it relates to and to related articles, and every service page to the articles that answer questions about it. Bots find pages more easily, and readers keep going. If you have a Google Business Profile, keep its details current.
7. Track citations
Google reports visits from AI Overviews and AI Mode in Search Console, in the Performance report under the “Web” search type, together with regular search. Since February 2026, Bing Webmaster Tools has an AI Performance report, in public preview. It shows how often your pages are cited in Microsoft Copilot and AI summaries in Bing, which pages and for which queries. Your server access log shows which bots actually reach the site and which status code they get.
What you do not need
- Special AI files. Google says you do not need new machine-readable files, AI text files or markup to appear in AI features.
- AI-only optimisation. The basics are the same as for search: an accessible page, clear text, accurate data.
- A promise of citations. Nobody decides what an AI answer will pick. You can remove the obstacles and write the best answer, but a citation cannot be guaranteed.
How we did it on this site
When we prepared this site, we went through the same list. We checked that search bots get the full content, that robots.txt closes off only the admin area, that structured data describes the organisation, services, articles and page path in line with the visible text, and that articles open with a short answer. The training policy is the owner’s decision and is set separately from search. We can run the same check on your site as part of our Digital marketing service.
In short
AI visibility rests on the same basics as search: the search bot has to reach the page, the page has to be indexed, and the answer has to be written clearly as text. Training is set separately. Track citations in Search Console and Bing Webmaster Tools. If you want us to check your site, get in touch.
