By Matthew Travers, President, Lead to Conversion.
Direct Answer
AI search engines choose sources by running a live web search, often as several related searches, and citing the pages whose passages answer part of the question. A page can be cited only if that engine’s search crawler can reach it and the page is allowed to show a snippet.
TL;DR
- AI answers cite pages found in a live search, not pages the model remembers from training.
- Each engine uses a named search crawler: Googlebot, OAI-SearchBot and PerplexityBot. Block it, and that engine can’t cite you.
- Training crawlers such as GPTBot and Google-Extended are separate switches, so you can opt out of training and keep your citations.
- Query fan out means you can be cited for related questions you rank for, beyond the one a patient typed.
- Google says optimizing for AI search is still SEO. Its AI features need no llms.txt file, chunking or special schema.
- Check the Search generative AI control in Search Console, live for all sites since August 31, 2026.
- Pages that answer one piece of a question clearly, with first-hand detail, are the ones that get picked.
One liner: Get your pages in front of each engine’s search crawler, then give it answers only your practice can give.
Key Numbers
| Fact | Value | Source |
|---|---|---|
| Search generative AI control rollout to all sites | August 31, 2026 | Search Console Help [6] |
| Default setting for the control | Include | Search Console Help [6] |
| Time for an exclusion to take effect | 1 to 2 days after the control goes live, some content longer | Search Console Help [6] |
| Time for robots.txt changes to reach ChatGPT search | About 24 hours | OpenAI [3] |
| Time for robots.txt changes to reach Perplexity | Up to 24 hours | Perplexity [4] |
| Extra technical requirements for AI Overviews and AI Mode | None beyond Search | Google, updated December 10, 2025 [5] |
| Special schema.org markup needed for Google’s AI features | None | Google, updated July 10, 2026 [1] |
A prospective patient opens ChatGPT and types a simple question: which concierge doctors near me are taking new patients? A few seconds later, the answer names three practices, each with a small link beside it. Your practice isn’t one of them, even after years of work on your website.
So how do AI search engines choose sources? They run a search, read what comes back, and cite the pages they used. The details of that search decide who gets named. Below, you’ll see how retrieval and selection work in Google’s AI features, ChatGPT and Perplexity. You’ll also see what the platforms themselves say matters, and the steps a practice can take to become a source AI can cite.
How Do AI Search Engines Choose Sources?
In short: they search, they read, and they cite what they used.
AI search engines choose sources by running a web search, often as several related searches at once, then reading the pages that come back. They keep the passages that answer part of the question, write a response from them, and link to the pages those passages came from [1][2].
Here’s how that plays out:
- Retrieve. The engine turns your question into one or more searches. Google calls this query fan out, a set of related searches that pull in more results than the original question would [1]. ChatGPT does something similar, rewriting a question into targeted queries it sends to its search partners [2].
- Select. The model reviews the specific information on the pages it retrieved and keeps what helps answer the question [1].
- Cite. The response links to the pages that support what it says. Those links are the AI citations you see beside an answer [1].
Think of a research assistant who can only quote from books already on the library shelf. However well they write, the shelf decides what they can quote. For AI search, the shelf is a search index, and a search crawler decides what makes it onto the shelf.
AI Citations Come From a Live Search, Not the Model’s Memory
In short: a page gets cited only if that engine’s search crawler can reach it.
The links in an AI answer come from a live search of an index that a named search crawler keeps stocked. Training runs on a separate crawler with a separate switch [3][4]. So being in a model’s training data doesn’t make a page citable. And staying out of training doesn’t have to cost you citations.
OpenAI is direct about this. To be eligible for ChatGPT search, a site must allow OAI-SearchBot and let OpenAI’s published crawler IP addresses through its host or CDN [2]. Sites opted out of OAI-SearchBot “will not be shown in ChatGPT search answers,” though they can still appear as navigational links [3]. GPTBot, the crawler tied to model training, is a separate setting [3].
Perplexity says PerplexityBot surfaces and links sites in its answers and isn’t used to crawl content for AI foundation models [4]. Google’s AI Overviews and AI Mode are built on its core Search ranking systems and Search index [1]. Robots.txt rules for Googlebot are how site owners manage that access [5].
| Engine | Crawler that decides if you can be cited | Crawler tied to training | What the platform says about access |
|---|---|---|---|
| Google AI Overviews and AI Mode | Googlebot | Google-Extended, for training and grounding in some of Google’s other systems | A page must be indexed and eligible for a snippet to be a supporting link [5] |
| ChatGPT search | OAI-SearchBot | GPTBot | Sites opted out of OAI-SearchBot are left out of search answers [3] |
| Perplexity | PerplexityBot | PerplexityBot is not used for model training | Allow PerplexityBot to appear in search results [4] |
Picture a practice that asks its web vendor to block “AI bots” to keep patient pages out of model training. A broad rule catches OAI-SearchBot and PerplexityBot along with GPTBot. The practice drops out of ChatGPT and Perplexity answers. A narrower rule, blocking GPTBot and allowing the search crawlers, keeps both the training opt-out and the citations [3][4].
A few more details matter. ChatGPT searches the web on its own when a question would benefit from current information, and responses that use web search may include citations [2]. ChatGPT-User and Perplexity-User visit pages when a person asks a question, so robots.txt rules may not apply to them [3][4]. OpenAI says ChatGPT-User isn’t used to decide what appears in search; OAI-SearchBot is [3]. And robots.txt changes take up to about 24 hours to show in ChatGPT and Perplexity search [3][4].
So start with access: can each engine’s search crawler reach your pages? In our audit of 100 concierge practice websites, none of the robots.txt files we could read blocked OAI-SearchBot or PerplexityBot.
LLM Retrieval: How AI Search Engines Find Candidate Pages
In short: the engine searches first, often several times, and only pages those searches return can be cited.
The search index makes the first cut
Google calls its method retrieval augmented generation, or RAG, also known as grounding. Its AI features rely on Google’s core Search ranking systems to pull relevant, up to date pages from the Search index [1]. The answer is built from what those pages say, so the index decides what the model has to work with. Google says grounding exists to make AI answers more accurate and more current [1], so pages you keep up to date have a better shot.
ChatGPT follows the same principle with different plumbing. OpenAI says ChatGPT search sometimes partners with other search providers, and its help page points to Microsoft’s privacy statement among them [2]. A strong Google ranking helps with Google’s AI features, but it isn’t the only index that matters. Answers vary even inside Google. AI Overviews and AI Mode may use different models and techniques, so the links they show differ [5]. Expect each engine to cite a different mix of pages.
Query fan out turns one question into many searches
Google’s AI Overviews and AI Mode may use query fan out [1]. The model runs several related searches at once to gather more results than the original question would. ChatGPT does something close. It rewrites a question into targeted queries, reviews the first results, and may send more specific follow-up queries [2]. For a “near me” question, it may add the person’s general location to the search [2].
Here’s how one question, “is a concierge doctor worth it?”, might fan out. The searches below are an illustration, not output from any engine.
| Related search the engine might run | Page that could earn the citation |
|---|---|
| concierge medicine membership cost | A membership or pricing page |
| concierge doctor vs primary care physician | A plain comparison article |
| concierge doctors taking new patients near Akron | A location page with current availability |
That’s how a practice gets cited for a question it never ranked for. The citation comes from a related search where one of its pages did show up. Tracking a single keyword undercounts where you can appear.
Don’t respond by building a page for every variation. Google says creating separate content for each fan out query mainly to manipulate rankings or AI answers violates its scaled content abuse policy [1]. A few strong pages that answer the real sub-questions serve patients better and hold up longer.
Why AI Cites Some Sites and Not Others
In short: of the pages retrieved, the cited ones answer part of the question clearly, add something new, and allow their text to be shown.
Retrieval gets a page into the pool. Selection decides whether it’s quoted. Google says its systems review the specific information on retrieved pages [1]. They link to the pages that support the response. OpenAI says ChatGPT ranks results on several factors meant to surface relevant, reliable information [2]. Placement isn’t guaranteed.
The page answers one piece of the question
Each link in an AI answer supports part of what the answer says [1]. A page that answers its piece plainly, under a clear heading, is easy to match to that part. Google’s advice is to organize content in paragraphs and sections with headings that help readers follow it [1].
It says something other pages don’t
Google puts the most weight here. It says unique, useful content will likely do more for your presence in AI search over time than any other tactic it lists [1]. It warns against commodity content based on common knowledge and favors first-hand, expert takes [1].
Here’s the difference for a practice, modeled on Google’s own example:
| Commodity page | Non-commodity page |
|---|---|
| “5 Benefits of Concierge Medicine” | A physician on what members use most in their first year, and what the membership doesn’t cover |
| Restates what other sites already say | Adds detail only the practice has |
| No clear author | Written by a named clinician with credentials listed |
For a practice, first-hand usually means a clinician writing from what they see with patients. That gives an engine, and a patient, a reason to trust the passage.
Trust and snippet settings decide what can be shown
Some pages rule themselves out. Google says nosnippet, data-nosnippet, max-snippet and noindex limit what it shows from a page in Search, AI features included [5]. If these were added to stop copying, they also limit what AI answers can use.
Shortcuts don’t help either. Google says chasing inauthentic mentions isn’t as helpful as it might seem [1]. Its AI features depend on both its quality ranking systems and its spam systems [1].
What Google Says You Do Not Need
In short: Google says its AI features run on the same SEO basics, so special files, chunking and AI-only markup don’t help in its results.
Terms like AEO and GEO come up a lot. Google’s view is that optimizing for generative AI search “is optimizing for the search experience, and thus still SEO” [1]. Its July 2026 guide lists several tactics site owners can skip for Google Search [1].
| Common advice | What Google says | What to do instead |
|---|---|---|
| Add an llms.txt or other AI text file | Google Search doesn’t use them. Keeping one for other services neither helps nor harms you in Google [1] | Make sure crawlers can reach your real pages |
| Break content into small chunks | There’s no requirement to split content into tiny pieces for AI [1] | Write pages at the length your patients need |
| Rewrite pages for every long-tail variation | You don’t need to write a special way for AI, since its systems understand synonyms and meaning [1] | Answer the questions patients actually ask |
| Add special schema for AI | Structured data isn’t required, and there’s no special markup for AI features [1] | Keep standard schema accurate and matched to the page |
Schema still earns its place. Google recommends it as part of an overall SEO strategy because it helps pages qualify for rich results [1]. It also asks that structured data match the visible text on the page [5]. For a practice, that means markup for the providers, services and locations the page itself describes. For the rich results side, see how structured data improves healthcare SEO.
A simple test before paying for an AI-only tactic: if Google says it doesn’t use it, put that budget into crawler access and better pages.
How to Get Cited by AI: Six Steps for Healthcare Practices
In short: open the door for search crawlers, confirm Google’s AI setting, then give each engine pages worth quoting.
These steps follow the order the engines work in, with access first and content second.
1. Let the search crawlers in
Check that robots.txt, your host and any CDN allow Googlebot, OAI-SearchBot and PerplexityBot [3][4][5]. OpenAI also asks sites to let its published search crawler IP addresses through [2]. To stay out of model training, block the training crawler by name instead of blocking every AI bot [3]:
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: PerplexityBot
Allow: /
2. Check Google’s Search generative AI control
Since August 31, 2026, every site has a Search generative AI control in Search Console, under Settings [6]. “Include” is the default. “Exclude” removes your links from AI Overviews and AI Mode and stops your content from grounding those answers [6]. The setting doesn’t affect training, which Google-Extended handles [6]. Properties can inherit the setting from a parent property, so groups with several locations should check each one [6].
3. Make key pages indexable and quotable
A page must be indexed and eligible for a snippet to be a supporting link in Google’s AI features [5]. Look for stray noindex or nosnippet tags on service pages. Keep important details in text rather than images, and link to key pages from elsewhere on your site [5].
4. Cover the related questions patients ask
Map the questions around your core services: cost, comparisons, availability and location. Build one strong page per real question and skip near-duplicates for every phrasing [1]. That’s the heart of generative engine optimization for healthcare.
5. Publish answers only your practice can give
Have clinicians write or review pages on what they see with patients, such as what a membership covers or how a first visit runs. Google says unique, first-hand content will likely matter most over time [1]. Good healthcare content marketing starts there.
6. Keep your Google Business Profile current
Google says Business Profiles can help your services show up in AI responses as well as regular results [1]. Keep hours, services and contact details up to date [5]. The same care sits at the center of local SEO for healthcare.
How to Check Your AI Search Visibility
In short: use Google’s own report for its AI features, then look at real answers in each engine.
Google counts traffic from AI Overviews and AI Mode in Search Console [5]. Its Generative AI performance report shows how people find your content through those features [1]. For ChatGPT and Perplexity, you check the answers themselves.
| Check | Where | What it tells you |
|---|---|---|
| Generative AI performance report | Google Search Console | How your pages perform in Google’s AI features [1] |
| Ask the related questions patients ask | ChatGPT, Perplexity and Google | Which pages get cited, yours or someone else’s |
| Crawler visits | Server or hosting logs | Whether OAI-SearchBot and PerplexityBot reach your pages [3][4] |
| Free AI visibility check | Lead to Conversion’s AI Visibility Tool | How AI engines describe your practice, with a list of fixes |
Keep one caution in mind. Google says no third-party tool has access to its internal ranking or AI systems [1]. Use tools that show you real AI answers, and be wary of any that promise a score from inside Google.
To see where your practice stands today, run a free AI visibility check. It shows how ChatGPT, Perplexity and Google AI describe your practice and gives you a prioritized list of fixes.
Frequently Asked Questions About AI Citations
In short: fast answers on access, files, schema and timing.
Want help putting these steps to work on your practice’s site? You can book a consultation with our AI digital marketing team, and we’ll start with what AI engines see today.
Definition Bank
| Term | Plain-English definition |
|---|---|
| AI citation | A linked source shown beside an AI answer, pointing to a page that supports part of what the answer says |
| Retrieval augmented generation (RAG) | A method where the AI searches an index first, then writes its answer from the pages it found |
| Grounding | Google’s other name for RAG: tying an AI answer to real, retrieved web pages |
| Query fan out | The AI running several related searches at once to gather more results than one question would |
| Search index | The stored copy of the web a search engine searches, built by its crawler |
| Search crawler | The bot that visits pages so an engine can show and cite them in search answers |
| Training crawler | A separate bot that collects content that may be used to train AI models |
| OAI-SearchBot | OpenAI’s search crawler; sites that block it are left out of ChatGPT search answers |
| GPTBot | OpenAI’s training crawler; blocking it does not affect ChatGPT search |
| PerplexityBot | Perplexity’s crawler for surfacing and linking sites; Perplexity says it is not used for model training |
| Google-Extended | Google’s control for limiting AI training and grounding in some of its other systems |
| Search generative AI control | A Search Console setting that includes or excludes a site from AI Overviews, AI Mode and AI features in Discover |
| Snippet controls | Tags such as nosnippet, data-nosnippet and max-snippet that limit what Google can show from a page |
| Non-commodity content | Content with first-hand or expert detail that goes beyond common knowledge |
| llms.txt | A text file some sites add for AI tools; Google Search does not use it |
Entity Cards
Google AI Overviews and AI Mode
| Property | Value |
|---|---|
| Retrieval method | RAG built on Google’s core Search ranking systems and Search index [1] |
| Uses query fan out | Yes, may run related searches across subtopics [1][5] |
| Crawler that decides eligibility | Googlebot [5] |
| Eligibility | Indexed and eligible to show with a snippet [5] |
| Site-level switch | Search generative AI control in Search Console [6] |
| Reporting | Generative AI performance report in Search Console [1] |
ChatGPT search
| Property | Value |
|---|---|
| Retrieval method | Rewrites questions into targeted queries sent to search partners [2] |
| Search crawler | OAI-SearchBot [3] |
| Training crawler | GPTBot, a separate setting [3] |
| User-triggered fetcher | ChatGPT-User, not used to decide search inclusion [3] |
| Eligibility | Allow OAI-SearchBot and OpenAI’s published IP addresses [2] |
| Placement | Not guaranteed [2] |
Perplexity
| Property | Value |
|---|---|
| Search crawler | PerplexityBot [4] |
| Used for model training | No, per Perplexity [4] |
| User-triggered fetcher | Perplexity-User, which generally ignores robots.txt [4] |
| Time for robots.txt changes | Up to 24 hours [4] |
Search generative AI control
| Property | Value |
|---|---|
| Where | Search Console, Settings, Search generative AI [6] |
| Rolled out to all sites | August 31, 2026 [6] |
| Default | Include [6] |
| Effect of Exclude | No links and no grounding in AI Overviews, AI Mode and Discover AI features [6] |
| Affects training | No; Google-Extended covers that [6] |
Sources
- Google Search Central, Optimizing your website for generative AI features on Google Search, updated July 10, 2026: https://developers.google.com/search/docs/fundamentals/ai-optimization-guide
- OpenAI Help Center, Searching the web with ChatGPT: https://help.openai.com/en/articles/9237897-searching-the-web-with-chatgpt
- OpenAI, Overview of OpenAI Crawlers: https://developers.openai.com/api/docs/bots
- Perplexity, Perplexity Crawlers: https://docs.perplexity.ai/guides/bots
- Google Search Central, AI features and your website, updated December 10, 2025: https://developers.google.com/search/docs/appearance/ai-features
- Google Search Console Help, Search generative AI control, rolled out to all sites August 31, 2026: https://support.google.com/webmasters/answer/16908024
