What is the Relationship Between RAG and AI Search? An Overview from Mechanism to Citation Flow

The Difference Between Viewing AI Search and RAG from the Inside or Outside

RAG and AI search are not separate technologies; the mechanism by which AI search constructs answers is RAG (Retrieval-Augmented Generation) itself.

The confusion over whether they are separate or the same arises not from a lack of understanding by the reader, but because the context of explanation is split into two. On one hand, it is discussed as a technology for building internal systems that search and respond to internal documents. On the other hand, it is discussed as the underlying technology behind ChatGPT Search and Perplexity (Perplexity AI). The same term circulates as both a "creator's story" and a "user's story," making it appear as if they are two different technologies.

What is often sought to be confirmed is the following.

What do ChatGPT Search, Perplexity, and AI Overviews (AI summaries) search for? And does our company's page appear there? The behavior of AI search running multiple searches behind a single question is detailed in the related article "[The Largest Query Fan-Out Survey in Japan] AI Searches Up to 33 Times Behind a Single Question."

It is not necessary to memorize the three stages of RAG. It is important to distinguish whether your company's page is not reaching the retrieval target in the first place or if it was retrieved but not chosen for citation. Different causes require different actions. Rewriting the headings of pages stopped by crawler settings will not change the situation.

RAG (Retrieval-Augmented Generation) is a System for Answering While Referring to External Materials

RAG refers to a configuration where an LLM (Large Language Model) searches for external information before generating an answer and uses that content as material to generate text. There are three stages.

1. Retrieval: Searching for and extracting documents related to the question

2. Augmented: Inserting the extracted documents into the prompt as material for generation

3. Generation: Writing the answer text based on the material

This configuration is necessary due to the nature of LLMs. The knowledge within the model (pre-training) is fixed at the time of learning, so it does not include information released immediately after or internal data not disclosed externally. A relatable example is the case of loading a file yourself and extracting information from it.

Relying solely on pre-training makes it easier to produce answers based on speculation in unknown areas. RAG mitigates this speculation by placing the basis of the answer in external documents, narrowing the room for hallucinations (falsehoods). Since the source document can be identified, it also has the secondary effect of being able to present the source. This concept of "linking the answer to external evidence" is called grounding.

It is easy to confuse this with fine-tuning. Fine-tuning is a means to change the behavior of the model itself through learning, requiring re-learning with each update. On the other hand, RAG changes the content by simply replacing the reference document without changing the model. In short, the former is a method of teaching how to speak, while the latter is a method of providing materials. If you want to first organize the design to prevent misinformation itself, refer to the related article "Risk Management and Practical Avoidance Techniques to Prevent Misinformation in AI Measures."

This is the premise before entering the discussion of AI search. There is no need to delve into RAG as a technology for building internal systems as long as the goal is to have your company site cited.

What Do ChatGPT Search and Perplexity Search For?

Each AI search adopts the RAG structure, but the way they approach search and present citations differs. Based on publicly available explanations and documents from each company, it can be organized as follows.

The internal specifications of each service are largely undisclosed and change over months. There was a slight change recently (early September 2026). When using it for operational decisions, confirm each company's official documents as primary information.

Structurally important is the point that the question text does not necessarily become the search query as is. In implementations that perform QFO, a single question is broken down into a group of sub-queries, each searched individually. In other words, it is questioned whether your company's page matches the multiple questions internally recreated by AI, not the words input by the reader.

The content of the search generally combines vector search and keyword search in a hybrid search. Vector search looks for semantic proximity, allowing for retrieval even if words do not match. On the other hand, keyword search works strongly with expression matching. The retrieved candidates are rearranged by re-ranking, and only those passed to answer generation become materials.

It is important to note that the unit being retrieved is not the entire page. Documents are cut into chunks (document segmentation units), and similarity is measured at the passage level. Even if a page is well-written, if the extracted segment does not make sense on its own, it is weakly treated as a candidate.

The Variables Controllable by a Company Are Completely Opposite in Internal RAG and AI Search RAG

Despite being the same technology, the variables a company can adjust in internal RAG and AI search RAG are almost opposite. Mixing these in discussions leads to the misconception that success experiences with internal RAG can be directly applied.

In summary, what the company controls in internal RAG is the search infrastructure itself. In contrast, in AI search, the search infrastructure side cannot be touched at all. What the company can control is the manuscript passed to the search infrastructure, namely the content of the published page and the access path to it.

This difference changes the nature of the actions. In internal RAG, if accuracy is not achieved, chunk size and re-ranker can be adjusted, but in AI search, there is no adjustment target. Practically, the design axis shifts from the idea of "optimizing the infrastructure" to "placing a manuscript that makes sense no matter how it is cut."

The Four Stages Leading to a Company's Page Being Cited in AI Search

The state of not being cited is not a single phenomenon. Whether it has not been reached, was retrieved but disappeared as a candidate, or remained a candidate but was not selected as a source changes the cause and response. When divided into stages, it looks like this. In other words, it is necessary to see at which stage of RAG the dropout occurs.

Stage 1: Dropping at Index Reach

AI crawlers have different uses. There are three categories: learning crawlers (GPTBot, ClaudeBot, CCBot, etc.), search crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot, etc.), and user operation-based retrieval (ChatGPT-User, etc.). If you refuse them collectively without distinguishing their uses in robots.txt, you will simultaneously close exposure via search while avoiding learning use. This is a typical opportunity loss.

Regarding llms.txt, among major services, only Perplexity has officially expressed support, and no clear statement has been confirmed from other major services. Although the burden of installation itself is small, it is reasonable to see the effective range as limited.

Stage 2: Dropping at Retrieval

Since questions are broken down into sub-queries, optimizing for one targeted keyword will not help if it does not match the decomposed question. As a criterion, list the questions readers have about one theme and see what percentage actually appear as vocabulary in the text.

Simultaneously effective is the unification of notation. If the same service name or concept is written in multiple notations, the connection as an entity is dispersed. Setting notation rules within the site and operating to unify them after the first appearance is a subtle but effective measure at the retrieval stage.

Stage 3: Dropping Without Remaining as a Candidate

This is the reality of "being recognized but not cited." If the extracted segment (chunk) is written in a way that does not make sense without reading the previous paragraph, it becomes weak as a candidate. Specifically, paragraphs starting with "as mentioned above" or "as shown above," and places where the correspondence between headings and text is loose, fall into this category.

Another is substitutability. If the same content can be written on other pages, domains with high authority are prioritized. Actual measured values and operational conditions unique to your company are effective at this stage.

Stage 4: Dropping Just Before Citation

Even if included in the materials for answer generation, it is not necessarily presented as a source. Information with a small contribution to the answer may be used only for its content and not remain as a source. Whether the definition sentence is definitive and whether the time point of the information can be determined are the dividing points at this stage. Pages where only the update date is updated without changing the content do not function well as freshness judgment materials.

Implementation steps and structured data design after confirming the stages are summarized in the related article "Practical Guide to Integrating LLMO and Content Marketing."

Headline Design for Writing in Units That Withstand Citation

Since the only variable a company can control is the "manuscript to be passed," the design focus is narrowed to creating units that make sense no matter where they are cut.

First, complete H2 and H3 in semantic units. Check whether the first paragraph under the heading answers the question of that heading when read alone. If the answer is several paragraphs ahead, the extracted segment does not contain the answer.

Second, isolate definition sentences. Place a single sentence in the form of "〇〇 is 〇〇 that does 〜" without modifiers or preambles. This sentence functions as both a retrieval target and a citation target.

Third, avoid making the paragraph dependent on the previous one with pronouns. If the subject is omitted at the beginning of the paragraph, the subject disappears when read alone. Although it means repeating the same words, consistency of notation is prioritized in this context.

Fourth, replace the aim from "taking first place" to "continuously remaining a candidate." AI search answers fluctuate even with the same question. Creating a state where it continuously enters the candidate in multiple sub-queries, rather than appearing once, results in a stable number of citations. Bundling internal links by topic rather than by page and covering related questions in a comprehensive manner corresponds to this aim.

Lingering Questions About RAG and AI Search

Should AI Crawlers Be Allowed or Rejected?

The basic approach is to separate judgment for learning and search use. If there is a policy to avoid learning use, control only the learning crawlers, and allow search crawlers and user operation-based retrieval, forming a structure that maintains exposure while implementing the policy. Rejecting them collectively excludes all questions at stage 1.

How to Confirm If It Is Being Cited

The most direct method is to ask the same question to major services and record whether your company's domain appears in the source section. Since answers fluctuate, do not judge by a single result, but try multiple times while changing the question text. On the access analysis side, also look at the transition of referrers from AI search and named searches. In situations where the ratio of zero-click searches increases, the display count and click rate diverge, so using only the inflow number as an indicator can misinterpret the actual situation.

Does Building Internal RAG Strengthen AI Search?

It does not directly connect. What is accumulated in internal RAG is design knowledge on the search infrastructure side, and what a company can control in AI search is only the content of the published page. However, the experience of observing "which writing styles are less likely to be picked up" in internal RAG operations can be directly applied to manuscript design assuming reading by chunk.

Conclusion

RAG and AI search are the same mechanism, and the difference lies in the range a company can touch. In internal RAG, the company controls the search infrastructure, but in AI search, only the manuscript to be passed can be controlled.

Instead of lumping together the state of not being cited, dividing it into four stages: index reach, retrieval, candidate retention, and citation, allows for a single action to be determined. If it does not appear in any question, adjust crawler settings; if it does not appear in general questions, adjust vocabulary and notation; if it appears or does not appear, ensure paragraph self-containment; if only the content is used, focus on definition sentences and freshness.

The smallest unit to start with is three. Review AI crawler usage in robots.txt, check if the paragraph immediately below the main page heading answers the question alone, and unify the notation of service names and terms within the site. All can be started within a week, and if the order is followed, what to measure next will also be determined.

If you want to organize the situation where the premise of search inflow itself is changing, refer to the related article "What is SEO? Explaining the Causes of Ineffectiveness and Measures for the AI Search Era in an Easy-to-Understand Manner." To check which stage your company site is currently stopped at, you can check the reachability and citation status of published pages for free with umoren.ai's LLMO diagnosis.

Articles to Connect Understanding of RAG to Improvement of Your Company Site

Understanding the mechanism of AI search is a separate judgment from where to start fixing your company site. After understanding the dropout causes at each stage, proceed to read articles that determine the order of implementation and measurement.

• How Should Owned Media Change in the AI Search Era?: Design method for owned media that has moved away from PV dependence

• What Companies with Few Named Searches Should Do First with LLMO: Priority for companies with few named searches to start with

• Practical Strategy Guide to Becoming "Chosen Information" in Search Results with AI Overview Tools: Judgment criteria for tools visualizing AI Overview citation status

• How Will Inquiry Routes Change with the Spread of AI Search?: Method for organizing KPIs in line with changes in inquiry routes

• Recommended 12 Companies for BtoB AI Search Measures (LLMO/GEO/AIO) Comparison: Judgment criteria when comparing LLMO support companies for BtoB