How Perplexity finds and cites sources: what it documents
How Perplexity finds sources and adds numbered citations, what its Search and Agent APIs reveal about results, and what Perplexity does not document.
By Crawlito · 8 min read
In short
- Perplexity's help center says it uses AI models to interpret a question, searches the internet for sources such as articles, websites and journals, and adds numbered citations that link to the original sources.
- Perplexity says its search index is built by PerplexityBot and by third-party crawlers it partners with, and that a page blocked in robots.txt may still have its domain, headline and a brief factual summary indexed.
- Perplexity's developer Search API returns ranked results with a title, URL, snippet, publication date and last-updated date, and its search_domain_filter accepts up to 20 domains or paths per request.
- The developer APIs describe Perplexity's product for developers, not the consumer app, and Perplexity does not document how either one ranks pages or chooses which to cite.
On this page
- What Perplexity's help center says about answers and citations
- Where Perplexity's index comes from
- What the developer APIs show about the search step
- How search domain filters work
- How to check whether Perplexity's search returns your pages
- What Perplexity does not document
- Frequently asked questions
- Check that Perplexity can read your pages
Perplexity finds sources by running a web search for your question and writing an answer with numbered citations that link to the pages it used. Its help center says it uses AI models to interpret the question, searches the internet for information from sources such as articles, websites and journals, and includes numbered citations that link to the original sources. Its index comes from its own crawler, PerplexityBot, and from third-party crawlers it partners with. What Perplexity does not publish is how it ranks pages or decides which ones to cite.
This post sticks to what Perplexity documents: the help center description of how answers are produced, what the developer APIs expose about search results and filters, and a plain list of what is not documented. It is the Perplexity companion to how ChatGPT search finds sources and how Claude's web search finds sources. Sources were read on 15 November 2026.
What Perplexity's help center says about answers and citations
The help center article How does Perplexity work? describes the consumer product in a few steps:
- It interprets the question. Perplexity says it goes beyond keyword search and uses AI models to understand what you are asking. The model selector is available to Pro subscribers, who can also leave it on "Best" to have Perplexity pick a suitable model for each question.
- It searches the web. It gathers information from what it calls authoritative sources, giving articles, websites and journals as examples.
- It writes an answer with numbered citations. Each answer includes numbered citations that link to the original sources, so readers can check a claim or read further.
The same article describes deeper modes. Pro Search researches across multiple sources for more in-depth answers and is applied automatically when a plan includes it, with a daily limit for free users. Research mode, which Perplexity also calls Deep Research, runs dozens of searches and reads hundreds of sources before writing a report.
That is the full extent of the official description. The article does not say how sources are ranked, how many are read for a normal answer, or what makes one page more likely than another to be cited. Figures you see elsewhere, such as an "average number of citations per answer", come from third-party studies, not from Perplexity.
Where Perplexity's index comes from
Perplexity documents two sources for its index and one limit on it.
- PerplexityBot. Perplexity's crawler documentation says PerplexityBot is designed to surface and link websites in Perplexity's search results, and recommends allowing it in robots.txt if you want to appear there.
- Third-party crawlers. Its help article How does Perplexity follow robots.txt? says Perplexity partners with third-party crawlers to help build its search index, and that it updated its agreements so those providers also respect robots.txt, particularly for news publisher sites. It does not name the providers.
- Blocked pages are not invisible. The same article says PerplexityBot will not index the full or partial text of a site that disallows it, but that for a blocked page Perplexity "may still index the domain, headline, and a brief factual summary."
Pages can also be read at answer time: Perplexity-User fetches a page when a user's question needs it, and Perplexity says it generally ignores robots.txt for those fetches. The robots.txt groups, user-agent strings, IP lists and WAF rules for both agents are covered in PerplexityBot and Perplexity-User, so they are not repeated here.
What the developer APIs show about the search step
Perplexity's developer documentation is the most detailed public description of its search layer. Read it with one caveat in mind: it describes the product Perplexity sells to developers, not the consumer app at perplexity.ai. Perplexity does not say that the app uses the same index, ranking or filters, so treat the API as a view into how Perplexity's search is built, not a specification of what the app does.
The Search API returns ranked results without an answer
The Search API overview says POST https://api.perplexity.ai/search returns "ranked web results as JSON, not a generated answer." Each result has these fields:
| Field | What the docs say |
|---|---|
title | Page title |
url | Page URL |
snippet | Query-relevant text extracted from the page; can be an empty string |
date | Publication date, or null |
last_updated | Date the page was last updated, or null |
The docs call the results ranked but do not explain the ranking. At the time of writing, web search costs $5.00 per 1,000 successful requests.
The Agent API attaches search results to the answer
For generated answers, Perplexity now points developers to the Agent API. Its migration guide says "Sonar Chat Completions support ended on September 27, 2026." Sonar returns sources in a citations array of URLs and a separate search_results array. The guide says synchronous and streaming Sonar requests keep working and are being reformulated as Agent API requests, rolling out gradually by model, while asynchronous Sonar requests are no longer supported.
In the Agent API, the web_search tool adds a search_results item to the response, before the answer text. It holds the queries that were run and the results, each with an id, url, title, snippet, date, last_updated and source. The docs say inline markers such as [1] are prompt-dependent, so a developer has to ask for them, and that each result's id and url are the source of truth for citations.
Two things follow for site owners, without promising anything about the app:
- The search step works on text extracted from your page. Snippets are excerpts taken from the page, so content that is missing from the HTML a crawler receives has nothing to be extracted from.
- Dates are first-class fields. Publication and last-updated dates are returned, and the API offers filters on both. Perplexity does not say how dates affect ranking, but a page with a clear, accurate date gives those fields something true to report.
How search domain filters work
The search domain filter guide explains how developers restrict which sites the search uses. The search_domain_filter parameter takes an array of strings, with these rules:
- Up to 20 domains or URLs per request.
- Allowlist mode lists domains to include; denylist mode prefixes each with
-. You cannot mix the two in one request. - Enter domains without the protocol:
nature.com, nothttps://nature.com. - A root domain matches its subdomains, so
wikipedia.orgcoversen.wikipedia.org. - A top-level domain such as
.govmatches every.govsite. - A path restricts to a section on segment boundaries:
example.com/docsmatches/docs/introbut not/documentation.
{
"query": "robots.txt rules for AI crawlers",
"max_results": 10,
"search_domain_filter": ["developers.google.com", "rfc-editor.org"]
}A denylist looks like ["-pinterest.com", "-quora.com"], and a path exclusion like ["-reddit.com/r/all"]. The Agent API's web_search tool accepts the same search_domain_filter, placed inside its filters object, with the same 20-entry limit.
This matters to you in two ways. Developers building on Perplexity can include or exclude your domain, or a section of it, by name, which is one more reason to keep a stable host and a clear folder structure. And you can use the same filter to test whether Perplexity's search returns your own pages.
How to check whether Perplexity's search returns your pages
With an API key, restrict a search to your own domain and see what comes back:
curl -s https://api.perplexity.ai/search \
-H "Authorization: Bearer $PERPLEXITY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"query": "your product category and city",
"max_results": 10,
"search_domain_filter": ["example.com"]
}' | jq '.results[] | {title, url, date, snippet}'Read the output this way:
- No results for a query your site clearly answers suggests the pages are not in the index the API searches. Check robots.txt, firewall rules and status codes for PerplexityBot first.
- Results with an empty or odd snippet suggest the extracted text is thin. Compare it with the raw HTML of the page: if the main copy only appears after JavaScript runs, move it into the server-rendered HTML.
- A wrong or missing
dateis worth fixing on the page itself. Perplexity does not document where it reads dates from, so the safe change is a visible date on the page and the same date in your structured data. - The wrong URL variant, such as a parameter or
httpversion, points to canonical and redirect problems.
Remember the limit of this test. It shows what the developer Search API returns; Perplexity does not document that the consumer app ranks or cites the same way. A result here does not mean the app will cite you, and an empty result for one phrasing is not proof you are absent. Run several queries. For referral tracking from the app itself, tracking AI traffic in GA4 covers the setup.
What Perplexity does not document
Knowing the gaps keeps you from optimising for guesses:
| Question | Documented? |
|---|---|
| How pages are ranked or chosen for citation | No |
| How many sources a normal answer reads or cites | No |
| Which third-party crawlers feed the index | Not named |
| Whether the consumer app uses the same index and ranking as the Search API | No |
| Whether sitemaps, structured data or llms.txt affect inclusion | No |
| How often PerplexityBot recrawls a page | No |
| A form to submit a site or request inclusion | No |
| How a page that is only partly indexed (domain, headline, summary) is treated in answers | No |
What Perplexity does document is access: allow PerplexityBot in robots.txt and let its published IP ranges through your firewall. Beyond that, the Search API's snippets are text extracted from the page, so main content that is present in the served HTML is the sensible baseline. Submitting a website to ChatGPT, Perplexity and Copilot compares what each platform offers for discovery, and training, search and user-triggered crawlers explains why Perplexity's agents belong in the search group of your robots.txt.
Frequently asked questions
How does Perplexity choose which sources to cite?
Perplexity does not publish its ranking or citation criteria. Its help center says only that it searches the web for authoritative sources and adds numbered citations linking to them.
Can a page blocked in robots.txt still appear in Perplexity?
Perplexity says PerplexityBot will not index the text of a page that disallows it, but it may still index the domain, headline and a brief factual summary. Perplexity-User, which fetches pages for user requests, generally ignores robots.txt.
Does the Perplexity API show what the Perplexity app will cite?
No. The API documentation describes Perplexity's developer product, and Perplexity does not say the consumer app uses the same index or ranking. Use the API as a rough check of whether your pages are searchable, not as a prediction.
Is the Sonar API still available?
Perplexity's docs say Sonar Chat Completions support ended on 27 September 2026 and recommend the Agent API for all new projects. Synchronous and streaming Sonar requests keep working while they are reformulated as Agent API requests, but asynchronous Sonar requests are no longer supported.
Can I ask Perplexity to include my site?
Perplexity does not document a submission form. Its documented advice is to allow PerplexityBot in robots.txt and permit its published IP ranges.
Check that Perplexity can read your pages
The parts of Perplexity's process you control are access and readable HTML. Run the free check on a page you care about to see how PerplexityBot and other AI crawlers are treated by your robots.txt and whether your main content is in the HTML they receive.
Filed under Strategy and audits. Topics: perplexity, ai-search, ai-crawlers, robots-txt