What we joined, and how
Two tables, both from production. The first is our own tracked questions for promvia.app, asked through OpenAI's API with web search and through Perplexity's API. There are four of them and every one names us: "Is Promvia any good?", "Promvia reviews", "Promvia vs Otterly" and "What is Promvia by Atlaven?". Between 26 August and 5 October 2026, 63 ChatGPT answers and 104 Perplexity answers cited at least one promvia.app URL. The second table is every request to promvia.app whose user agent claimed to be ChatGPT-User, OAI-SearchBot, GPTBot, PerplexityBot or Perplexity-User, logged from 25 August.
- Verified means the request came from an IP address inside the range the crawler's operator publishes. A user agent alone is a claim anyone can make.
- URLs are compared by path, with query strings and trailing slashes removed, so the ?utm_source=openai that ChatGPT appends does not break a match.
- The check time is when we stored the answer, after the provider returned it. "Before the answer" means the 10 minutes up to that moment.
- We mark two sources of requests as our own: Promvia's weekly crawler-access check, which requests our homepage once with each crawler user agent every Monday at 13:00 UTC, and one address we used for manual tests.
Fetches at answer time looked like background noise
Seven of the 63 ChatGPT answers had a verified ChatGPT-User request in the 10 minutes before them, and only one of those requests was for a page the answer then cited. Seven in 63 sounds like something until you look at the rest of the log. ChatGPT-User made 593 verified requests in the period, 561 of them for our homepage, about 14 a day around the clock. At that rate, any 10 minutes picked at random contains at least one such request 9.6% of the time. Our 7 of 63 is 11.1%, with an interval of 5.5–21.2%. It is the same number.
Perplexity was quieter still. Two of 104 answers had a verified PerplexityBot request in the 10 minutes before, and neither was for a cited page. Perplexity-User, the agent Perplexity uses for fetches a person asks for, never appeared in the log at all.
There is a trap here that nearly made it into the numbers. Eight Perplexity answers did have a "PerplexityBot" request just before them. All eight were for our homepage, and seven of those answers cited it. Every one of the eight failed IP verification. They were our own weekly crawler-access check, which runs at the same time on Monday as our weekly citation checks. Without verification, that is seven perfect answer-time fetches of the cited page, and all of them were us.
The index crawl came first
The pattern that did hold is older and slower. For each URL an answer cited, we looked for any verified request from the engine's search crawler for that same path before the answer.
- ChatGPT: OAI-SearchBot had fetched the page first in 116 of 129 cases (89.9%, interval 83.5–94.0%). All 13 misses were on 27–29 August, within four days of our log starting, so some of those pages may have been crawled before we were recording.
- Perplexity: PerplexityBot had fetched the page first in 174 of 174 cases. The answers kept returning to the same five pages: one comparison guide 60 times, the help page 43, the homepage 33, a tools roundup 31 and the imprint 7.
For these API answers, everything in our log is consistent with the engines answering from pages they had collected earlier. The request that lined up with a citation was the search crawler's earlier visit, not a fetch while the question was being answered.
Most of the fake bots in our log were us
Raw failure rates for IP verification in the same period look alarming: GPTBot 16 of 68 requests (24%), PerplexityBot 9 of 34 (26%), ChatGPT-User 17 of 610 (3%), OAI-SearchBot 7 of 390 (2%). Then we took out our own requests: the weekly access check and our manual tests.
- GPTBot: all 16 failures were ours. The rest: 0 of 52.
- PerplexityBot: all 9 were ours. The rest: 0 of 25.
- OAI-SearchBot: all 7 were ours. The rest: 0 of 383.
- ChatGPT-User: 7 were ours and 10 are unexplained, 10 of 603 (1.7%). Those 10 are the only requests in the period that we cannot account for.
If you run an uptime monitor, a crawler-access checker or an SEO tool that sends bot user agents, it is in your log too. Subtract it before you write about impostors.
How long an edit waits for the crawler
OAI-SearchBot made 383 verified requests for 89 different pages. When it fetched a page again, the median gap was 24.0 hours. That median is set by about a dozen pages it visited roughly daily; 49 of the 89 pages were fetched once in 41 days. PerplexityBot made 25 requests for 11 pages, fetched only 3 of them more than once, and the median gap for those was 40.7 hours.
A known change makes this concrete. On 4 October at 18:05 UTC we shipped a release that changed the text of four pages: the homepage, /pricing, /methodology and /help. OAI-SearchBot fetched the homepage and /methodology again 36 hours later. As of 7 October at 09:00 UTC it had not fetched /pricing (last visit 1 October) or /help (last visit 30 September). PerplexityBot had fetched none of the four; its last verified request for any page on our site was on 26 September. Perplexity still cited our homepage and /help on 5 October; PerplexityBot had last fetched them on 20 and 12 September.
What to do with this
- Verify crawler IP addresses before you count anything, and remove your own monitoring before you call a request fake.
- Do not read live-fetch agents such as ChatGPT-User as a sign that an answer just used your page. On our site a steady stream of them hit the homepage all day, and the ones next to our answers were no more frequent than anywhere else.
- Watch the search crawlers instead. For every page an answer cited, OAI-SearchBot or PerplexityBot had almost always been there first.
- After you change a page, check when each search crawler last fetched it before you judge whether the answers reflect the change. If it has not been back since the edit, that crawler has not seen the new version.
- If you send Promvia your CDN logs, the AI crawlers page shows a "Later cited" column beside each crawled page: the same join, run on your own site, and labelled as an association between the crawl and the citation.
For the crawler tokens themselves, see how to audit your robots.txt for AI crawlers. For how visits from AI answers appear in analytics, see why AI traffic shows as direct.
Frequently asked questions
Does ChatGPT fetch my page when it answers a question?
For answers from OpenAI's API with web search, almost never in our log: 1 of 63 answers that cited us had a verified ChatGPT-User request for a cited page in the 10 minutes before. In 116 of 129 cases OAI-SearchBot had crawled the cited page earlier. We did not measure ChatGPT in the consumer app, which may behave differently.
How can I tell a real GPTBot or PerplexityBot from a fake one?
Check the request's IP address against the ranges OpenAI and Perplexity publish for each crawler; the user agent alone proves nothing. In our log, every GPTBot, PerplexityBot and OAI-SearchBot request that failed that check came from our own monitoring.
How long until ChatGPT or Perplexity sees a page I edited?
Until their search crawler fetches it again, and that varies by page. OAI-SearchBot came back to pages it revisits after a median of 24 hours, but fetched 49 of our 89 pages only once in 41 days. After our 4 October edit it returned to two of the four changed pages within 36 hours and not yet to the other two. PerplexityBot had not returned to any of them.
Can I check these numbers?
Yes. We publish the rows, one line per cited URL and one per crawler request with IP addresses removed, and a short script that recomputes every figure in this guide from them.