Why Your CDN Is Serving Different Content to Googlebot

Published Read 5 min read

On this page

When Google’s copy of a page shows content your visitors no longer see, the first question isn’t which cache is at fault. It is whether Google is still receiving the old version today, or simply hasn’t come back since you changed the page. Search Console’s URL Inspection tool answers those two questions separately, and the answer decides the fix. If Google is still being served the old version, look at the layers between your origin and the crawler: an edge cache holding HTML past your deploy, and optimizers that change how the page’s scripts load. If the live test keeps getting the current version, what remains is getting Google to recrawl, which you can request for priority pages.

Two views in URL Inspection

Google’s help for the URL Inspection tool separates the indexed result from the live test. The indexed status is not a live test: the results come from the most recently indexed version of a page, not the live version on the web. The live test checks the current version of the page as Google would see it. View crawled page shows the HTTP request and response and the returned HTML; View tested page shows the page code and more information for the live test.

Compare both with the page in a private browser window:

Indexed version Live test What it means Next step
Old content Old content Google is still being served the old page Find the layer serving it, then purge
Old content Current content Google's fetch just now got the current page; Google hasn't recrawled Repeat the live test, then request indexing for priority pages
Current content Current content Google has the current page Check what your visitors get; the stale copy may be on their side

The first row is a delivery problem. The second row is at least partly fixed, but one live test is one fetch, and Google’s requests can reach different CDN locations. Repeat the live test before you conclude the delivery is fixed.

Why Google can get a different copy than you do

A cached copy can outlive your deploy. An edge cache serves its stored copy until it expires or is purged. If HTML is cached with a long lifetime, a copy stored before a redesign can keep going out after the redesign ships. On Cloudflare this applies only when a rule makes HTML cacheable: its default cache behavior page says the CDN does not cache HTML or JSON by default.

Googlebot may request from somewhere else. Google’s page on locale-adaptive pages says the default IP addresses of the Googlebot crawler appear to be based in the USA, and that it also crawls from IP addresses outside the USA. Its requests can reach different CDN locations from yours, and a copy that has already refreshed near you can still be stale where Google fetches.

Optimizers can change the page Google renders. Google renders JavaScript, so a feature that rewrites how scripts load can change the rendered page, not only its speed. Cloudflare’s documentation for Rocket Loader describes it as asynchronous loading that handles inline and external scripts while maintaining their order, and says that if you observe JavaScript or jQuery issues, you should disable Rocket Loader and retest. If the live test’s rendered HTML is missing content that the raw HTML has, test with the optimizer off.

Read the cache headers

Request the page yourself and read the cache status your CDN returns. On Cloudflare that is cf-cache-status, alongside the Age header. Cloudflare’s page on cache responses adds a detail that matters after a redesign: with Tiered Cache, Age reflects the object’s age in Cloudflare’s network-wide cache, so a HIT served locally can carry an age inherited from an upper tier. If a HIT’s Age is longer than the time since your deploy, that copy was stored before the deploy.

Your own request shows the state of the location nearest to you. It doesn’t prove what the location serving Googlebot holds, which is why the live test stays the deciding check.

PageSpeed Insights isn’t a crawl check

A good PageSpeed Insights score says nothing about which version Google indexed. Google’s PageSpeed Insights documentation says PSI provides lab data and field data: the lab data comes from Lighthouse analyzing the URL in a simulated environment, and the field data describes real users’ experience. Neither is the response Googlebot fetched and indexed. Use PSI for speed and URL Inspection for content.

Purge, then prove the purge

Cloudflare’s guide to purging the cache calls purging by single file (by URL) the recommended method, and gives the check: request the asset after purging and confirm that cf-cache-status is no longer HIT. Then run the URL Inspection live test again. When it shows the current page, request indexing for your key pages. On other CDNs, use your provider’s purge and cache-status equivalents.

If the live test still shows the old version after a confirmed purge, look beyond the CDN cache. Check the origin, and any layer between the CDN and the origin, for a copy of its own.

Is this cloaking?

Google’s spam policies define cloaking as presenting different content to users and search engines with the intent to manipulate search rankings and mislead users. The definition turns on intent. A stale cache is a configuration fault, and the practical cost doesn’t wait for a label: Google indexes the version it received, so search results can show content you have already replaced.

Put the check in your release process

Any change to cache rules, lifetimes or script optimizers can change what Google receives, and unless your release checks for it, you may not notice until results show it. After such a change, run the live test on one page per important template and compare its HTML with the page in a private browser window. Treat a mismatch the way you would treat a broken page.

Frequently asked questions

Why does the page look right in my browser when Google has the old version?

Your request may reach a different CDN location or a fresher copy than Google’s. The URL Inspection live test shows what Google gets now; the indexed view shows the most recently indexed version.

Does a good PageSpeed Insights score mean Google sees the right content?

No. PSI’s lab data comes from Lighthouse in a simulated environment. It measures performance and doesn’t show the page Googlebot fetched.

Is my site cloaking if the CDN serves Google an old page?

Google’s spam policies define cloaking by intent to manipulate rankings and mislead users. Whatever the classification, fix the delivery: Google indexes what it receives.

Related posts:

  1. Your Mega Menu Is Leaking PageRank to Low-Value Pages A sitewide mega menu and a bloated footer put the same large set of links on every page of your site, so your top categories share every page's navigation with......
  2. How to Do SEO for Affiliate Sites Post-HCU Start by correcting the premise: there's no standalone "helpful content update" left to escape. Google's guide to its ranking systems lists the helpful content system among its retired systems and......
  3. SEO Tool Stack Architecture: Building an Integrated Workflow A working SEO stack follows one central rule: one source of truth per data type. Assemble tools for complementary coverage and clean data flow, not to own every category twice;......
  4. SEO Contractor Management: SOWs, Deliverables, and Quality Gates Contractor outcomes can be decided before the work begins. A statement of work with explicit exclusions and observable acceptance criteria, paired with a tiered quality gate, does much to settle......
  5. Your Chatbot Widget Is Injecting Thousands of Hidden Links A third-party widget, whether live chat, a social embed, a review badge or a support tool, loads its own markup into your pages, and that markup can include links to......
  6. Your Site Got Caught in a Neighborhood Penalty "Neighborhood penalty" isn't a term in Google's current guidance. It doesn't appear in Google's spam policies, its help for the Manual actions report or its guidance on disavowing links. What......
  7. Parameter URLs Are Eating Your Crawl Budget Filter, sort and tracking parameters multiply URLs, and the result is two separate problems: crawling spent on copies, and copies sitting in the index. They need different tools, and the......
  8. SEO and Engineering Collaboration: Communication Protocols for Technical Requests SEO requests may stall when they arrive in a form engineering can't consume. An email saying "we need to fix our canonical tags for SEO" has no problem engineering can......
  9. Your Pricing Page Is Invisible to Search When review aggregators outrank your own pricing page for "[your product] pricing," the query can be asking more than your page answers. Someone searching it may want to know what......
  10. SEO Traffic Forecasting: Building Predictive Models with Historical Data A defensible SEO forecast is a decomposition problem, not a prediction stunt. You separate three things in your history, the underlying trend, the repeating seasonal pattern and the unexplained residual,......
  11. How to Fix a Site Hit by Multiple Algorithm Updates A site that lost ground in several updates can have more than one problem, and a costly mistake is treating them as one and applying one fix. Recovery starts with......
  12. What Is Search Intent and How to Optimize for It Search intent is the goal behind a query: what the person wants when they type or speak it. Much of the SEO industry sorts it into four types, informational, navigational,......
  13. SEO Competitive Response Playbook: Reacting to Competitor Moves Competitive SEO response is triage, not reflex. When a competitor publishes a new hub, ships a technical change or picks up a burst of links, the first action is not......
  14. How to Write Meta Descriptions That Increase Click-Through Rate Start with how Google builds the text under your result. Its documentation on snippets says Google primarily uses the content on the page to determine the snippet automatically, and may......
  15. Why Query Parameters After Hash Symbols Don’t Exist to Google Everything after the # in a URL is a fragment, and the browser doesn't send it to the server. So when a filter lives after the hash, /properties#bedrooms=3 and /properties#city=seattle......

Leave a comment

Your email address will not be published. Required fields are marked *