Google Keeps Crawling URLs You Deleted Years Ago

Published Read 7 min read

On this page

Deleting a page does not remove its URL from Google’s memory. Google maintains a persistent per-domain database of URLs it has ever discovered, fed by your sitemaps, your internal links, external backlinks from other sites, and its own crawl history, and that database keeps a deleted URL in the crawl queue long after the page is gone. The dominant reason old deletes keep getting recrawled is external backlinks: every time Google recrawls a page that still links to your dead URL, it re-finds the reference and re-queues it, so the queue refills for as long as those links exist.

The good news is that 404s are not a ranking penalty. The cost is missed link equity, and the fix is a triage: 301-redirect the deleted URLs that have real backlinks, fix internal links pointing at dead pages, and return 410 for content that is permanently gone so Google gives up on it faster.

This is the discovery-persistence case: pages already deleted, often long ago, and a “Not found (404)” report filling with URLs you do not recognize. It is not a CMS migration where redirects fail to fire, and it is not a deliberate date-segment removal.

Why deleted URLs keep coming back

Google’s URL memory persists by design, because pages return and the web is full of temporary outages, so it does not discard a known URL just because it 404s once. External backlinks are the engine that keeps refilling the queue. A site that linked to your old article still links to it; each time Google crawls that linking page, it re-discovers your dead URL and schedules another crawl. You do not control those external pages, so you cannot stop the rediscovery at the source, which is why URLs you removed three years ago still show up.

Two smaller sources add to it. Internal links left pointing at deleted pages keep feeding Googlebot the dead URL from inside your own site, which you can fix. And tracking-parameter variants multiply the count: every distinct query string, every ?utm_source= and ?ref= permutation that was ever linked, is a separate URL in Google’s eyes, so a single deleted page can appear as many dead URLs.

Reassurance and the real cost

404s do not harm your rankings. Google has stated repeatedly that returning a 404 for a page that genuinely no longer exists is normal and expected, and a “Not found (404)” report with many entries is not a penalty to panic over. Crawling those 404s is not a sign of a problem with the rest of your site.

The real cost is opportunity, not punishment. A backlink pointing at a 404 delivers no ranking equity, the link’s value is stranded at a dead end. That is the lever worth pulling: not making the errors disappear from a report, but recovering the equity that the valuable dead URLs are wasting.

Diagnose: find the URLs worth recovering

Start in Search Console. The Page Indexing report’s “Not found (404)” detail view lists the dead URLs Google is crawling, and grouping them by pattern (a legacy date-based path, a discontinued product directory, a parameter signature) tells you whether you are dealing with old content, internal mistakes, or spam.

Then find which dead URLs are worth a redirect. A backlink tool (Ahrefs, Semrush, or similar) can report broken backlinks or “best by links” filtered to your 404s, showing which deleted URLs still have referring domains. That referring-domain count is your triage key: prioritize the dead URLs with real external links and ignore the ones with none. Finally, run a site crawler to find internal links still pointing at deleted URLs, since those are entirely within your control to fix.

The fix hierarchy

Work in order of value. First, 301-redirect the deleted URLs that have genuine backlinks to the most relevant live page, so the equity from those links consolidates onto a page that exists. Redirect to a true topical equivalent or the closest relevant page, not the homepage, which Google can treat as a soft 404.

Second, fix internal links that point at dead URLs by repointing them to live pages, removing your own site as a source of the dead-URL crawling.

Third, for content that is intentionally and permanently gone with nothing to redirect to and no backlinks worth keeping, return 410 (Gone) instead of 404. Google treats 410 as a stronger, more definite “this is permanently gone” signal than 404 and tends to drop those URLs faster, since a 404 leaves open the possibility the page is temporarily missing. In practice the Search Console report may still label removed URLs as “Not found (404)” regardless, so judge the 410 by the faster drop-off, not by the report label.

A note on the Removals tool: it temporarily hides a URL from search results for roughly six months and does not stop Google from crawling it. Use it for urgent suppression of sensitive content, never as a way to make Google quit crawling, that is what 410 and a clean URL inventory do.

Legacy patterns and spam

When a permalink structure changed without redirects, for example moving from /2015/03/title to /title, Google ends up crawling both the old and new patterns indefinitely. Rather than mapping each old URL individually, a single wildcard redirect rule can catch the entire legacy pattern and send it to the current structure, clearing a whole class of dead URLs at once.

Finally, ignore pure spam and probe URLs, the random paths and exploit attempts that show up in the 404 report. They have no backlinks worth recovering and no relationship to your content, so leave them returning 404 and spend no effort on them.

Why 410 drops faster than 404

The practical difference between 404 and 410 is how Google interprets the likelihood that the page returns. A 404 means “not found,” which is ambiguous: the page could be permanently gone, or the server could have hiccuped, or a deploy could have temporarily broken a route. Because of that ambiguity Google keeps the URL around and rechecks it periodically, which is why a 404 tends to be revisited more often and lingers longer in the crawl queue.

A 410 means “gone” and carries an explicit assertion of permanence, so Google treats it as stronger evidence that the URL should be dropped and generally stops crawling it sooner. Keep the size of that advantage in proportion, though: Google’s own public position is that the practical difference is small, often a matter of a couple of recrawl cycles rather than a dramatic gap, so 410 is a minor optimization on cleanup speed, not a lever that changes rankings or recovers anything.

That makes 410 the right code only for content you are certain is permanently removed and that has no backlinks worth preserving, because 410, like 404, passes no equity. If a dead URL has valuable referring domains, a 410 throws that equity away just as a 404 does; the recovery move there is a 301 to a relevant live page.

So the decision is a two-step triage: if the URL has backlinks worth keeping, redirect it; if it does not and the content is permanently gone, return 410 to speed Google’s giving up; if it is spam or a probe, leave it on a plain 404 and ignore it. One caveat on verification: Google’s report may still display removed URLs under the “Not found (404)” label even when the server returns 410, so judge the effect by how quickly the URL stops being crawled, not by the report’s wording.

Frequently Asked Questions

Do all these 404 errors in Search Console hurt my rankings?

No. Google has stated that returning 404s for pages that no longer exist is normal and not a ranking penalty, and crawling them is not a problem signal. The thing worth acting on is link equity: a backlink pointing at a 404 wastes that link’s value, so redirect the dead URLs that have real backlinks and ignore the rest.

What is the difference between using 404 and 410 for a deleted page?

A 404 says “not found,” which Google may read as possibly temporary, so it keeps checking. A 410 says “gone,” a stronger permanent signal that tends to make Google drop the URL faster. Use 410 for content you are sure is permanently removed and has no backlinks worth keeping; use a 301 redirect instead when the dead URL has valuable backlinks.

Sources

Google Search Central, “Page Indexing report” (Not found 404 status): https://support.google.com/webmasters/answer/7440203
Google Search Central, “Removals and SafeSearch reports tool”: https://support.google.com/webmasters/answer/9689846

Related posts:

  1. Redirects Are Working But Traffic Still Dropped A redirect that verifies clean in a crawler proves one thing only: the hop resolves with a 301 and lands on a live page. It does not prove that the......
  2. Why Your Keyword Rankings Differ Between Countries on the Same Language Google localizes results by the searcher's country even when the query string and the language are identical, so the same English page can sit at position 3 in the United......
  3. Google Demoted Your Page for a Schema Error You Can’t Find Your schema passes every validator, the Rich Results Test reports no errors, and yet the rich results are gone and there is no manual action in Search Console to appeal.......
  4. How to Do SEO for Seasonal Businesses Seasonal SEO is counter-cyclical: the work that wins a December peak happens in March, because Google needs months to discover, crawl, index, and develop enough trust in new content for......
  5. How to Do SEO for a Multi-Language Site Without Hreflang Disasters When a German page outranks your French page in France, or Google serves the wrong-language version in search results, the cause is almost never Google ignoring hreflang on principle. It......
  6. Google Keeps Choosing the Wrong Canonical A canonical tag is a hint, not a directive, and Google says so directly: it takes your declared canonical into consideration, but it can and does select a different URL......
  7. Original Research as an SEO Moat Original research is defensible for one reason: the asset is data your competitors cannot replicate. Informational content gets reverse-engineered within weeks, because anyone can read the same sources and rewrite......
  8. Your A/B Testing Tool Is Showing Googlebot the Control Variant The conversion winner you just shipped may be content Google never evaluated. Most client-side experimentation tools exclude bots from tests by default, which means Googlebot spent the entire experiment seeing......
  9. Google Is Treating Your Tabs as Hidden Content The flat claim that Google "deprioritizes" content hidden in tabs and accordions is a contested myth, and shipping it as settled fact is the most common error in this corner......
  10. Why Your CDN Is Serving Different Content to Googlebot If Googlebot is indexing a version of your pages that no longer exists for users, your CDN is almost certainly serving the crawler a stale or differently-processed response. This usually......
  11. Your Bot Detection Is Blocking Googlebot and You Don’t Know It Aggressive bot protection can challenge or block legitimate crawlers when the rules are ordered so that a strict challenge fires before the verified-bot allowance does. That is why a sudden,......
  12. Google Is Indexing Your Search Results Pages Internal site-search results pages get indexed for a simple reason: they are crawlable and discoverable like any other URL. A search box that submits via a GET request turns every......
  13. Why Your Accordion Content Doesn’t Rank for Long-Tail Queries Your collapsed accordion content is almost certainly indexed. It sits in the rendered DOM, Googlebot reads it, and you can confirm it is there. The problem is weight, not visibility......
  14. SEO Process Documentation: Playbooks, SOPs, and Knowledge Management Documentation earns its keep only when it lowers the cost of doing the work correctly below the cost of doing it from memory. Most SEO teams get this backwards: they......
  15. SSR vs CSR vs ISR: Rendering Strategy Selection for SEO Rendering choice decides one thing that matters most for SEO: whether your content is in the HTML when Google first crawls the page. Client-side rendering (CSR) defers content behind a......