What your Search Console 'Crawled - currently not indexed' pages say about your site structure

If you've ever poked around in Google Search Console—and let's be honest, it's not exactly a thrilling Saturday night activity—you may have stumbled across a category called "Crawled - currently not indexed". It sounds ominous, doesn't it? Like Google's had a look at your pages, had a good think, and decided they're simply not worth the bother.

The good news is that Google isn't being deliberately rude. The less good news is that these pages are telling you something quite important about your website, and you're probably ignoring it. When Google crawls a page but doesn't index it, it's essentially saying "I've seen this, but I don't think anyone searching for anything would actually want to land here." Which is useful feedback, really, if you can be bothered to listen.

What does 'Crawled - currently not indexed' actually mean?

Let's start with the basics. Google's crawler (they call it Googlebot, which is charmingly literal) visits your page. It reads the content, looks at the structure, checks the links, does its thing. But then, instead of adding that page to its index—the giant filing cabinet of web pages it uses to answer search queries—it just... doesn't. It files your page under "seen it, not interested."

This is different from pages that aren't crawled at all, or pages that are blocked, or pages with errors. Google successfully crawled these pages. It just chose not to index them. That's the key bit.

Why Google might not index pages it's crawled

There are several reasons Google might do this, and most of them boil down to "you've created pages that don't really need to exist." Here are the main culprits:

Duplicate or near-duplicate content

This is probably the biggest one. If you've got multiple pages that say essentially the same thing, Google will pick what it thinks is the best version and ignore the rest. It's not going to fill up its index with seventeen variations of the same content just because you've structured your site in a way that creates them.

Common examples include product pages with only colour or size differences, blog posts tagged and categorised in ways that create multiple URLs for the same content, or paginated archives that don't use proper pagination signals. Google sees all these similar pages and thinks "right, I'll just index one of these and be done with it."

Thin or low-quality content

Pages with barely any content on them often fall into this category. A category page with just three products and no descriptive text. A tag page with two blog posts. A location page that's just a postcode and a map. Google's crawled it, seen there's not much there, and decided it's not worth indexing because nobody's going to find it useful in search results.

The thing is, these pages might be perfectly fine for navigation purposes. You might need that category page for your site structure. But Google doesn't need to include it in its search results, and it's telling you so.

Pages created by faceted navigation

If you've got filters on your site—sort by price, filter by colour, show only items in stock—you might be generating hundreds or thousands of URL combinations that create essentially the same page with minor variations. Google crawls a few of these, realises what's happening, and stops indexing them.

This is actually quite sensible of Google. Nobody's searching for "women's shoes under £50 sorted by price descending in blue excluding out of stock items." They're searching for "women's blue shoes" and they'll use your filters once they land on your site.

Internal linking that elevates unimportant pages

Sometimes your site structure inadvertently tells Google that certain pages are important when they're really not. If you've got pagination links in your footer, or overly complex breadcrumb trails, or "related posts" widgets that create hundreds of links to tangential content, you might be asking Google to crawl pages that don't deserve the attention.

Google follows these links because that's its job, but then it looks at the destination and thinks "this isn't actually important, is it?" So it doesn't index them.

What patterns in these pages tell you

Here's where it gets interesting. If you actually look at which pages are in the "Crawled - currently not indexed" bucket, you'll start to see patterns. And those patterns tell you quite a lot about your site structure.

If they're all tag pages: your tagging system is probably creating more pages than you need. If they're all old blog posts: your older content might be thin or outdated and needs refreshing or removing. If they're all product variations: you need to think about how you're structuring similar products and whether they need separate pages.

It's like having a very blunt consultant come in and point at parts of your website saying "nobody needs this." Except it's free and you just have to look at a spreadsheet.

What to actually do about it

First, don't panic. Having pages in this category isn't a catastrophe. It doesn't mean your entire site is rubbish or that Google hates you. It just means you've got some pages that aren't pulling their weight.

Start by exporting the list from Search Console and having a proper look at it. Group the URLs by type. Are they all from a particular section of your site? Are they a particular template or content type? Once you know what you're dealing with, you can decide what to do.

For genuinely duplicate or thin pages, you might want to add canonical tags pointing to the version you actually want indexed. For pages created by filters or pagination, you probably want to noindex them or use parameter handling. For old or outdated content, you might need to either update it properly or just delete it.

And sometimes—this is important—you might decide to do absolutely nothing. If these pages serve a purpose for your site navigation but don't need to be in Google's index, that's fine. Not every page needs to be indexed. Google's telling you it doesn't think they're useful for search, and you might agree.

The broader point about site structure

What's really useful about this Search Console category is that it holds up a mirror to your site structure and says "this is what you've actually built, regardless of what you thought you were building." It shows you where your navigation creates unnecessary URLs, where your content strategy is thin, and where you're accidentally telling search engines that unimportant pages are important.

Most people build websites from the inside out, thinking about what makes sense for their business or their content management system. Google crawls websites from the outside in, following links and making judgements about what matters. When those two perspectives don't match up, you get pages that are crawled but not indexed.

So treat it as feedback. Free feedback, at that. Google's essentially audited your site structure and told you where the problems are. You just have to be bothered to look at the data and do something about it.

Why we spend so much time thinking about this sort of thing

One of the main reasons we write about Google Search Console and Analytics is because understanding this stuff is literally part of our job. We don't just build websites for clients and wave them off into the sunset—we actually run a management service where we handle all their content, social media, and yes, the tedious-but-important business of monitoring what Google thinks of their website.

Our clients don't want to spend their evenings staring at Search Console reports trying to work out why their pagination is creating indexing issues. They've got businesses to run. That's rather the point of the management service: we understand how all this works so they don't have to. We build their websites properly in the first place, then we actually maintain them, create content that works, watch what the data's telling us, and fix things before they become problems.

It's the kind of ongoing support that means your website doesn't just sit there gathering dust and broken links whilst you're busy doing the actual work of running your business. Someone's actually paying attention, making improvements, and ensuring Google doesn't quietly decide half your pages aren't worth indexing.

If that sounds like the sort of thing you can't live without—and frankly, if you've read this far, it probably should—we'd be happy to chat about how it works and whether it makes sense for your situation.

How We Build Your New Website

Ongoing Website Management Prices

If you want to talk more then Get In Touch

And if you'd rather get to grips with this yourself, that's perfectly fine too. We've written quite a bit about search, analytics, and making sense of what Google's trying to tell you. Have a browse through our other articles, and you'll probably find something useful. Either way, we're here if you need us.

How Search Engines Learnt to Read Like Humans

The "Left on Read" Problem: Why Google Knows Your Page Exists But Won't Reply

Why your 'Average position' in Search Console doesn't mean what you think it means