Part of the E-commerce SEO series. For the full picture, start with E-commerce technical SEO: the complete guide for online stores.
Filters are one of the best things about shopping online and one of the worst things for crawling. A category with eight filters, each with ten values, can produce more URLs than the rest of the store combined, and crawlers can’t tell which combinations matter until they’ve fetched them.
This guide explains how faceted navigation creates crawl problems, what Google recommends, and how to decide which filtered pages are worth indexing.
What is faceted navigation?
Faceted navigation is the set of filters that narrow a list of products: size, colour, brand, price, rating, availability, and sort order. On most platforms each choice changes the URL:
/shoes/
/shoes/?color=red
/shoes/?color=red&size=9
/shoes/?size=9&color=red&sort=price
/shoes/?color=red&size=9&sort=price&page=3
Five filters with a handful of values each, in any order, combined with sort orders and pages, multiply into thousands of URLs per category, most showing nearly the same products.
Why do filters cause SEO problems?
Google’s guide to faceted navigation names the core issue: faceted URLs “can generate infinite URL spaces”, and because the URLs “seem to be novel and crawlers can’t determine whether the URLs are going to be useful without crawling first, the crawlers will typically access a very large number of faceted navigation URLs.”
The consequences:
- Overcrawling. Crawl activity goes to filter combinations instead of products. On large stores this delays discovery of new products and price changes; see crawl budget.
- Duplicate and thin pages. Many combinations show the same products, or none.
- Diluted signals. Internal links spread across thousands of URLs instead of the categories that should rank.
How does Google recommend handling filters?
Keep filter URLs out of the crawl
Google’s preferred approach for filters that don’t need to be indexed is robots.txt. Its example disallows filter parameters while allowing a specific URL:
user-agent: Googlebot
disallow: /*?*products=
disallow: /*?*color=
disallow: /*?*size=
allow: /*?products=all$
Two related options from the same guide:
- URL fragments: “If your filtering mechanism is based on URL fragments, it will have no impact on crawling (positive or negative)”, because Google ignores fragments.
rel="canonical"andrel="nofollow"are “generally less effective in the long term” than keeping the URLs out of the crawl.
Remember what robots.txt does and doesn’t do: it stops crawling, not indexing, and a blocked URL linked from elsewhere can still be indexed without content. Don’t add noindex to pages you’ve blocked; Google can’t see it. See robots.txt for SEO and noindex vs robots.txt.
If filtered pages should be crawled
When some filtered URLs should be crawled and indexed, Google’s guide recommends making them predictable:
- “Use the industry standard URL parameter separator ‘&’.”
- Keep filters in a consistent order, so the same combination always produces the same URL.
- “Return an HTTP 404 status code when a filter combination doesn’t return results.”
Which filtered pages are worth indexing?
Most aren’t. A filtered page deserves to be indexed when:
- People search for it. “Red running shoes” or “Nike trail shoes” have real search demand; “size 9, under $80, sorted by newest” doesn’t. Check the queries in Search Console and keyword data.
- It shows a distinct, useful set of products, not the same items as its parent.
- You’ll treat it like a category: a clean URL, a unique title, H1 and short intro, and links to it from the parent category.
Everything else stays out of the crawl. A common pattern is to turn the valuable combinations into real subcategory pages with their own paths (/shoes/trail/nike/), and keep the generic filter parameters disallowed.
How do you find a facet problem?
In a crawl. Crawl the store from the homepage with robots.txt respected. If the crawl fills up with parameter URLs, your current rules let crawlers into the facets. In Crawlens:
- Search the URLs table for
?or a parameter name to count filter URLs. - Duplicate content, Near-duplicate content and Duplicate page titles cluster around filter combinations.
- Thin content often flags empty filter results.
- Run a second crawl with the filter patterns in Exclude URLs matching: the difference in URL count shows how much of the crawl the facets take.
- Crawl with the Googlebot Smartphone user agent to see what Googlebot’s robots.txt group allows.
In server logs. Logs show whether Googlebot actually spends its time on filter URLs. Crawlens’s log analysis shows Where Googlebot spends its time by directory and Budget on non-indexable URLs. See log file analysis.
In Search Console. The Page indexing report often lists filter URLs under Crawled - currently not indexed or Duplicate without user-selected canonical. See crawled - currently not indexed.
Checklist
- Filter, sort and view parameters identified for every category template
- Generic filter parameters disallowed in robots.txt
- No
noindexon URLs that robots.txt blocks - Valuable combinations turned into linked, indexable pages with unique titles and H1s
- Parameters use
?key=valueand&, in a consistent order - Empty filter combinations return 404
- A crawl from the homepage isn’t dominated by parameter URLs
- Server logs show Googlebot’s time going to products and categories, not filters
Frequently asked questions
What is faceted navigation?
Faceted navigation is the set of filters on category and search pages, such as size, colour, brand, price range and sort order, that narrow down the products shown. When each filter changes the URL, combinations of filters can create a very large number of URLs.
Why is faceted navigation bad for SEO?
It isn't bad for shoppers. The problem is crawlable filter URLs: Google says they can generate infinite URL spaces, so crawlers spend time on near-duplicate pages, and new or updated products may be discovered more slowly.
Should I use canonical tags for filter pages?
Google says rel=canonical and rel=nofollow are generally less effective in the long term for faceted navigation than keeping filter URLs out of the crawl. A canonical still requires Google to crawl each URL to see it.
Which filtered pages should be indexed?
Only those that match real searches and offer a distinct, useful set of products, such as a brand within a category or a major colour. Give them a clean URL, a unique title and heading, and link to them from the category page.
What should a filter page return when there are no products?
Google recommends returning an HTTP 404 status code when a filter combination doesn't return results, rather than an empty page with a 200.