Skip to main content
Faceted navigation is filtering and sorting on collection pages — by size, colour, price, brand. Each combination generates a URL, and combinations multiply. Six filters with five options each produce more URL combinations than most stores have products. Left crawlable, this consumes crawl budget that should be spent on products.

​
What Shopify generates

Sort parameters are pure duplication — the same products in a different order, with no unique value at all.

​
The decision, per facet

The principle: if people search for it, it deserves a page. If they do not, it should not be crawlable.

​
How to control it

robots.txt to prevent crawling. This is the effective lever, because it stops the crawl before it happens. Shopify’s default robots.txt already disallows sort URLs (/collections/*sort_by*), tag combinations (/collections/*+*) and multi-filter combinations (/collections/*filter*&*filter*). What stays crawlable by default is a single filter on a collection, such as ?filter.v.option.size=42. If those single-filter URLs are eating crawl budget, add a robots.txt.liquid template to your theme — built with Liquid, it keeps Shopify’s default rules — and add one rule for the * user agent:
Follow Shopify’s guide to editing robots.txt.liquid so the defaults stay in place. This is a manual theme edit: the Technical Audit doesn’t propose new Disallow rules — it flags rules that block Google or AI crawlers. Canonical tags on filtered pages pointing to the unfiltered collection. Useful, but weaker — the page still gets crawled, so budget is still spent. noindex removes pages from the index but requires them to be crawled first. It does not solve a crawl budget problem.
Do not combine robots.txt blocking with noindex on the same URLs. A blocked page cannot be crawled, so its noindex is never read.
Avoid linking to filtered URLs internally. Internal links are the primary discovery path. If filtered URLs are not linked, they are largely not found.

​
Promote high-demand facets properly

Where a filter combination has real search demand, do not rely on the parameter URL. Create a genuine collection with its own title, description, and content. See collection page optimisation.

​
Diagnosing a problem

Look for:
  • Search Console showing far more discovered URLs than you have products
  • Large numbers of “Discovered — currently not indexed”
  • Crawl stats dominated by parameter URLs
  • Deep products not being indexed while filter URLs are crawled repeatedly