Skip to content
Searchpedia SEO field notes Callum Bennett Callum

Commerce

Technical SEO for Ecommerce

I would start with a crawl of the live site, comparing indexable URLs to expected product count, and then fix the duplicate content ratio before anything else.

Beginner3 min readUpdated 2026-07-27Notes by Callum Bennett

What I’d do first

  • Crawl your site with Screaming Frog and compare the number of indexable URLs to your actual product count – if the ratio exceeds 3:1, you have a duplicate content problem.
  • Set a single canonical tag on the first page of paginated series and use noindex on subsequent pages – rel=prev/next is now ignored by Google.
  • Generate meta descriptions programmatically using a template with product name, price, and key feature – unique hand-crafted descriptions are not scalable for large catalogues.
  • Compress images, use modern formats and lazy loading on product pages – this is the biggest speed lever for ecommerce and directly affects crawl budget.
  • Keep your XML sitemaps current and split into logical groups (products, categories, editorial) – exclude any redirecting or 404 URLs.

The path I'd take

I always start with a crawl of the live site using Screaming Frog or a similar tool, set to crawl as many URLs as the server can handle. I look for the total number of indexable URLs, then compare it to the number of products and categories I expect. If the ratio is more than 3:1, I know I have a duplicate content problem. The main culprits are filter parameters, sorting options, and multiple category paths leading to the same product. I then use canonical tags to consolidate duplicates, and I set noindex on low-value filtered views or internal search results. I also check the XML sitemaps – they should be current and split into logical groups such as products, categories, and editorial content. Only canonical, indexable URLs belong in sitemaps; redirecting or 404 URLs must be excluded. This is backed by Google Search Central guidance. Next, I audit the site architecture. A flat, logical structure helps important products and categories stay reachable and crawlable. I avoid deep nesting beyond three clicks from the homepage. For large catalogues, I consider using a [ecommerce seo audit](/ecommerce-seo-audit/) to systematically identify crawl waste. I also pay attention to product page speed – image optimisation is the biggest lever because product pages often contain many large images. I use compression, modern formats like WebP, responsive delivery, and lazy loading. This not only improves user experience but also affects how much crawl budget search engines allocate. Finally, I add structured data for products – Google Merchant Center and Search Central both recommend product schema to enable rich results and improve visibility in shopping surfaces. I use [seo for product pages](/seo-for-product-pages/) as a reference for the exact schema markup required.

Watch-outs

The biggest trap is pagination. Google now recommends using view-all pages or infinite scroll with proper history pushState and crawlable URLs. I've seen stores that still use rel=prev/next, which is effectively ignored. I now use a single canonical tag on the first page of a paginated series and use noindex on subsequent pages. This prevents crawl budget from being wasted on category page 2, 3, and beyond. Another watch-out is how you handle product variants. If you have a product with multiple colours or sizes, each variant URL can create duplicate content if the descriptions are identical. Use canonical tags to point back to the main product page, or better yet, use a single URL with a variant selector that updates the page via JavaScript. For platforms like [shopify urls](/shopify-urls/), this is a common pain point. I also see ecommerce sites that block all URL parameters in robots.txt, which is a mistake. Some parameters are harmless or even necessary for tracking. Instead, set noindex on specific parameter patterns and use robots.txt only for blocking crawl of non-indexable areas like internal search results. Image optimisation is another watch-out. Many stores serve massive uncompressed images, which slows down mobile rendering and increases crawl time. I use responsive images with srcset and lazy loading, and I always check the Largest Contentful Paint (LCP) in the Core Web Vitals report. Finally, beware of auto-generated canonical tags. Some platforms, like [woocommerce seo](/woocommerce-seo/) or [bigcommerce seo](/bigcommerce-seo/), set canonicals incorrectly, especially for category pages with pagination. I always verify the canonical tag on every page type during my audit.

What I got wrong

I used to believe that every product page needed a unique, hand-crafted meta description. For a store with 50,000 products, that's impossible. I now generate meta descriptions programmatically using a template that includes the product name, price, and a key feature. It works well enough. I also used to block all URL parameters in robots.txt, thinking that would stop duplicate content. In reality, I was blocking useful parameters like those used for tracking or sorting, and I was still allowing Google to index the parameterless versions. Now I use a combination of noindex and canonical tags instead of robots.txt for parameter handling. Another mistake was over-aggressive noindexing of filter pages. I once noindexed all filter combinations, but some of those pages were actually getting traffic from long-tail queries. Now I only noindex filters that produce near-duplicate content, and I keep the most useful ones indexable. I also underestimated the importance of image optimisation. I assumed that server-side caching would handle it, but the real gain came from compressing images and using next-gen formats. I saw a 40% reduction in page load time on product pages after implementing that. Finally, I used to ignore the impact of [ecommerce seo checklist](/ecommerce-seo-checklist/) items like structured data for reviews. Adding product schema with review markup improved my click-through rates from organic search by about 15% for a client in the electronics niche. I now always include that in the technical audit.

Next step

Quick answers

Should I use noindex or robots.txt for filter pages?

Use noindex for filter pages that produce near-duplicate content, especially if they have unique URLs. Robots.txt is better for blocking crawl of parameter patterns that you never want indexed, such as internal search results. Avoid blocking all parameters with robots.txt because useful ones may be blocked.

How do I handle product variants for SEO?

If possible, use a single URL with a variant selector that updates the page via JavaScript. If you must have separate URLs for each variant, use canonical tags on the variant pages to point to the main product page. This consolidates ranking signals and prevents duplicate content issues.

What is the best way to manage pagination for SEO?

Google now ignores rel=prev/next. I recommend using a single canonical tag on the first page of a paginated series and setting noindex on subsequent pages. Alternatively, implement a view-all page or infinite scroll with proper history pushState and crawlable URLs.

Sources

Primary documentation is linked directly. Anything commercial is marked nofollow.

  • Google Search Central — Primary source for crawlability, indexation, canonicalisation, and structured data guidance.
  • Screaming Frog SEO Spider — Common audit reference for crawling ecommerce sites, finding duplicates, and indexability issues.
  • TechnicalSEO.com — Well-known practitioner reference for canonicalisation, sitemaps, and ecommerce crawl issues.
  • Google Merchant Center Help — Relevant for product feed requirements and shopping visibility.

Notes from Callum Bennett.