Skip to content
Searchpedia SEO field notes Callum Bennett Callum

Site ops

How to Do a Technical Site Audit

I learned the hard way that a technical site audit needs Search Console data before you touch a crawler, or you waste hours on phantom issues.

Beginner3 min readUpdated 2026-07-27Notes by Callum Bennett

Start here

  • Start with Google Search Console's coverage report before running any crawler to get Google's actual index view.
  • Map your site by page type (product, blog, category) so you can spot issues per template, not globally.
  • Audit sitemaps for completeness against Google's indexed URLs and site architecture.
  • Check robots.txt to confirm it blocks only non-essential pages, not important content.
  • Compare index status across Search Console, site: searches, and your expected indexable set to find gaps.

Plain-English take

A technical site audit is a health check for your website's relationship with search engines. You are answering three questions: Can Google find my pages? Can it read them? Can it decide which ones to show in results? If any of those break, your content might as well be invisible. The audit is just a systematic way to find the break points.

Start with Google Search Console and a crawl tool because audits usually need both search-engine data and crawler data to identify issues accurately. I pull the coverage report first. That shows me how Google sees my index status — which pages are valid, which have errors, which are excluded. Then I run Screaming Frog or Sitebulb to get a raw crawl of the site. The crawler catches things like missing meta descriptions, broken links, and duplicate titles, but it does not tell you what Google actually indexed.

A good audit begins by mapping the site into page types or templates. I separate product pages, blog posts, category pages, and landing pages. That way I can spot issues per section. A problem that affects every product page is higher priority than a one-off error. Without this mapping, you might fix twenty individual 404s without realising they all stem from a single broken template.

Core audit areas commonly include indexability, rendering and JavaScript, duplicate content, pagination, URL structure, schema, mobile, and page speed. But I do not audit everything every time. I prioritise based on what the site needs. If the site is a small blog, I skip pagination complexity. If it is an ecommerce site with faceted navigation, I focus on [duplicate content](/duplicate-content/) and crawl budget.

When it actually matters

A full technical audit is not something you do monthly for a static brochure site. It matters most at specific moments.

After a site migration — new domain, CMS change, or URL structure overhaul — you need to confirm everything moved cleanly. I learned this after migrating a client from WordPress to a custom React site. The redirects looked fine at first, but I missed a handful of orphaned pages that never got 301s. Traffic dropped 20% across those pages. A proper audit after migration catches that.

Before a major content launch — if you are adding hundreds of pages, you want the technical foundation solid first. I once launched 200 product pages without checking the [canonical tags](/canonical-tag/). Google indexed them all with conflicting canonicals, and the whole set struggled for months. An audit beforehand would have shown the configuration error in the template.

When traffic drops without explanation — a sudden ranking loss of 30% or more. Do not assume it is algorithm. Check crawl errors, index coverage, and server responses. I have seen cases where a misconfigured [robots.txt](/robots-txt/) blocked an entire section for weeks. The traffic only recovered after I fixed the directive and requested re-crawl.

Also before migrating to HTTPS, changing URL structures, or switching to a new hosting provider. Each of those introduces risk of broken links, incorrect redirects, or changes in crawl behaviour. An audit gives you a baseline to compare against after the change.

What I got wrong

Relying only on a crawler — I used to think Screaming Frog was enough. But Google Search Console shows you what Google actually sees, which can differ wildly from a crawler's view. The crawler does not execute JavaScript unless you configure it to, and even then it is not Google's renderer. I missed JavaScript-rendered links for months because my crawler did not process them. Now I always check the [JavaScript SEO](/javascript-seo/) aspect by comparing Google's rendered page to the source HTML.

Auditing everything at once — early on, I would dump every issue into a spreadsheet. I ended up overwhelmed and rarely finished. Now I group by impact and effort. I fix things that block indexation first: soft 404s, noindex on important pages, blocked resources in robots.txt. Then I move to performance and structure. The [technical SEO](/technical-seo/) audit should produce a prioritised list, not a firehose.

Assuming sitemaps are always correct — I used to submit a sitemap and forget it. Then I found pages in the sitemap that returned 404s and important pages missing. Now I validate every URL in the sitemap against the live site and check it matches the [sitemap](/sitemap/) specifications. Also, I compare the sitemap URLs to Google's indexed URLs to spot gaps.

Underestimating mobile differences — I used to run an audit on desktop only. But mobile crawl errors can differ. Google's mobile-first indexing means you need to check mobile coverage reports separately. I now include [mobile SEO](/mobile-seo/) specific checks.

Next step

Quick answers

How long should a technical site audit take?

For a small site under 500 pages, I spend about half a day crawling, analysing, and writing up findings. For larger sites with thousands of pages or complex JavaScript, expect two to three days. The crawl itself takes minutes; the analysis and prioritisation take the bulk of the time.

What is the first thing I should fix in an audit?

Fix anything that prevents Google from indexing important pages. That means removing accidental noindex tags, fixing soft 404s, and ensuring robots.txt does not block key sections. After that, address duplicate content and canonical issues because they directly affect which URL shows in results.

Do I need a paid tool to do a proper audit?

No. Google Search Console, a free crawler like Screaming Frog (free for up to 500 URLs), and a bit of manual checking cover the essentials. Paid tools like Sitebulb or Lumar add convenience and reporting, but the core analysis is the same. I started with free tools and only upgraded when I needed speed.

Sources

Primary documentation is linked directly. Anything commercial is marked nofollow.

Notes from Callum Bennett.